Commit graph

84 commits

Author SHA1 Message Date
Ullrich Schäfer
b78c7b77e0
ci: validate openspec specs and changes
Adds a fast standalone job that runs `openspec validate --all --strict`
via npx, gating PRs on spec/change well-formedness.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 02:33:39 +02:00
Ullrich Schäfer
bdf9270f88
Update single PR comment across preview deploy + teardown
The deploy job was creating a fresh comment on every push, so PRs
accumulated a wall of preview-URL comments. Now we look up the existing
comment by an HTML marker (`<!-- cd-staging:preview -->`) and edit it
in place — both on push and on teardown. Comment also gains a link to
the GitHub Actions run that produced the preview, plus the head SHA.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-04 07:17:57 +02:00
Ullrich Schäfer
e93cfd2adb
Trigger cd-staging on its own config changes + workflow_dispatch
So a port bump or compose edit doesn't sit unapplied until the next
apps/ push, and so persistent staging can be redeployed manually
without forcing a no-op apps/ change.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 22:39:16 +02:00
Ullrich Schäfer
34711550fc
Move persistent staging off port 3100 to avoid Loki conflict
Loki binds 10.0.0.2:3100 on the vSwitch interface so the BRouter host can
ship logs in. Persistent staging was trying to publish 0.0.0.0:3100 which
conflicts because Linux refuses 0.0.0.0:<P> when any specific-interface
:<P> is already in use. Move staging journal to 3110, planner to 3111.

PR previews are unaffected — they're already on 3200+2N.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 22:38:38 +02:00
Ullrich Schäfer
25d21161c5
Fix staging port binding + diagnostic ps env file
Two bugs in the staging-environments rollout:

1. Staging containers published on 127.0.0.1 are unreachable from the
   production Caddy container, which connects via the Docker bridge IP
   (host.docker.internal:host-gateway resolves to the bridge, not
   loopback). Bind to 0.0.0.0 instead — Hetzner Cloud firewall blocks
   ports 3000+ from the public internet, so it stays internal-only.

2. The trailing 'docker compose ps' diagnostic in deploy-staging /
   deploy-preview was missing --env-file staging.env, so compose
   failed env interpolation and the job exited non-zero even when the
   actual deploy succeeded.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 22:30:05 +02:00
Ullrich Schäfer
c8a7a0b253
Add staging + PR-preview environments on the flagship
Implements the staging-environments OpenSpec change. Persistent staging at
staging.trails.cool / planner.staging.trails.cool deploys from main; PR
opens get a journal-only preview at pr-<N>.staging.trails.cool that shares
the persistent planner. cd-staging.yml builds tagged images, manages
per-PR Postgres databases and Caddyfile snippets, evicts the oldest
preview at the cap of 3, and tears everything down on PR close.
staging-cleanup.yml runs weekly to sweep orphaned previews.

DNS records (staging + *.staging A/AAAA) already applied to production
via tofu.

Caddy approach: per-PR Caddyfile snippets imported from /etc/caddy/sites/
and reloaded on each PR event — no wildcard / on-demand TLS, no router
service. Production compose gains a trails-shared network for the staging
project to reach Postgres, and host.docker.internal on Caddy so it can
reverse-proxy staging containers published on the host loopback.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 22:03:38 +02:00
Ullrich Schäfer
db3eeed60f
Apply wahoo-route-update: PUT on re-push instead of duplicate POST
Re-pushing an edited route to Wahoo now updates the existing remote
route via PUT against the stored remote_id instead of POSTing a new
copy. external_id drops the version suffix and identifies the logical
route. sync_pushes is keyed by (user, route, provider) and tracks
last_pushed_version for the "local newer" UI state.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 12:08:16 +02:00
Ullrich Schäfer
7e28e72c29 cd-apps: reload Caddy after deploy so Caddyfile changes take effect
cd-apps already scp's `infrastructure/Caddyfile` to the server
alongside docker-compose.yml — but the running Caddy container
doesn't auto-pick-up config changes. cd-infra is the workflow that
calls `caddy reload`, and it only triggers on `infrastructure/`
paths. So any Caddyfile edit shipped through cd-apps (e.g. the
`lb_try_duration` block in PR #329, deployed as part of an apps
push) sits on disk unapplied until the next cd-infra run.

This was real today: PR #329 added `lb_try_duration 30s` to silence
deploy-time 502s. PR #331 (a vite.config edit, apps path) merged
right after and triggered cd-apps, which copied the new Caddyfile
but didn't reload Caddy. The very next planner restart in that
deploy promptly produced three 502s — the exact thing #329 was
supposed to prevent. We applied the reload manually via SSH after
the fact.

This commit makes that reload happen on every cd-apps deploy. The
operation is idempotent (Caddy validates first, swaps live, no
downtime) so doing it even when Caddyfile is unchanged costs
nothing. `|| true` keeps the deploy from failing if Caddy is
unhealthy at deploy time.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 20:09:43 +02:00
Ullrich Schäfer
55c9154f05 Revert cd-apps annotation path: GRAFANA_SERVICE_TOKEN is in .env, not app.env
The token lives in `secrets.infra.env`, which `cd-infra.yml` merges
together with `secrets.app.env` into the server's
`/opt/trails-cool/.env`. The cd-apps workflow's own `app.env`
intentionally does NOT carry the token (apps don't need it at
runtime), so the original `grep ... .env` was correct. My earlier
edit in this branch swapped the path to `app.env` and would have
broken the annotation hook the moment it actually worked.

Restored `.env` and updated the inline comment to make the file
ownership explicit (cd-infra populates it; cd-apps reads it).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 11:53:55 +02:00
Ullrich Schäfer
5c4b6fd9af Stop the caddy-502-rate alert firing on every deploy
The journal/planner deploy in cd-apps.yml does `docker compose up -d
journal planner`, which stops the old container and starts the new
one — Caddy keeps forwarding requests during the ~10–30s gap and
returns 502s. The caddy-502-rate alert (threshold > 0 for 2m)
correctly trips, every time.

Two production changes plus a long-broken workflow detail:

- infrastructure/Caddyfile — add `lb_try_duration 30s` /
  `lb_try_interval 250ms` to the journal and planner reverse_proxy
  blocks. Caddy now holds and retries the upstream for up to 30s
  during a restart instead of 502'ing immediately. Real outages
  (upstream unreachable longer than 30s) still 502 and the alert
  still fires for those.
- infrastructure/grafana/provisioning/alerting/alerts.yml — add a
  comment documenting why caddy-502-rate stays at threshold > 0:
  with lb_try_duration in front of it, the alert no longer
  conflates "deploy in flight" with "real outage."
- .github/workflows/cd-apps.yml — fix a long-silent bug: the
  Grafana deploy-annotation step was reading
  GRAFANA_SERVICE_TOKEN from `.env`, but the secrets file we scp
  to /opt/trails-cool is named `app.env`. The token check failed
  silently and the curl was being skipped on every deploy.
  Switching to `app.env` so deploys actually annotate Grafana.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 11:51:13 +02:00
Ullrich Schäfer
f0ef989ac3 Rebuild the Journal home around a public feed and flagship marketing
The old home was an h1, subtitle, and two auth buttons — visitors
arriving at trails.cool had no idea what it was, and self-hosted
instances had nothing interesting to land on.

The new home is one layout that serves both audiences:

- Hero: a product-describing h1 ("Federated outdoor journal") +
  tagline. The site name already lives in the top banner and nav brand,
  so the h1 carries the pitch instead of triplicating "trails.cool".
- Auth CTAs: Register (blue) + Sign In (outlined) get primary weight.
  Planner demotes to a small "Or try the Planner without an account →"
  link below them — it's a nice escape hatch but not the goal.
- Marketing cards: on the flagship only, a 2x2 grid of emoji-icon cards
  for Planner / Journal / Federation / Ownership, matching the Planner
  home's visual weight. `IS_FLAGSHIP=true` toggles it. Self-hosters get
  a "Powered by trails.cool — about the project" link back to the
  flagship instead, so they don't have to write their own marketing.
- Public feed: reverse-chronological list of the 20 most recent public
  activities on the instance, with owner + distance + date. Empty state
  for fresh installs.

Plumbing:
- `listRecentPublicActivities(limit)` in activities.server.ts joins
  users so the feed can render "by <displayName>" in one query.
- `IS_FLAGSHIP` env wired through docker-compose.yml; cd-apps and
  cd-infra both write `IS_FLAGSHIP=true` alongside `DOMAIN=trails.cool`,
  so self-hosters (who deploy without these workflows) default to off.

Specs:
- `activity-feed`: new "Instance-wide public activity feed" requirement
  so the visibility contract is pinned (public in feed, unlisted +
  private are not).
- `journal-landing`: new small spec capturing the hero / marketing /
  feed layout contract and the `IS_FLAGSHIP` toggle, so future
  instance-branding or empty-state changes have a stable anchor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 20:00:23 +02:00
Ullrich Schäfer
bddc3d7620 Copy init-grafana-user.sql on every cd-infra deploy
The postgres init script grants read permissions to `grafana_reader` on
`public`, `planner`, `journal`, and `pgboss` schemas. It's mounted into
the postgres container via bind mount from
`/opt/trails-cool/postgres/init-grafana-user.sql`, and cd-infra re-runs
it with `psql -f` after postgres comes up.

But `cd-infra.yml` only scp's `infrastructure/postgres/queries.yml` to
the flagship — not `init-grafana-user.sql`. The file on the server was
the pre-pg-boss-upgrade March version with no pgboss grants. After
pg-boss v12 landed and (re)created the `pgboss` schema, the
grafana_reader lost schema-level USAGE, the "Background job failures"
alert query started erroring with `permission denied for schema pgboss`,
and the alert fired as "Alerting (Error)" for hours.

Fix: add the init SQL to the scp source list so changes to it actually
reach the flagship. I also manually re-granted on the live postgres to
clear the firing alert; the next cd-infra deploy will now pick up future
edits to the init file too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:47:18 +02:00
Ullrich Schäfer
2e5606458d Fix Planner/Journal healthchecks; auto-clean compose orphans
The compose healthcheck for both apps was `curl -sf http://localhost:.../health`,
but node:25-slim (Debian trixie-slim) ships neither curl nor wget, so
the check always reported the containers as `unhealthy` even while they
were happily serving 200s. Install curl in both Dockerfiles.

Separately, today's flagship BRouter cleanup (PR #297) left an orphan
`brouter` container behind because cd-infra's `docker compose up -d <list>`
doesn't pass `--remove-orphans`. I had to remove it by hand. Pass the
flag in both cd-infra and cd-apps so future service deletions clean up
on the next deploy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:28:04 +02:00
Ullrich Schäfer
6bae26de6e
Update brouter deploy scp to single ssh 2026-04-24 14:41:30 +02:00
Ullrich Schäfer
3c4c1eebd8
Fix BRouter first-deploy issues discovered during cutover
Three fixes after the first cd-brouter run on the dedicated host:

1. **BRouter 1.7.8 → 1.7.9** (`docker/brouter/Dockerfile`). Planet RD5
   segments on brouter.de are now version 11; 1.7.8's `lookups.dat`
   is v10, causing `lookup version mismatch (old rd5?)
   lookups.dat=10 E10_N45.rd5=11` on every route request.

2. **cd-brouter.yml: docker login to ghcr.io before pull**.
   ghcr.io/trails-cool/brouter is private, and the dedicated host's
   Docker daemon isn't logged in by default. Extract DEPLOY_GHCR_TOKEN
   from SOPS at runner side, pass to the SSH step via envs, and
   `docker login` before `docker compose pull`. Credential is
   `::add-mask::`-ed so it doesn't show in logs.

3. **Drop custom healthcheck** on the brouter service. The image
   strips wget/curl post-build, and /bin/sh in the base doesn't
   support /dev/tcp, so there's no in-image way to do an HTTP probe.
   Real health is observed via Caddy's upstream 502 behavior on
   outage and the Planner-side `brouter_request_duration_seconds`
   metric. caddy's `depends_on` drops from service_healthy to
   service_started.

End-to-end verified on the dedicated host after applying the compose
fix manually:
- Caddy enforces auth: 403 without header, proxies with.
- BRouter 1.7.9 will resolve the segment-version error once the image
  is rebuilt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 23:21:46 +02:00
Ullrich Schäfer
9e598fb6a1
BRouter host observability + docs
Lands sections 6 and 8 of relocate-brouter-to-dedicated-host on top
of the host compose in #291.

## Observability (section 6)

- **Prometheus**: new `brouter-cadvisor` job scraping
  `10.0.1.10:8080` over the vSwitch, labeled `host="brouter"`.
- **cAdvisor sidecar** on the dedicated host's compose
  (`--docker_only --whitelisted_container_labels=trails.cool.service`)
  so metrics only cover trails containers, never the operator's
  unrelated workloads on the shared box.
- **Promtail sidecar** on the dedicated host, Docker SD with relabel-
  drop on missing `trails.cool.service` label, pushing to flagship
  Loki at `http://10.0.0.2:3100/loki/api/v1/push`.
- **Flagship compose**: Loki now publishes port 3100 on the vSwitch
  IP only (10.0.0.2:3100) — Hetzner Cloud firewall blocks it from
  the public internet.
- **Grafana dashboard**: `brouter.json` — scrape up/down, request
  rate (from Planner-side `brouter_request_duration_seconds`),
  p50/p95/p99, container memory/CPU, Loki logs panel.
- **Alert**: `brouter-scrape-down` fires on
  `up{job="brouter-cadvisor"} < 1 for 2m`; `noDataState: Alerting`
  so a total scrape failure still pages.

Operator needs one UFW rule on the dedicated host for the cAdvisor
port — documented in `infrastructure/brouter-host/README.md`.

## Documentation (section 8)

- `CLAUDE.md` — hosts table + updated deployment table with SSH
  targets per workflow; BRouter host SSH is `-p 2232 trails@...`,
  different key.
- `docs/architecture.md` — Hosting section rewritten to cover both
  hosts, vSwitch boundary, and the observability-scoping rationale
  for the shared dedicated host.
- `docs/deployment.md` (new) — full operator runbook: host layout,
  first-time BRouter provisioning, SOPS rotation (including the
  macOS `SOPS_AGE_KEY_FILE` gotcha), cutover procedure with
  rollback, manual workflow triggers.

Task 8.4 (infrastructure/README.md) skipped: that file doesn't
exist and the ground is covered by brouter-host/README.md +
docs/deployment.md.

## Validation

- `docker compose config` on both the flagship and brouter-host
  compose files — both validate.
- `pnpm typecheck`, `pnpm lint`, `pnpm test` — all clean (full
  turbo cache hits; no code changes in this commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 23:10:45 +02:00
Ullrich Schäfer
c49047fd33
BRouter host compose + Planner auth + cd-brouter rewrite
Lands sections 3-5 of the relocate-brouter-to-dedicated-host change:
everything needed to run BRouter on the dedicated Hetzner Robot host
and have the Planner talk to it with the shared-secret header. Does
NOT flip the cutover — the flagship BRouter stays warm during soak.

## BRouter host compose (section 3)

New `infrastructure/brouter-host/` — a standalone compose project that
runs as the `trails` user on `ullrich.is`:

- `docker-compose.yml` — brouter + caddy sidecar. BRouter has no host
  port; caddy binds only to `10.0.1.10:17777` (vSwitch IP). Every
  service explicitly overrides the host's default Loki logging driver
  to `json-file` so logs don't leak to the operator's personal Loki.
- `Caddyfile` — single-purpose reverse proxy that requires
  `X-BRouter-Auth: ${BROUTER_AUTH_TOKEN}` on every request. `auto_https
  off` (vSwitch-only); default access log format omits request
  headers, so the token is never written to disk.
- `download-segments.sh` — crawls brouter.de, pulls planet-wide RD5
  tiles via `wget -N` (incremental). Idempotent, safe to cron.
- `README.md` — one-shot provisioning + token rotation + rollback
  notes.

`docker/brouter/Dockerfile` is patched to honor `JAVA_OPTS` (was
hardcoded `-Xmx1024M` in CMD). Default keeps the flagship's current
heap; compose on the dedicated host overrides to `-Xmx8g` for planet
scale on a 32 GB box.

## Planner shared-secret header (section 4)

`apps/planner/app/lib/brouter.ts`:

- Module-level guard: throws at startup in production if
  `BROUTER_AUTH_TOKEN` is unset.
- `authHeaders()` helper (reads env at call time, so tests can
  `vi.stubEnv` without module reset).
- Header attached on both `computeRoute` (per-segment) and
  `computeSegmentGpx`.

3 new unit tests cover header attachment + the no-token path.

`infrastructure/docker-compose.yml` passes `BROUTER_AUTH_TOKEN` to
the Planner service, and makes `BROUTER_URL` overridable via SOPS so
the cutover is a one-variable flip.

## cd-brouter workflow (section 5)

Rewritten to deploy to the dedicated host:

- SSH as `trails@${BROUTER_DEPLOY_HOST}` on port
  `${BROUTER_DEPLOY_SSH_PORT}` (2232) using
  `${BROUTER_DEPLOY_SSH_KEY}`.
- Decrypts SOPS, extracts ONLY `BROUTER_AUTH_TOKEN` into a `.env`
  file, scp'd alongside the compose project.
- `paths:` trigger now includes `infrastructure/brouter-host/**`.
- Segment download is NOT run here — first-time seed is a manual
  operator step (multi-hour). Routine re-runs are cron-able on the
  dedicated host.
- Grafana annotation step preserved (reaches flagship Grafana as
  before).

## What's NOT here

- `brouter:` service on the flagship is intentionally left in place
  (removed in section 7.5 after the 48 h soak window post-cutover).
- Observability (section 6) — Prometheus scrape + Loki shipping from
  the dedicated host — comes in a follow-up PR.
- Cutover itself (section 7) — flip `BROUTER_URL`, verify, remove the
  flagship brouter — is an operator action gated on first-time
  provisioning + smoke testing.

## Verification

`pnpm typecheck && pnpm lint && pnpm test` all clean; planner build
passes (the CI regression from #286 was fixed in #290).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 22:52:48 +02:00
Ullrich Schäfer
ec93d37470
Prefix dedupe commit message + set GH_TOKEN for push step
- Commit subject prefixed with "[github-actions]" so the audit trail
  is obvious at a glance in `git log`.
- GH_TOKEN exposed as an env var on the commit step so the PAT is
  also available to any `gh` invocations the step might grow, and the
  token plumbing is visible at the site where the push happens — not
  only up at the checkout step.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 21:06:38 +02:00
Ullrich Schäfer
da56eb2c19
Stop over-claiming the persist-credentials mechanism
The previous comment claimed `git push` reads the token via a git
credential helper wired into $RUNNER_TEMP. I don't actually know that's
how it works end to end — only that checkout stashes the token there
and that subsequent git ops in the same workspace end up authenticating
as the PAT. Describe just the observable contract, not an unverified
internal path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 20:58:26 +02:00
Ullrich Schäfer
5d9ffae38a
Correct comment about where checkout persists the PAT
Current actions/checkout stores the token in $RUNNER_TEMP and wires a
git credential helper, rather than writing the token into .git/config's
http.extraheader. The mechanism that makes `git push` pick up the PAT
is unchanged at the caller's level; the comment was just wrong about
where the secret lives on disk.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 20:56:54 +02:00
Ullrich Schäfer
32ea0f66f5
Make persist-credentials explicit in dedupe checkout
`actions/checkout` has `persist-credentials: true` as its default, but
that default is the load-bearing part of how the later `git push`
authenticates — it's what writes the token into .git/config's
http.extraheader. Making it explicit so a reader of the YAML can see
the mechanism without having to know the action's defaults.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 20:55:47 +02:00
Ullrich Schäfer
e505cfa2b9
Require DEPENDABOT_DEDUPE_TOKEN for the dedupe workflow
Previously the workflow fell back to GITHUB_TOKEN when the PAT wasn't
set — which was silently broken: the push succeeds but GitHub's
anti-workflow-loop safeguard means no new CI runs on the dedupe commit.
Reviewers see stale green CI and the dedupe itself isn't tested until
the next dependabot rebase.

Make the PAT a hard requirement: preflight check fails with a clear
message when the secret is missing, and the checkout step uses it
(which is what persists auth for the later `git push`). Intent is now
obvious from the YAML.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 20:53:57 +02:00
Ullrich Schäfer
f085b78fd7
Auto-dedupe lockfile on dependabot PRs
pnpm install after a dependabot bump keeps duplicate versions of
packages that happen to be pinned via overlapping semver ranges. For
singleton-ish libraries (i18next, react-i18next) that's a latent bug:
the server's module instance and the client's module instance each
hold their own state, and SSR output doesn't match CSR.

#272 ran into this with an i18next patch bump — the bumped version
stayed on some dep paths while other paths kept the old version. A
manual `pnpm dedupe` collapsed them and hydration worked again.

This workflow fires on every dependabot PR, runs `pnpm dedupe`, and
pushes the resulting lockfile update back to the PR branch so CI
runs against the deduped tree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 11:59:33 +02:00
Ullrich Schäfer
debaf769d5
Tighten dependabot ignore rules
- Skip major bumps on @types/jest + @types/node so the types stay
  aligned with the runtime (jest 29 with @types/jest 29, not 30).
- Skip react-native itself since it's pinned by the Expo SDK and
  should only move when we bump Expo. Community react-native-*
  libraries stay on the normal weekly cadence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 10:58:05 +02:00
Ullrich Schäfer
7c207a2a98 Add CI check for missing workspace packages in Dockerfiles
Adds a script that verifies every packages/*/package.json is listed in
all app Dockerfiles. Catches the recurring issue where adding a new
workspace package breaks production because the Dockerfile deps stage
doesn't know about it.

Also fixes two pre-existing missing packages in the planner Dockerfile
(api, map-core) caught by the new check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 12:15:25 +02:00
dependabot[bot]
5bd4866ca9
Bump pnpm/action-setup from 5 to 6
Bumps [pnpm/action-setup](https://github.com/pnpm/action-setup) from 5 to 6.
- [Release notes](https://github.com/pnpm/action-setup/releases)
- [Commits](https://github.com/pnpm/action-setup/compare/v5...v6)

---
updated-dependencies:
- dependency-name: pnpm/action-setup
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-12 19:51:20 +02:00
Ullrich Schäfer
3adf6fbfc4
Fix empty Caddy and pg_stat_statements panels in service-health dashboard
- Caddy Response Status Codes: caddy_http_responses_total doesn't exist,
  use caddy_http_response_duration_seconds_count which has the code label
- pg_stat_statements: enable extension in init script, add custom queries
  config for postgres-exporter, remove invalid datname filter from query
- Add postgres/queries.yml to cd-infra SCP sources

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 16:02:23 +02:00
Ullrich Schäfer
271dfa8b01
Add Promtail to cd-infra SCP sources and deploy service list
The previous PR added Promtail to docker-compose but the CD workflow
didn't copy the config file or include promtail in the service list.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 15:51:53 +02:00
Ullrich Schäfer
836e0a2b6a
Expand BRouter routing coverage to all of Europe
Add ~67 segment tiles covering W10-E40, N35-N70 (was 4 Germany-only
tiles). Total disk usage ~2.9 GB. Tiles that don't exist on the
BRouter CDN are silently skipped.

Already downloaded to the server and BRouter restarted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 22:48:11 +02:00
Ullrich Schäfer
87be96db67
Switch deploy annotations from BusyBox wget to curl
BusyBox wget in the Grafana Alpine container doesn't support
--post-data/--header. Grafana's image includes curl.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 20:49:40 +02:00
Ullrich Schäfer
c2b9032686
Fix deploy annotation reading token from wrong env file
GRAFANA_SERVICE_TOKEN is in secrets.infra.env, which the infra
workflow merges into .env on the server. The apps workflow was
reading from app.env (which only has app secrets).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 19:59:17 +02:00
Ullrich Schäfer
6c482238e8
Add Grafana deploy annotations to all CD workflows
Posts an annotation to Grafana after each successful deploy, tagged
with "deploy" and the service name. Annotations appear as vertical
markers on dashboards, making it easy to correlate deploys with
metric changes.

Uses GRAFANA_SERVICE_TOKEN from SOPS secrets — must be added via
sops infrastructure/secrets.infra.env. Gracefully skips if the
token is not configured.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 19:44:15 +02:00
copilot-swe-agent[bot]
2079b7e6de
Simplify copilot-instructions.md to point to CLAUDE.md instead of duplicating content
Agent-Logs-Url: https://github.com/trails-cool/trails/sessions/bc100497-d093-4f44-bf81-5fbd357af6b3

Co-authored-by: stigi <13815+stigi@users.noreply.github.com>
2026-03-29 12:41:30 +00:00
copilot-swe-agent[bot]
0ac1d4ca39
Add .github/copilot-instructions.md and update README for GitHub Copilot
Agent-Logs-Url: https://github.com/trails-cool/trails/sessions/d830c6a7-0796-4aeb-8169-d7a7718a6170

Co-authored-by: stigi <13815+stigi@users.noreply.github.com>
2026-03-29 12:30:59 +00:00
Ullrich Schäfer
d797564201
Skip Gitleaks scan for Dependabot PRs
Gitleaks requires a license key for org repos. Skip the step (not the
job) for dependabot[bot] so the Security Scan job still passes as a
required status check. pnpm audit still runs for all PRs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 10:40:27 +02:00
Ullrich Schäfer
424eecd274
Add PostgreSQL datasource to Grafana with read-only user
- Create grafana_reader role with SELECT-only access on all schemas
- Init script runs on postgres first boot (docker-entrypoint-initdb.d)
- Grafana PostgreSQL datasource provisioned with read-only credentials
- Enables SQL queries in dashboards (user count, routes, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:07:27 +01:00
Ullrich Schäfer
9a7ecd8911
Simplify restart-all: workflow_dispatch only
Remove commit message [restart-all] detection — doesn't work with
GitHub merge commits. Keep only the explicit workflow_dispatch input.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 19:15:22 +01:00
Ullrich Schäfer
424e692ee0
Add service health monitoring: postgres, node, cAdvisor exporters + dashboard
Exporters:
- postgres_exporter: DB connections, transactions, cache hit ratio, query stats
- node_exporter: host CPU, memory, disk, network
- cAdvisor: per-container CPU and memory usage

PostgreSQL:
- Enable pg_stat_statements for query-level performance tracking
- Track index scans vs sequential scans, cache hit ratio

Dashboard (service-health.json):
- DB: connections, size, transactions/s, slow queries, cache hit ratio, index usage
- Host: disk gauge, CPU, memory, network I/O, disk I/O
- BRouter: request latency p50/p95/p99, container CPU + memory
- All containers: CPU and memory comparison

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:50:42 +01:00
Ullrich Schäfer
318f0c93c1
Strip trailing newline from Sentry auth token
The token file may have a trailing newline that causes sentry-cli
API requests to fail with "API request failed".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:23:20 +01:00
Ullrich Schäfer
5d7b8c82e8
Fix Sentry build: disable telemetry, fix release version passing
- Disable Sentry Vite plugin telemetry (telemetry: false)
- Pass SENTRY_RELEASE as build-arg (not secret) since it's just the git sha
- Keep only SENTRY_AUTH_TOKEN as Docker secret

The release version was showing as *** because Docker secrets get masked
by GitHub Actions. Git sha is not sensitive — safe as a build arg.

The sourcemap warning from react-router is a known upstream issue (ships
without sourcemaps) and is harmless.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:18:00 +01:00
Ullrich Schäfer
5e200c8b62
Fix Sentry source map upload in Docker builds
- Approve @sentry/cli and esbuild build scripts so postinstall runs
- Use --mount=type=secret instead of ARG for SENTRY_AUTH_TOKEN
  (fixes Docker warning about secrets in build args)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:01:54 +01:00
Ullrich Schäfer
06916868fe
Fix cd-apps: extract SENTRY_AUTH_TOKEN instead of sourcing env file
The decrypted secrets.app.env contains values with angle brackets
(SMTP_FROM) that bash interprets as redirects. The build job only
needs SENTRY_AUTH_TOKEN, so extract it with grep instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:52:41 +01:00
Ullrich Schäfer
e880e3e9d9
Fix SCP path for decrypted secrets in CD workflows
Write decrypted env files to infrastructure/ instead of /tmp/ so
strip_components: 1 correctly places them on the server.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:43:30 +01:00
Ullrich Schäfer
9673496211
Add GitHub environments to CD workflows
- cd-apps: uses `production` environment for AGE_SECRET_KEY
- cd-infra: uses `infra` environment for AGE_SECRET_KEY

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:34:28 +01:00
Ullrich Schäfer
57094323d2
SOPS+age secrets, split CD workflows, GitHub OAuth for Grafana
Secrets:
- Add .sops.yaml with age encryption config
- Add encrypted secrets.app.env (app secrets) and secrets.infra.env (Grafana OAuth)
- CD decrypts at deploy time with AGE_SECRET_KEY — all other secrets
  move out of GitHub Actions into version-controlled encrypted files

Split CD:
- cd-apps.yml: triggered by apps/packages changes, builds Docker images, deploys apps
- cd-infra.yml: triggered by infrastructure/ changes, copies configs, restarts services
- Remove monolithic cd.yml

Grafana auth:
- GitHub OAuth (trails-cool org), disable login form
- Remove Caddy basic_auth block and all GRAFANA_* env vars/secrets

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:28:04 +01:00
Ullrich Schäfer
b01624b31a
Sync Grafana admin password on each deploy
GF_SECURITY_ADMIN_PASSWORD only works on first boot. Reset the admin
password via grafana-cli after each deploy to stay in sync with secrets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 14:53:54 +01:00
Ullrich Schäfer
ed33fcf9ae
Deploy monitoring configs (Prometheus, Loki, Grafana) via CD
Add prometheus.yml, loki-config.yml, and grafana provisioning/dashboards
to the SCP step so they're copied to the server on each deploy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 13:55:09 +01:00
Ullrich Schäfer
e105382c8f
Wire Grafana secrets through CD pipeline
Pass GRAFANA_USER, GRAFANA_PASSWORD, and GRAFANA_PASSWORD_HASH
from GitHub Actions secrets to the deploy script.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 08:39:25 +01:00
Ullrich Schäfer
ed4757382a
Use forked gitleaks-action with merge_group support
The upstream gitleaks/gitleaks-action@v2 errors on merge_group events.
Use our fork which includes https://github.com/gitleaks/gitleaks-action/pull/186.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 19:26:18 +01:00
Ullrich Schäfer
746b09276c
Replace automerge action with GitHub native merge queue
The automerge GitHub Action (pascalgn/automerge-action) is removed in
favor of GitHub's built-in merge queue, which is now enabled on the
main branch ruleset.

- Delete automerge.yml workflow
- Add merge_group trigger to CI so it runs on queue candidates
- Remove workflow_run trigger for Automerge from CD
- Update CLAUDE.md to document gh pr merge --merge-queue

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:40:11 +01:00