Every compose service used docker's default json-file driver with no
rotation, so a service logging in a loop grew unbounded — a 2.6 GB
container log contributed to a 2026-07-14 disk-full outage (/ at 100%,
Postgres down, apps 500ing, deploys failing at the SCP step).
Add an `x-logging` anchor (10 MB x 3 files = 30 MB max per container) and
apply it to all 11 services. Complements disk-maintenance.yml (which prunes
images but can't touch container logs). Takes effect when cd-infra recreates
the containers. Documented the disk-pressure runbook in docs/deployment.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the planner's Overpass proxy with a self-hosted POI index:
- map-core POI categories become structured tag selectors (single source of
truth) + osmium filter / classification helpers
- planner.pois PostGIS table (centroid points, GiST + category indexes)
- /api/pois serving route (session + same-origin + rate limit, 100 cap)
- lib/pois.ts client (renamed from overpass.ts; GET, quantized bbox)
- metrics: poi_api_requests_total + DB-collected index rows/age gauges;
drop overpass_* metrics
- remove /api/overpass proxy + dead OVERPASS_URLS on the planner service
(Journal surface backfill still uses it via code default)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Single-file bind mounts (./foo.yml:/etc/foo.yml) pin to the host file's
inode at container-create time. The CD pipeline scp's a replacement file
(new inode), so the running container keeps reading the OLD inode —
`docker compose up -d` won't recreate on a content-only change, and
neither restart/SIGHUP/`caddy reload` re-reads the new file. Net effect:
config-only infra PRs deployed "successfully" but never took effect
(confirmed with PR #500's prometheus.yml; the WAL retention work only
applied because #498 also changed docker-compose.yml, forcing a
recreate). Caddyfile changes had the same latent gap.
Switch the four single-file config mounts to DIRECTORY mounts, which
resolve children live so a reload/restart picks up the new file:
- prometheus, loki, promtail: mount ./<svc> dir (loki/promtail
--config.file paths updated to the real filenames).
- caddy: move Caddyfile into caddy/ and mount the dir; ./sites overlays
/etc/caddy/sites (caddy/sites/.gitkeep keeps the mountpoint). Container
path /etc/caddy/Caddyfile is unchanged, so every `caddy reload --config
/etc/caddy/Caddyfile` ref still works. scp source paths in cd-infra and
cd-apps updated to ship the caddy/ dir.
cd-infra now applies config-only changes after `up -d`: SIGHUP prometheus
(zero downtime), restart loki+promtail (no SIGHUP reload), caddy reload
(graceful). Mirrored prometheus/loki mounts in docker-compose.dev.yml.
Validated: docker compose config (both files), caddy validate from the
new path, read-only-parent + sites-overlay mount mechanics, workflow YAML.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prometheus size-based retention counts the WAL toward its budget. With
retention.size=1GB and a stuck/corrupt WAL that had grown to 2.1GB, the
total was permanently over budget, so Prometheus deleted every
freshly-compacted block the moment it was written. Net effect: only the
in-memory ~2h head block was ever queryable — metrics appeared to "start
this morning" no matter when you looked, and last night's Caddy 502
spike (planner /api/overpass upstream stall, 18:33-18:34 UTC) was
unobservable in Prometheus (logs survived in Loki).
The corrupt WAL was cleared out-of-band on the flagship. This raises the
size backstop to 6GB so compacted blocks survive to the intended 15d
time-based retention. Disk is 38G (43% used), so 6GB is safe headroom.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Garmin Connect as the third connected-services provider (spec:
garmin-import). The interesting parts:
- Push-first ingestion: Garmin has no list endpoint. The webhook
normalizes ping (callbackURL) and push (inline) notification batches
into events; the slow work (authorized FIT download, FIT→GPX via the
shared converter, activity creation) runs in a garmin-import-activity
pg-boss job so the webhook answers fast. Callback URLs are validated
against Garmin's API host before any fetch (SSRF guard).
- History via backfill requests: /sync/import/garmin is a date-range
requester with honest async progress (no pick list — the concept
doesn't exist in a push model). Ranges chunk to Garmin's 90-day cap;
overlaps are free via sync_imports dedupe. Requests persist in
import_batches via two new nullable columns (range_start/range_end).
- OAuth2 + PKCE on the existing oauth credential kind. Design
correction from apply: the verifier rides a short-lived httpOnly
cookie scoped to the callback path — the state param is visible in
redirect URLs and must never carry it. Manifests opt in via pkce:true.
- Deregistration notifications flip the connection to 'revoked'
(row kept for audit, imports retained, re-connect prompt shown).
- Framework evolutions, all additive: parseWebhook returns
WebhookEvent[] (Garmin batches; Wahoo adapted), manifest gains
configured()/importUrl/pkce, importActivity accepts summary stats
for FIT-less imports, manager gains markRevoked.
- Env-gated: no GARMIN_CLIENT_ID → provider hidden on
/settings/connections. Privacy manifest entry (DE+EN). i18n en+de.
Rollout (§6) stays gated on the Garmin Developer Program application
(submitted 2026-06-07). Fixtures are doc-shaped; the staging soak
swaps in recorded payloads if shapes differ.
Gate: typecheck ✓ lint ✓ unit+integration ✓ e2e 70/72 + both known
flakes green isolated ✓ openspec validate ✓
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wiring follows the IS_FLAGSHIP pattern: the compose env defaults every
FEDERATION_* var to empty (off — the safe self-host default; all
federation surfaces 404), and cd-apps.yml appends
FEDERATION_ENABLED=true + FEDERATION_LOG_LEVEL=info to the flagship's
app.env. FEDERATION_KEY_ENCRYPTION_KEY was already in SOPS.
Rollout 12.2/12.3 soaked on staging against a real Mastodon
(2026-06-06/07); the trails↔trails Accept bug found by the §11 harness
is fixed and deployed. Rollback: drop the two echo lines, merge, rerun
cd-apps — instant off, follow rows persist.
First boot with the flag enqueues backfill-user-keypairs (idempotent)
and registers the federation jobs. Applied via manual cd-apps dispatch
after merge since workflow-file changes don't trigger the path filter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to #466: staging and preview services attach only to the
externally-named trails-shared network (created by the production
compose project) — they have no per-project default network, so the
enable_ipv6 added to the staging file was dead config. Move the flag
to trails-shared where it takes effect.
Rollout requires recreating trails-shared on the flagship: detach
staging, previews, and production postgres, remove the network, re-up.
Production journal/planner are unaffected (they use the default
network).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The journal container could not reach v6-only fediverse instances
(e.g. social.ullrich.is) because the compose project network was
IPv4-only. The flagship host has working IPv6, and Docker >= 27
auto-allocates a ULA subnet with NAT66 masquerading when a network
sets enable_ipv6, so the fix is a one-line opt-in per network.
Applying it to an existing deployment requires recreating the
network (docker compose down && up -d) — a brief full-stack
restart on the flagship and the persistent staging project.
PR-preview networks are created fresh per PR and need nothing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Self-hosted instances no longer inherit the trails.cool flagship Sentry
DSNs. Code paths now read DSN strictly from env — unset = no Sentry
init, no events sent.
Flagship continues to report unchanged: cd-apps.yml and cd-staging.yml
inject the public DSNs as workflow env vars, which feed into:
- runtime via \`infrastructure/app.env\` / \`staging.env\`
(\`SENTRY_DSN_JOURNAL\` / \`SENTRY_DSN_PLANNER\` → compose env per service)
- build time via docker build-arg \`VITE_SENTRY_DSN\` (journal only;
planner has no client Sentry init).
Sentry DSNs are public-by-design (transmitted unencrypted from the
client JS bundle), so embedding them as plaintext workflow env vars
is no worse than the runtime exposure. Forks should replace these with
their own DSNs or remove the workflow env lines to ship Sentry-free.
Files changed:
- apps/journal/server.ts: \`process.env.SENTRY_DSN\` only, no fallback
- apps/planner/server.ts: same
- apps/journal/app/lib/sentry.client.ts: \`import.meta.env.VITE_SENTRY_DSN\`
only; falsy = skip init
- apps/journal/Dockerfile: new \`ARG VITE_SENTRY_DSN\` baked into build
- infrastructure/docker-compose.yml: pass \`SENTRY_DSN\` per service
- infrastructure/docker-compose.staging.yml: same
- .github/workflows/cd-apps.yml: workflow env + build-arg + app.env echo
- .github/workflows/cd-staging.yml: same
Full repo: pnpm typecheck, pnpm lint, pnpm test, journal build all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
infrastructure/docker-compose.yml had \`\${VAR:-default}\` fallbacks for
JWT_SECRET, SESSION_SECRET, and POSTGRES_PASSWORD that silently
substituted known-weak values (\`change-me-in-production\`, \`trails\`)
when the env wasn't set. A misconfigured prod deploy would happily come
up with these defaults — JWT/session forgery + a guessable Postgres
password.
Switched the four critical substitutions to \`\${VAR:?message}\` so
\`docker compose up\` fails loud instead. Mirrors the existing pattern
already used for BROUTER_URL/BROUTER_AUTH_TOKEN and (in staging) for
JWT_SECRET/SESSION_SECRET.
Sites fixed:
- journal: DATABASE_URL (POSTGRES_PASSWORD), JWT_SECRET, SESSION_SECRET
- planner: DATABASE_URL (POSTGRES_PASSWORD)
- postgres: POSTGRES_PASSWORD (container itself)
- exporter: DATA_SOURCE_NAME (POSTGRES_PASSWORD)
- staging: both DATABASE_URLs
Production already provides all three via SOPS-encrypted
secrets.app.env, so this is a defense-in-depth change with no behavior
change on a properly-configured deploy.
Updated infrastructure/.env.example to make SESSION_SECRET explicit
(was missing) and call out the trio as required.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add INTEGRATION_SECRET to journal service in docker-compose.yml with
:? guard so a missing value fails loudly at compose-up time
- Add INTEGRATION_SECRET to E2E test step in ci.yml via GitHub secret
(unit tests already set their own value in the test file)
- Archive openspec/changes/komoot-import → archive/2026-05-23-komoot-import
- Sync delta specs: new openspec/specs/komoot-import/spec.md,
updated openspec/specs/route-management/spec.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the staging-environments OpenSpec change. Persistent staging at
staging.trails.cool / planner.staging.trails.cool deploys from main; PR
opens get a journal-only preview at pr-<N>.staging.trails.cool that shares
the persistent planner. cd-staging.yml builds tagged images, manages
per-PR Postgres databases and Caddyfile snippets, evicts the oldest
preview at the cap of 3, and tears everything down on PR close.
staging-cleanup.yml runs weekly to sweep orphaned previews.
DNS records (staging + *.staging A/AAAA) already applied to production
via tofu.
Caddy approach: per-PR Caddyfile snippets imported from /etc/caddy/sites/
and reloaded on each PR event — no wildcard / on-demand TLS, no router
service. Production compose gains a trails-shared network for the staging
project to reach Postgres, and host.docker.internal on Caddy so it can
reverse-proxy staging containers published on the host loopback.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The old home was an h1, subtitle, and two auth buttons — visitors
arriving at trails.cool had no idea what it was, and self-hosted
instances had nothing interesting to land on.
The new home is one layout that serves both audiences:
- Hero: a product-describing h1 ("Federated outdoor journal") +
tagline. The site name already lives in the top banner and nav brand,
so the h1 carries the pitch instead of triplicating "trails.cool".
- Auth CTAs: Register (blue) + Sign In (outlined) get primary weight.
Planner demotes to a small "Or try the Planner without an account →"
link below them — it's a nice escape hatch but not the goal.
- Marketing cards: on the flagship only, a 2x2 grid of emoji-icon cards
for Planner / Journal / Federation / Ownership, matching the Planner
home's visual weight. `IS_FLAGSHIP=true` toggles it. Self-hosters get
a "Powered by trails.cool — about the project" link back to the
flagship instead, so they don't have to write their own marketing.
- Public feed: reverse-chronological list of the 20 most recent public
activities on the instance, with owner + distance + date. Empty state
for fresh installs.
Plumbing:
- `listRecentPublicActivities(limit)` in activities.server.ts joins
users so the feed can render "by <displayName>" in one query.
- `IS_FLAGSHIP` env wired through docker-compose.yml; cd-apps and
cd-infra both write `IS_FLAGSHIP=true` alongside `DOMAIN=trails.cool`,
so self-hosters (who deploy without these workflows) default to off.
Specs:
- `activity-feed`: new "Instance-wide public activity feed" requirement
so the visibility contract is pinned (public in feed, unlisted +
private are not).
- `journal-landing`: new small spec capturing the hero / marketing /
feed layout contract and the `IS_FLAGSHIP` toggle, so future
instance-branding or empty-state changes have a stable anchor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The notification policy and contact points live in
`infrastructure/grafana/provisioning/alerting/alerts.yml`, so
UI-created contact points can't be attached to provisioned alerts.
Provision the Pushover integration instead.
Rather than add a second contact point and do the fan-out in the
notification tree (Grafana's tree only fires one receiver per match
without duplication workarounds), put both email + pushover under a
single `default` contact point. Every alert now fans out to both.
Secrets go to SOPS `secrets.infra.env`. The Grafana container reads
`PUSHOVER_API_TOKEN` and `PUSHOVER_USER_KEY_ULLRICH` from env; the
provisioning YAML references them via `$VAR` and Grafana substitutes
at startup, so no secret lands in git.
Priority 1 on firing (bypass quiet hours), 0 on resolve. User can
tweak in the YAML later.
Post-deploy: delete the UI-created "Pushover Ullrich" contact point
in Grafana to avoid a duplicate. It's unused (no policy references it)
but shows up in the contact-point list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Users land via GitHub OAuth gated by the trails-cool org allowlist, so
Viewer-only is more restrictive than we actually want — no Explore
access, no ad-hoc Loki/Prometheus queries. Promote the default role to
Editor so new sign-ins get it automatically.
Note: GF_USERS_AUTO_ASSIGN_ORG_ROLE only applies to new users on first
sign-in. Existing Viewers (including the single current user) need a
one-off promotion via the Grafana admin UI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After PR #293 flipped BROUTER_URL to the dedicated host and Grafana +
manual smoke tests confirmed clean routing (US route worked, request
rate/latency/memory/logs all green), there's no reason to keep the
in-tree flagship brouter container warm any longer.
- Remove the `brouter:` service and its `./segments` bind mount from
`infrastructure/docker-compose.yml`.
- Remove `depends_on: brouter` from the planner service.
- Tighten the BROUTER_URL and BROUTER_AUTH_TOKEN env wiring from
`${…:-default}` to `${…:?message}` so a missing SOPS value fails the
compose up loudly instead of silently pointing at the removed service
(or missing auth).
- Tick tasks 5.5, 7.5, 8.4, 9.3 in the OpenSpec change; 9.4 (archive)
is the last remaining step once this merges.
Post-merge operator step: `docker image prune -f` on the flagship to
reclaim the brouter image.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands sections 6 and 8 of relocate-brouter-to-dedicated-host on top
of the host compose in #291.
## Observability (section 6)
- **Prometheus**: new `brouter-cadvisor` job scraping
`10.0.1.10:8080` over the vSwitch, labeled `host="brouter"`.
- **cAdvisor sidecar** on the dedicated host's compose
(`--docker_only --whitelisted_container_labels=trails.cool.service`)
so metrics only cover trails containers, never the operator's
unrelated workloads on the shared box.
- **Promtail sidecar** on the dedicated host, Docker SD with relabel-
drop on missing `trails.cool.service` label, pushing to flagship
Loki at `http://10.0.0.2:3100/loki/api/v1/push`.
- **Flagship compose**: Loki now publishes port 3100 on the vSwitch
IP only (10.0.0.2:3100) — Hetzner Cloud firewall blocks it from
the public internet.
- **Grafana dashboard**: `brouter.json` — scrape up/down, request
rate (from Planner-side `brouter_request_duration_seconds`),
p50/p95/p99, container memory/CPU, Loki logs panel.
- **Alert**: `brouter-scrape-down` fires on
`up{job="brouter-cadvisor"} < 1 for 2m`; `noDataState: Alerting`
so a total scrape failure still pages.
Operator needs one UFW rule on the dedicated host for the cAdvisor
port — documented in `infrastructure/brouter-host/README.md`.
## Documentation (section 8)
- `CLAUDE.md` — hosts table + updated deployment table with SSH
targets per workflow; BRouter host SSH is `-p 2232 trails@...`,
different key.
- `docs/architecture.md` — Hosting section rewritten to cover both
hosts, vSwitch boundary, and the observability-scoping rationale
for the shared dedicated host.
- `docs/deployment.md` (new) — full operator runbook: host layout,
first-time BRouter provisioning, SOPS rotation (including the
macOS `SOPS_AGE_KEY_FILE` gotcha), cutover procedure with
rollback, manual workflow triggers.
Task 8.4 (infrastructure/README.md) skipped: that file doesn't
exist and the ground is covered by brouter-host/README.md +
docs/deployment.md.
## Validation
- `docker compose config` on both the flagship and brouter-host
compose files — both validate.
- `pnpm typecheck`, `pnpm lint`, `pnpm test` — all clean (full
turbo cache hits; no code changes in this commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands sections 3-5 of the relocate-brouter-to-dedicated-host change:
everything needed to run BRouter on the dedicated Hetzner Robot host
and have the Planner talk to it with the shared-secret header. Does
NOT flip the cutover — the flagship BRouter stays warm during soak.
## BRouter host compose (section 3)
New `infrastructure/brouter-host/` — a standalone compose project that
runs as the `trails` user on `ullrich.is`:
- `docker-compose.yml` — brouter + caddy sidecar. BRouter has no host
port; caddy binds only to `10.0.1.10:17777` (vSwitch IP). Every
service explicitly overrides the host's default Loki logging driver
to `json-file` so logs don't leak to the operator's personal Loki.
- `Caddyfile` — single-purpose reverse proxy that requires
`X-BRouter-Auth: ${BROUTER_AUTH_TOKEN}` on every request. `auto_https
off` (vSwitch-only); default access log format omits request
headers, so the token is never written to disk.
- `download-segments.sh` — crawls brouter.de, pulls planet-wide RD5
tiles via `wget -N` (incremental). Idempotent, safe to cron.
- `README.md` — one-shot provisioning + token rotation + rollback
notes.
`docker/brouter/Dockerfile` is patched to honor `JAVA_OPTS` (was
hardcoded `-Xmx1024M` in CMD). Default keeps the flagship's current
heap; compose on the dedicated host overrides to `-Xmx8g` for planet
scale on a 32 GB box.
## Planner shared-secret header (section 4)
`apps/planner/app/lib/brouter.ts`:
- Module-level guard: throws at startup in production if
`BROUTER_AUTH_TOKEN` is unset.
- `authHeaders()` helper (reads env at call time, so tests can
`vi.stubEnv` without module reset).
- Header attached on both `computeRoute` (per-segment) and
`computeSegmentGpx`.
3 new unit tests cover header attachment + the no-token path.
`infrastructure/docker-compose.yml` passes `BROUTER_AUTH_TOKEN` to
the Planner service, and makes `BROUTER_URL` overridable via SOPS so
the cutover is a one-variable flip.
## cd-brouter workflow (section 5)
Rewritten to deploy to the dedicated host:
- SSH as `trails@${BROUTER_DEPLOY_HOST}` on port
`${BROUTER_DEPLOY_SSH_PORT}` (2232) using
`${BROUTER_DEPLOY_SSH_KEY}`.
- Decrypts SOPS, extracts ONLY `BROUTER_AUTH_TOKEN` into a `.env`
file, scp'd alongside the compose project.
- `paths:` trigger now includes `infrastructure/brouter-host/**`.
- Segment download is NOT run here — first-time seed is a manual
operator step (multi-hour). Routine re-runs are cron-able on the
dedicated host.
- Grafana annotation step preserved (reaches flagship Grafana as
before).
## What's NOT here
- `brouter:` service on the flagship is intentionally left in place
(removed in section 7.5 after the 48 h soak window post-cutover).
- Observability (section 6) — Prometheus scrape + Loki shipping from
the dedicated host — comes in a follow-up PR.
- Cutover itself (section 7) — flip `BROUTER_URL`, verify, remove the
flagship brouter — is an operator action gated on first-time
provisioning + smoke testing.
## Verification
`pnpm typecheck && pnpm lint && pnpm test` all clean; planner build
passes (the CI regression from #286 was fixed in #290).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Public Overpass instances go bad without warning — we just lived
through a day where overpass.private.coffee (our single upstream)
returned 504s after 60s while lz4.overpass-api.de served the same
query in 0.6s. Survive that by trying a list of upstreams in order
until one responds usefully.
Server changes:
- OVERPASS_URLS env — comma-separated list, tried in order. Falls
back to OVERPASS_URL for backward compat, then to a built-in
default pair (lz4.overpass-api.de, overpass-api.de).
- Per-upstream timeout of 10s via AbortSignal.timeout, so a saturated
instance fails over in seconds, not minutes.
- Client abort propagation via AbortSignal.any — if the browser
cancels (e.g. user panned the map), the in-flight upstream fetch is
aborted too. Stops wasting upstream capacity on requests nobody
wants.
- Rate-limited-body detection (Overpass returns 200 with a
`rate_limited` marker when throttling) triggers failover.
- Per-upstream labels on overpass_upstream_requests_total and
overpass_upstream_duration_seconds so the dashboard can break out
health + latency by instance. Histogram buckets extended to 30s.
Compose: OVERPASS_URLS passthrough with the default pair hard-wired,
overridable via SOPS.
Tests: 8 cases covering the new fetchWithFailover helper — happy
path, 5xx failover, rate_limited failover, network error failover,
timeout tagging, all-upstreams-fail, client abort stops the loop,
per-attempt latency observation.
Follow-ups left out of scope:
- Client-side debounce (UI concern).
- Moving `.observe()` after body read for accuracy in measurement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds DEMO_BOT_PERSONA env var (inline JSON or file:<path>) so
self-hosted instances can replace Bruno-in-Berlin with their own
demo account identity and content pools. No-op for trails.cool — the
built-in Bruno persona remains the default.
- DemoPersona type + Zod schema validation
- loadPersona() cached at boot; falls back to default on any failure
- ensureDemoUser throws DemoPersonaUsernameClashError when the
persona username is already a real user; server declines to
schedule demo jobs for that process
- isDemoUser flag moved from hardcoded "bruno" check to loader-supplied
boolean computed from the running persona
- docs/demo-persona.md explains the schema + operator flow
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a background demo user (`bruno`) plus two pg-boss jobs that generate
public trekking routes and activities in inner Berlin and prune them
after 14 days. Disabled by default everywhere; flip `DEMO_BOT_ENABLED`
only in prod.
- Schema: `synthetic` boolean on routes + activities
- `ensureDemoUser()` idempotent insert + badge on /users/bruno
- Generation gate: 07-21 Berlin local, p=0.12, 40-route cap in 14d
- Initial 4-walk backfill on first enable; retention via daily prune
- Prometheus gauges `demo_bot_synthetic_routes_total` / `_activities_total`
- Unit + DB-gated integration + E2E tests
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds apps/planner/app/routes/api.overpass.ts — a same-origin, rate-limited
server-side proxy that forwards Overpass QL to the upstream endpoint
configured via OVERPASS_URL (default: overpass.private.coffee).
Motivation:
- private.coffee's best-practices require a meaningful User-Agent
identifying the project. Browsers cannot set User-Agent on fetch()
(forbidden header), so the request has to originate server-side.
- Collaborative sessions commonly have N clients pan/zoom the same map,
so the same bbox query arrives multiple times within seconds.
- Setting up the proxy now lets us swap OVERPASS_URL to a self-hosted
Overpass later without client changes.
What the proxy does:
- Sets User-Agent "trails.cool Planner (https://trails.cool; legal@trails.cool)"
- Enforces same-origin via the Origin header
- Rate-limits per client IP (120 req/min)
- In-memory LRU cache of upstream responses keyed on the form-encoded body
(TTL 10 min, max 200 entries)
- Coalesces concurrent misses for the same key so N simultaneous clients
in one session incur exactly one upstream call
Client change: apps/planner/app/lib/overpass.ts POSTs to /api/overpass
instead of iterating over public Overpass endpoints. Fallback list
removed; resilience is now the proxy's responsibility.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch journal Docker health check from `node -e "fetch(...)"` to `curl -sf`,
matching the planner. The Node.js process spawn was heavy and prone to timeout
under memory pressure, causing false health check failures and 502s.
Upgrade Promtail from static file scraping to Docker service discovery so logs
get `service` and `container` labels. Parse Pino JSON logs to extract the
`level` label, enabling Loki queries filtered by log level.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dashboard: switch error rate panel from app-level http_request_duration
(never observed) to Caddy's caddy_http_response_duration_seconds_count.
Journal: add custom server.ts (matching planner pattern) that observes
http_request_duration_seconds on every request. Replaces react-router-serve
with a custom Node HTTP server that handles /api/health, /api/metrics,
static assets, and request timing. Removes redundant React Router routes
for health and metrics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Caddy Response Status Codes: caddy_http_responses_total doesn't exist,
use caddy_http_response_duration_seconds_count which has the code label
- pg_stat_statements: enable extension in init script, add custom queries
config for postgres-exporter, remove invalid datname filter from query
- Add postgres/queries.yml to cd-infra SCP sources
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Loki was running but had no log shipper — no labels or logs were
visible in Grafana. Adds Promtail to scrape all Docker container
logs and push them to Loki.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WAHOO_CLIENT_ID, WAHOO_CLIENT_SECRET, and WAHOO_WEBHOOK_TOKEN were in
SOPS secrets but never passed through to the journal service in
docker-compose.yml, causing the OAuth authorize URL to have an empty
client_id.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Without this, GitHub OAuth creates a new Grafana user on every login
instead of linking to the existing account by email.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Create grafana_reader role with SELECT-only access on all schemas
- Init script runs on postgres first boot (docker-entrypoint-initdb.d)
- Grafana PostgreSQL datasource provisioned with read-only credentials
- Enables SQL queries in dashboards (user count, routes, etc.)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Exporters:
- postgres_exporter: DB connections, transactions, cache hit ratio, query stats
- node_exporter: host CPU, memory, disk, network
- cAdvisor: per-container CPU and memory usage
PostgreSQL:
- Enable pg_stat_statements for query-level performance tracking
- Track index scans vs sequential scans, cache hit ratio
Dashboard (service-health.json):
- DB: connections, size, transactions/s, slow queries, cache hit ratio, index usage
- Host: disk gauge, CPU, memory, network I/O, disk I/O
- BRouter: request latency p50/p95/p99, container CPU + memory
- All containers: CPU and memory comparison
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Secrets:
- Add .sops.yaml with age encryption config
- Add encrypted secrets.app.env (app secrets) and secrets.infra.env (Grafana OAuth)
- CD decrypts at deploy time with AGE_SECRET_KEY — all other secrets
move out of GitHub Actions into version-controlled encrypted files
Split CD:
- cd-apps.yml: triggered by apps/packages changes, builds Docker images, deploys apps
- cd-infra.yml: triggered by infrastructure/ changes, copies configs, restarts services
- Remove monolithic cd.yml
Grafana auth:
- GitHub OAuth (trails-cool org), disable login form
- Remove Caddy basic_auth block and all GRAFANA_* env vars/secrets
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add DOMAIN, GRAFANA_USER, GRAFANA_PASSWORD_HASH as environment vars
on the Caddy container so Caddyfile can reference them
- Fix Loki compactor: add delete_request_store required when retention is enabled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add wildcard DNS record for *.internal.trails.cool (A + AAAA) so
internal services don't leak hostnames in individual DNS entries.
Move Grafana from grafana.trails.cool to grafana.internal.trails.cool.
Future internal services just need a Caddy block — no DNS changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Health endpoints:
- /api/health (Journal) and /health (Planner) with DB connectivity check
- Docker healthchecks updated to use app health endpoints
Structured logging:
- Pino with JSON output in production, pretty-print in dev
- Request logging middleware in Planner (method, path, status, duration)
- Replaced console.log/error with structured logger in email and auth flows
Prometheus metrics:
- prom-client with default Node.js metrics + custom histograms/gauges
- /metrics endpoints on both apps
- http_request_duration, planner_active_sessions, brouter_request_duration
Monitoring stack:
- Prometheus, Loki, Grafana containers in docker-compose
- Grafana provisioned with datasources, dashboards, and alert rules
- Caddy access logging (JSON to stdout for Loki)
- grafana.trails.cool with basic auth via Caddy
Dashboards and alerting:
- Overview: request rate, error rate, latency p50/p95/p99
- Planner: active sessions, connected clients, BRouter latency
- Infrastructure: memory, CPU, event loop lag
- Alerts: disk >80%, app down 2min, error rate >5%
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Transactional emails:
- Add nodemailer SMTP email module with dev-mode console logging
- Magic link template and welcome template with HTML + plain text
- Wire sendMagicLink into login flow, sendWelcome into registration
- Update privacy page and deploy docs for SMTP configuration
Planner features:
- No-go areas: draw polygons on map (leaflet-geoman), synced via Yjs,
passed to BRouter as nogos parameter, route recomputes on change
- Session notes: collaborative Y.Text textarea in sidebar tab
- Crash recovery: periodic localStorage save of Yjs state, restore on reconnect
- Rate limit session creation (10/IP/hour) in /new route
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- @sentry/vite-plugin uploads source maps during Docker build
- Release tagged with git SHA (SENTRY_RELEASE) on both client and server
- Environment set to production/development for filtering
- CD passes SENTRY_AUTH_TOKEN + SENTRY_RELEASE as Docker build args
- docker-compose passes SENTRY_RELEASE to runtime containers
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CD workflow:
- Uncommented deploy step
- Copy docker-compose.yml + Caddyfile to server via SCP
- Login to ghcr.io with DEPLOY_GHCR_TOKEN (classic PAT, read:packages)
- Download Germany BRouter segments on first deploy (~750MB)
- Pull images, restart services, verify
Infrastructure:
- BRouter segments as bind mount (./segments) not Docker volume
- Added ORIGIN, PLANNER_URL, SESSION_SECRET, NODE_ENV to Journal
- Added NODE_ENV to Planner
- Disabled Garage for now (no media storage yet)
Required secrets: DEPLOY_HOST, DEPLOY_SSH_KEY, DEPLOY_GHCR_TOKEN
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Terraform:
- Hetzner CX22 server with Docker, firewall (80/443/22)
- SSH key and output for server IP
Docker Compose:
- Caddy reverse proxy (auto HTTPS)
- Journal and Planner app containers
- BRouter routing engine container
- PostgreSQL + PostGIS
- Garage S3 storage
BRouter:
- Dockerfile based on eclipse-temurin:11-jre
- Segment download script for Germany (4 tiles, ~750MB)
CD Pipeline:
- Build & push Docker images to ghcr.io on main push
- Deploy step commented out (enable when server provisioned)
Scripts:
- PostgreSQL daily backup with 14-day retention
- Dockerfiles for both apps (multi-stage builds)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
