Commit graph

11 commits

Author SHA1 Message Date
Ullrich Schäfer
b45e69885d
poi-index: map-core selectors, planner.pois schema, /api/pois route + client, metrics
Replace the planner's Overpass proxy with a self-hosted POI index:
- map-core POI categories become structured tag selectors (single source of
  truth) + osmium filter / classification helpers
- planner.pois PostGIS table (centroid points, GiST + category indexes)
- /api/pois serving route (session + same-origin + rate limit, 100 cap)
- lib/pois.ts client (renamed from overpass.ts; GET, quantized bbox)
- metrics: poi_api_requests_total + DB-collected index rows/age gauges;
  drop overpass_* metrics
- remove /api/overpass proxy + dead OVERPASS_URLS on the planner service
  (Journal surface backfill still uses it via code default)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 22:38:03 +02:00
Ullrich Schäfer
0360757ae8 feat(journal): Garmin activity import — provider, webhook pipeline, backfill (§1–5)
Garmin Connect as the third connected-services provider (spec:
garmin-import). The interesting parts:

- Push-first ingestion: Garmin has no list endpoint. The webhook
  normalizes ping (callbackURL) and push (inline) notification batches
  into events; the slow work (authorized FIT download, FIT→GPX via the
  shared converter, activity creation) runs in a garmin-import-activity
  pg-boss job so the webhook answers fast. Callback URLs are validated
  against Garmin's API host before any fetch (SSRF guard).
- History via backfill requests: /sync/import/garmin is a date-range
  requester with honest async progress (no pick list — the concept
  doesn't exist in a push model). Ranges chunk to Garmin's 90-day cap;
  overlaps are free via sync_imports dedupe. Requests persist in
  import_batches via two new nullable columns (range_start/range_end).
- OAuth2 + PKCE on the existing oauth credential kind. Design
  correction from apply: the verifier rides a short-lived httpOnly
  cookie scoped to the callback path — the state param is visible in
  redirect URLs and must never carry it. Manifests opt in via pkce:true.
- Deregistration notifications flip the connection to 'revoked'
  (row kept for audit, imports retained, re-connect prompt shown).
- Framework evolutions, all additive: parseWebhook returns
  WebhookEvent[] (Garmin batches; Wahoo adapted), manifest gains
  configured()/importUrl/pkce, importActivity accepts summary stats
  for FIT-less imports, manager gains markRevoked.
- Env-gated: no GARMIN_CLIENT_ID → provider hidden on
  /settings/connections. Privacy manifest entry (DE+EN). i18n en+de.

Rollout (§6) stays gated on the Garmin Developer Program application
(submitted 2026-06-07). Fixtures are doc-shaped; the staging soak
swaps in recorded payloads if shapes differ.

Gate: typecheck ✓ lint ✓ unit+integration ✓ e2e 70/72 + both known
flakes green isolated ✓ openspec validate ✓

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:47:22 +02:00
Ullrich Schäfer
1a8ec4e8b9 docs(staging): un-stale the federation comment in compose
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 13:41:36 +02:00
Ullrich Schäfer
2d8046b9a6 fix(infra): IPv6 belongs on trails-shared, not a staging default network
Follow-up to #466: staging and preview services attach only to the
externally-named trails-shared network (created by the production
compose project) — they have no per-project default network, so the
enable_ipv6 added to the staging file was dead config. Move the flag
to trails-shared where it takes effect.

Rollout requires recreating trails-shared on the flagship: detach
staging, previews, and production postgres, remove the network, re-up.
Production journal/planner are unaffected (they use the default
network).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 23:21:25 +02:00
Ullrich Schäfer
915b8121d0 feat(infra): enable IPv6 egress on Docker compose networks
The journal container could not reach v6-only fediverse instances
(e.g. social.ullrich.is) because the compose project network was
IPv4-only. The flagship host has working IPv6, and Docker >= 27
auto-allocates a ULA subnet with NAT66 masquerading when a network
sets enable_ipv6, so the fix is a one-line opt-in per network.

Applying it to an existing deployment requires recreating the
network (docker compose down && up -d) — a brief full-stack
restart on the flagship and the persistent staging project.
PR-preview networks are created fresh per PR and need nothing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 23:12:03 +02:00
Ullrich Schäfer
9a1ce61604 feat(journal): surface Fedify logs via LogTape console sink
Inbound Mastodon Follows are being rejected 401 (signature
verification) on staging and Fedify's diagnostics were invisible:
it logs through LogTape, which is silent until configured.

- Configure LogTape with a console sink for the 'fedify' category when
  the federation instance is built; level via FEDERATION_LOG_LEVEL
  (default info).
- Staging deploy sets FEDERATION_LOG_LEVEL=debug for the soak —
  signature-verification detail per request; dial down once proven.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 21:41:28 +02:00
Ullrich Schäfer
57f9440cf5 feat(staging): enable federation on persistent staging
social-federation rollout step 12.2 — inbound soak. Persistent staging
gets FEDERATION_ENABLED=true; PR previews keep the flag empty so
preview journals never emit federation traffic.

- docker-compose.staging.yml: journal accepts FEDERATION_ENABLED +
  FEDERATION_KEY_ENCRYPTION_KEY (both default empty).
- cd-staging.yml: persistent staging env sets FEDERATION_ENABLED=true;
  the encryption key arrives via the existing SOPS decryption.
- secrets.app.env: new FEDERATION_KEY_ENCRYPTION_KEY (SOPS-encrypted,
  generated with openssl rand -hex 32).

Production is unaffected: the flagship compose doesn't pass these vars
and the flag defaults off in code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 15:45:44 +02:00
Ullrich Schäfer
edb618fe40
fix(sentry): remove hardcoded DSN fallbacks; supply via env in CI
Self-hosted instances no longer inherit the trails.cool flagship Sentry
DSNs. Code paths now read DSN strictly from env — unset = no Sentry
init, no events sent.

Flagship continues to report unchanged: cd-apps.yml and cd-staging.yml
inject the public DSNs as workflow env vars, which feed into:
- runtime via \`infrastructure/app.env\` / \`staging.env\`
  (\`SENTRY_DSN_JOURNAL\` / \`SENTRY_DSN_PLANNER\` → compose env per service)
- build time via docker build-arg \`VITE_SENTRY_DSN\` (journal only;
  planner has no client Sentry init).

Sentry DSNs are public-by-design (transmitted unencrypted from the
client JS bundle), so embedding them as plaintext workflow env vars
is no worse than the runtime exposure. Forks should replace these with
their own DSNs or remove the workflow env lines to ship Sentry-free.

Files changed:
- apps/journal/server.ts: \`process.env.SENTRY_DSN\` only, no fallback
- apps/planner/server.ts: same
- apps/journal/app/lib/sentry.client.ts: \`import.meta.env.VITE_SENTRY_DSN\`
  only; falsy = skip init
- apps/journal/Dockerfile: new \`ARG VITE_SENTRY_DSN\` baked into build
- infrastructure/docker-compose.yml: pass \`SENTRY_DSN\` per service
- infrastructure/docker-compose.staging.yml: same
- .github/workflows/cd-apps.yml: workflow env + build-arg + app.env echo
- .github/workflows/cd-staging.yml: same

Full repo: pnpm typecheck, pnpm lint, pnpm test, journal build all green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:07:07 +02:00
Ullrich Schäfer
e2bf3ddb94
fix(infra): refuse compose up when production secrets are missing
infrastructure/docker-compose.yml had \`\${VAR:-default}\` fallbacks for
JWT_SECRET, SESSION_SECRET, and POSTGRES_PASSWORD that silently
substituted known-weak values (\`change-me-in-production\`, \`trails\`)
when the env wasn't set. A misconfigured prod deploy would happily come
up with these defaults — JWT/session forgery + a guessable Postgres
password.

Switched the four critical substitutions to \`\${VAR:?message}\` so
\`docker compose up\` fails loud instead. Mirrors the existing pattern
already used for BROUTER_URL/BROUTER_AUTH_TOKEN and (in staging) for
JWT_SECRET/SESSION_SECRET.

Sites fixed:
- journal:   DATABASE_URL (POSTGRES_PASSWORD), JWT_SECRET, SESSION_SECRET
- planner:   DATABASE_URL (POSTGRES_PASSWORD)
- postgres:  POSTGRES_PASSWORD (container itself)
- exporter:  DATA_SOURCE_NAME (POSTGRES_PASSWORD)
- staging:   both DATABASE_URLs

Production already provides all three via SOPS-encrypted
secrets.app.env, so this is a defense-in-depth change with no behavior
change on a properly-configured deploy.

Updated infrastructure/.env.example to make SESSION_SECRET explicit
(was missing) and call out the trio as required.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 22:59:09 +02:00
Ullrich Schäfer
25d21161c5
Fix staging port binding + diagnostic ps env file
Two bugs in the staging-environments rollout:

1. Staging containers published on 127.0.0.1 are unreachable from the
   production Caddy container, which connects via the Docker bridge IP
   (host.docker.internal:host-gateway resolves to the bridge, not
   loopback). Bind to 0.0.0.0 instead — Hetzner Cloud firewall blocks
   ports 3000+ from the public internet, so it stays internal-only.

2. The trailing 'docker compose ps' diagnostic in deploy-staging /
   deploy-preview was missing --env-file staging.env, so compose
   failed env interpolation and the job exited non-zero even when the
   actual deploy succeeded.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 22:30:05 +02:00
Ullrich Schäfer
c8a7a0b253
Add staging + PR-preview environments on the flagship
Implements the staging-environments OpenSpec change. Persistent staging at
staging.trails.cool / planner.staging.trails.cool deploys from main; PR
opens get a journal-only preview at pr-<N>.staging.trails.cool that shares
the persistent planner. cd-staging.yml builds tagged images, manages
per-PR Postgres databases and Caddyfile snippets, evicts the oldest
preview at the cap of 3, and tears everything down on PR close.
staging-cleanup.yml runs weekly to sweep orphaned previews.

DNS records (staging + *.staging A/AAAA) already applied to production
via tofu.

Caddy approach: per-PR Caddyfile snippets imported from /etc/caddy/sites/
and reloaded on each PR event — no wildcard / on-demand TLS, no router
service. Production compose gains a trails-shared network for the staging
project to reach Postgres, and host.docker.internal on Caddy so it can
reverse-proxy staging containers published on the host loopback.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 22:03:38 +02:00