The routeData Y.Map's ~15 string keys (geojson, coordinates,
segmentBoundaries, road metadata, profile, colorMode, baseLayer,
overlays, poiCategories) were read and written raw at ~30 call sites,
each with its own JSON parsing and casts; parseJsonArray existed twice
and waypoint extraction four times. GPX assembly was duplicated between
SaveToJournalButton and ExportButton, so the saved plan and the
exported file could silently diverge.
- new lib/route-data.ts owns the routeData (+ noGoAreas) schema:
typed read/write, JSON encoding internal, ColorMode moves here
(re-exported from ColoredRoute for existing importers)
- new lib/gpx-export.ts owns GPX assembly: buildRouteGpx /
buildPlanGpx / buildDayGpxFiles / hasDayBreaks; multi-day splitting
becomes a pure, tested function
- waypoint-ymap.ts gains extractWaypoints / extractWaypointData; the
four hand-rolled copies (use-routing, use-waypoint-manager,
WaypointSidebar, use-days) now share it, and WaypointSidebar's
moveWaypoint reuses the round-trip helpers instead of re-listing
every waypoint field
- all hooks/components consume the seam; no raw routeData key strings
remain outside route-data.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
## Symptom
Grafana's \`app-crash-log\` alert fires on every deploy. The Loki query
that backs it (\`ERR_|FATAL|uncaughtException|…\`) matches:
Error [ERR_MODULE_NOT_FOUND]: Cannot find module
'/app/apps/journal/app/lib/logger.server' imported from
'/app/apps/journal/app/lib/email.server.ts'
A real bug, not a false positive: \`email.server.ts\` line 2 was
\`import { logger } from \"./logger.server\"\` — no extension.
## Why typecheck didn't catch it
\`tsconfig.base.json\` sets \`moduleResolution: \"bundler\"\`. The bundler
resolver accepts extensionless relative imports because Vite / esbuild
/ webpack resolve them at build time. TypeScript was happy.
Production runs \`node --experimental-strip-types server.ts\`, which
uses Node's NodeNext ESM resolver — strict about extensions. The two
resolvers disagree silently for files Node loads directly (server.ts,
\`.server.ts\` modules dynamically imported from it, and jobs).
## Fix
1. **Added \`.ts\` extensions to the 4 broken imports** I could find:
- \`apps/journal/app/lib/email.server.ts\` → \`./logger.server.ts\`
(the actual deploy-blocker)
- \`apps/planner/app/lib/brouter.ts\` → \`./route-merge.ts\`, \`./http.server.ts\`
- \`apps/planner/app/lib/use-nearby-pois.ts\` → \`./overpass.ts\`
- \`packages/map/src/MapView.tsx\` → \`./layers.ts\` (caught by the rule)
2. **Added \`eslint-plugin-import-x\` with \`import-x/extensions: always\`**,
scoped to files Node actually executes raw:
- \`apps/*/server.ts\`
- \`apps/*/app/lib/**/*.server.ts\`
- \`apps/*/app/jobs/**/*.ts\`
- \`packages/*/src/**/*.{ts,tsx}\`
Ignored: route-scoped \`*.server.ts\` files (bundled by Vite into the
React Router build, never loaded by Node), and test files (Vitest's
own resolver).
## What's still possible
- Non-\`.server.ts\` files in \`app/lib\` (e.g. \`legal.ts\`, \`actor-iri.ts\`)
could still ship extensionless imports. They're not in the lint
scope. If one breaks at runtime we can extend the glob — for now
they tend to be tiny utility modules that don't import other
relatives.
- The deeper fix would be \`moduleResolution: nodenext\` for server-side
tsconfig, or bundling the server code so Node never sees raw \`.ts\`.
Bigger surgery; the lint rule covers the failure mode for now.
Full repo: pnpm typecheck / lint / test all green after \`pnpm install\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After this, \`grep -rn 'eslint-disable' apps/ packages/\` returns 0
(excluding tests and node_modules).
**\`apps/planner/app/lib/brouter.ts\`** — \`while (true)\` in
\`readBodyWithCap\` tripped \`no-constant-condition\`. Replaced with
\`for (;;)\` which the rule explicitly allows. No behavior change.
**\`packages/gpx/src/parse.ts\`** — the sync \`parseGpx\` exported a
fallback path that did \`require(\"linkedom\")\` for non-browser sync
use, with an \`eslint-disable\` for \`no-require-imports\`. It was dead
code: \`packages/gpx/src/index.ts\` only re-exports \`parseGpxAsync\`,
and grepping the monorepo finds no caller of the sync version. Deleted
the function entirely; \`getDOMParser\` (the async helper) still serves
the async path with a clean ESM \`await import(\"linkedom\")\`.
Full repo: pnpm typecheck / lint / test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- RouteMapThumbnail.client.tsx: `{ type: 'Feature', ... } as unknown
as GeoJsonObject` → `as GeoJsonObject`. The literal already
satisfies the type; the unknown bridge was unnecessary.
- use-yjs.ts:102: keep the coercion (y-websocket's `.on()` signature
doesn't include the legacy `"synced"` event even though the
runtime still fires it), but document why with a comment so future
readers don't try to drop it.
Most of the existing coercions are legitimate (Drizzle raw-SQL row
shapes, linkedom DOMParser interop, fit-file-parser's loose \`any\`
callback API) — leaving those is honest. The ones cleaned up here
were shims around APIs that have real types we just hadn't wired:
- **\`window.__leafletMap\`** (4 sites: MapHelpers.tsx, SessionView.tsx ×3)
Added \`Window\` declaration in \`apps/planner/app/types/global.d.ts\`.
Callers now use plain \`window.__leafletMap\` typed as
\`L.Map | undefined\`.
- **\`L.markerClusterGroup\`** (PoiPanel.tsx) — leaflet.markercluster
augments the global \`L\` namespace at runtime but ships no types.
Added a \`declare module \"leaflet\"\` block in the same global.d.ts
with a minimal signature for what we actually use.
- **\`L.DomEvent.preventDefault / .stop\`** taking a Leaflet event
rather than a native one (PlannerMap.tsx, RouteInteraction.tsx,
NoGoAreaLayer.tsx). Leaflet events carry the native event under
\`.originalEvent\`; using that drops the cast entirely.
- **\`_creds as unknown as OAuthCredentials\`** orphan in
wahoo/webhook.ts — \`_creds\` was unused inside the callback (the
void-cast was a no-op preserving the type for documentation).
Replaced with \`async ()\`, dropped the unused import.
Net: 26 \`as unknown as\` / \`as any\` sites → 19, all remaining ones
gated by external lib interop or Drizzle raw-SQL.
Full repo: pnpm typecheck / lint / test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Planner-audit #9. \`fetchSegment\` previously \`await response.json()\`
on any body BRouter returned. A misbehaving or compromised upstream
could OOM the planner process by returning gigabytes of JSON.
New \`readBodyWithCap()\`:
- Reject upfront when \`content-length\` declares over the cap.
- Stream the body and abort the reader once received bytes exceed
the cap (handles the case where upstream lies about content-length
or omits it).
Cap chosen at 10 MB — real per-segment GeoJSON is <100 KB; even the
longest realistic multi-day route stays well under 2 MB. Above 10 MB
is upstream bug or abuse; we'd rather error.
Applied to both \`fetchSegment\` (GeoJSON path) and \`computeSegmentGpx\`
(GPX path).
Tests: 3 cases (within cap, content-length over cap, streamed body
mid-cap abort).
Addresses planner-audit #8. `listSessions()` had no LIMIT — every call
to `GET /api/sessions` returned every non-closed session and did a
full table scan ordered by last_activity. On a long-running planner
host that's both a memory cliff and a query-plan footgun.
Now: defaults to 50 rows, clamps to [1, 200], accepts `?limit=` for
pagination-light scenarios. `sessions_last_activity_idx` (added in
#438) backs the ORDER BY + LIMIT efficiently.
The Save-to-Journal flow had the browser fetch the journal with a
\`Bearer \${callbackToken}\` header. The JWT was visible in DevTools,
exfiltratable via any XSS or browser extension, and the planner's
\`loader\` shipped it down to the client as part of the page payload.
Now:
- **New action**: \`POST /api/save-to-journal\` (\`routes/api.save-to-journal.ts\`).
Body: \`{ sessionId, gpx }\`. The action loads \`callbackUrl\` +
\`callbackToken\` from \`planner.sessions\` (set at /new time when the
user came from the journal), POSTs to the journal server-to-server
with the Bearer, and forwards the response.
- **\`SaveToJournalButton\`**: drops the \`callbackUrl\` + \`callbackToken\`
props. Takes \`sessionId\` only and POSTs to the planner action.
- **\`session.\$id.tsx\` loader**: stops returning \`callbackUrl\` /
\`callbackToken\` to the client. Returns a single \`hasJournalCallback\`
boolean so the button still knows whether to render.
- **\`SessionView\`**: same prop simplification.
Trust model is unchanged: the same \`sessionId\` that grants Yjs
membership grants save authority. Knowing the URL = ability to act.
The action only adds a server-side hop so the JWT never reaches
browser JS.
Phase B (jti single-use enforcement on the journal side) follows in
a separate PR — needs a journal DB column + verifier change.
Full repo: pnpm typecheck / lint / test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses planner-audit #1 (CRITICAL — WebSocket joins unauthenticated)
with the narrow fix. The upgrade handler used to immediately
\`handleUpgrade\` for any \`/sync/<id>\` path, then \`getOrLoadDoc\` *created*
a Y.Doc on demand. An attacker connecting to random sessionIds could:
- exhaust process memory by spinning up Y.Doc objects for sessions
that don't exist (no DB row backed them), and
- resurrect closed/expired sessions by reconnecting (closed rows
were still cacheable to \`docs\`).
Now: the upgrade handler calls \`getSession(sessionId)\` first
(\`SELECT \\* FROM planner.sessions WHERE id = ? AND closed = false\`).
Missing or closed → \`socket.destroy()\` before handshake. DB unreachable
also closes — fail closed; the client reconnects after backoff.
Costs one DB roundtrip per upgrade. Upgrades are rare vs message
volume, so the overhead is negligible.
Note: we are NOT adding a join token. The planner is anonymous-by-design
(see CLAUDE.md \"sessions are anonymous and ephemeral\") — knowing the
URL still equals membership. The check here only guards against
attacker-supplied sessionIds with no backing row.
Full repo: pnpm typecheck / lint / test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses planner audit #5 — a connected client could feed unlimited
waypoints / no-go / notes into the Yjs doc, growing the in-memory map
and the persisted \`sessions.yjs_state\` blob until OOM or DB blowout.
Two guards:
1. **Per-frame**: \`MAX_MESSAGE_BYTES = 256 KB\`. Anything bigger arrives
from a buggy or hostile client — a single waypoint add is hundreds
of bytes. Oversized frame → \`ws.close(1008)\` and drop the message.
2. **Per-doc**: \`MAX_DOC_BYTES = 5 MB\`. After a sync-message apply
succeeds, recompute \`Y.encodeStateAsUpdate(doc).byteLength\`. If it
crossed the cap, mark the session \"quarantined\": close every
connected client (\`1008 policy violation\`), and short-circuit
subsequent handleMessage calls so no further bytes can be applied
and the debounced save can't write an oversized blob to Postgres.
The 5 MB limit is generous — a typical multi-day route's serialized
state is well under 100 KB. The cap exists to make abuse expensive,
not to constrain real use.
Exports \`MAX_MESSAGE_BYTES\`, \`MAX_DOC_BYTES\`, and \`docByteSize\`
for testing. Tests cover:
- constants are positive and ordered
- docByteSize is 0 for unknown sessions
- a realistic 500-waypoint route stays well under the cap
- ~6MB of garbage in a Y.Text trips the cap (smoke test for the guard)
Full repo: pnpm typecheck / lint / test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses planner audit #3 (SSRF via callbackUrl) and #7 (URL-param
size). Two attack surfaces hardened:
1. /new loader — \`callback\`, \`token\`, \`returnUrl\`, \`gpx\` query
params now validated:
- callbackUrl: must be a valid absolute http(s) URL ≤ 2048 chars.
If \`PLANNER_CALLBACK_ALLOWED_HOSTS\` is set (comma-separated),
the host must match — defense-in-depth SSRF guard for self-
hosted instances. Unset = no allowlist (dev / open self-host).
- token: max 2048 chars.
- returnUrl: must be a same-origin path or absolute http(s) URL
≤ 2048 chars. Rejects \`javascript:\`, \`data:\`, and
protocol-relative \`//host\` (which would resolve to a remote
origin on HTTPS pages).
- gpx: ≤ 2 MB encoded.
Invalid input throws 400 from the loader.
2. /session/:id default-export component — \`waypoints\`, \`noGoAreas\`,
\`notes\`, \`returnUrl\` URL params now bounded before
\`JSON.parse\` / use:
- waypoints / noGoAreas: ≤ 50KB each; over-cap returns undefined
(component starts with empty initial state, same as malformed).
- notes: ≤ 10KB.
- returnUrl: ≤ 2KB + same scheme rules as #1.
Pulled the URL validation into \`lib/url-validation.server.ts\` so
both routes (and any future caller) share the same rules.
Tests: \`url-validation.server.test.ts\` (14 cases — schemes,
allowlist, length caps, protocol-relative guards, env parsing).
Full repo: pnpm typecheck / lint / test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors journal PR #5. `fetchSegment()` and `computeSegmentGpx()` in
lib/brouter.ts called `fetch()` with no AbortSignal — a hung or slow
BRouter would stall the request handler indefinitely.
New `lib/http.server.ts::fetchWithTimeout()` (same as the journal's
helper) wraps fetch with `AbortSignal.timeout(30_000)`, composable
with a caller-supplied signal via `AbortSignal.any()`.
Tests: lib/http.server.test.ts (2 cases — timeout abort, happy path).
Self-hosted instances no longer inherit the trails.cool flagship Sentry
DSNs. Code paths now read DSN strictly from env — unset = no Sentry
init, no events sent.
Flagship continues to report unchanged: cd-apps.yml and cd-staging.yml
inject the public DSNs as workflow env vars, which feed into:
- runtime via \`infrastructure/app.env\` / \`staging.env\`
(\`SENTRY_DSN_JOURNAL\` / \`SENTRY_DSN_PLANNER\` → compose env per service)
- build time via docker build-arg \`VITE_SENTRY_DSN\` (journal only;
planner has no client Sentry init).
Sentry DSNs are public-by-design (transmitted unencrypted from the
client JS bundle), so embedding them as plaintext workflow env vars
is no worse than the runtime exposure. Forks should replace these with
their own DSNs or remove the workflow env lines to ship Sentry-free.
Files changed:
- apps/journal/server.ts: \`process.env.SENTRY_DSN\` only, no fallback
- apps/planner/server.ts: same
- apps/journal/app/lib/sentry.client.ts: \`import.meta.env.VITE_SENTRY_DSN\`
only; falsy = skip init
- apps/journal/Dockerfile: new \`ARG VITE_SENTRY_DSN\` baked into build
- infrastructure/docker-compose.yml: pass \`SENTRY_DSN\` per service
- infrastructure/docker-compose.staging.yml: same
- .github/workflows/cd-apps.yml: workflow env + build-arg + app.env echo
- .github/workflows/cd-staging.yml: same
Full repo: pnpm typecheck, pnpm lint, pnpm test, journal build all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors #422 (fail-loud DB URL) and #430 (health-check pool reuse) for
the planner app, which still had:
- two \`process.env.DATABASE_URL ?? \"postgres://trails:trails@localhost:5432/trails\"\`
call sites that would silently boot a misconfigured prod against
localhost
- a /health handler that opened a fresh postgres client + connection on
every probe (no pool, per-call TCP/TLS handshake)
Both now route through @trails-cool/db's \`getDatabaseUrl()\` (refuses to
start in production if unset / matches the dev default; E2E=true is the
opt-out) and a module-level singleton client (max: 2, idle_timeout: 30).
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors PR #429 for the planner app. Each HTTP request now gets a
requestId (inbound X-Request-Id honored, otherwise a fresh UUID),
echoed on the response, and propagated to every downstream log call via
AsyncLocalStorage + pino's \`mixin\`.
Tests: planner logger.server.test.ts (2 cases, same shape as the
journal version).
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Sentry DSNs were hardcoded in journal/server.ts, planner/server.ts,
and journal/app/lib/sentry.client.ts. Self-hosted instances inheriting
the trails.cool flagship DSN would silently ship their errors to our
Sentry account.
Now each init site reads its DSN from env:
- journal/server.ts: SENTRY_DSN (server runtime env)
- planner/server.ts: SENTRY_DSN (server runtime env)
- journal/app/lib/sentry.client.ts: VITE_SENTRY_DSN (build-time bake)
The flagship DSN is kept as the fallback so the production deploy keeps
reporting without an infra change — but self-hosters can:
- set SENTRY_DSN=\"\" / VITE_SENTRY_DSN=\"\" to ship their own builds
without Sentry, or
- set SENTRY_DISABLED=true to skip init entirely at runtime, or
- set SENTRY_DSN=/VITE_SENTRY_DSN= to their own DSN.
Follow-up: a future PR can remove the hardcoded fallbacks once
infrastructure/docker-compose.yml + the cd-apps.yml workflow are wired
to pass SENTRY_DSN explicitly. That requires SOPS edits + workflow
changes I want isolated from this purely-code change.
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce waypoint-ymap.ts with typed helpers so all Yjs↔Waypoint
conversions go through one place. New Waypoint fields now only need
to be added once rather than in every consumer.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- use-yjs: seed note from initialWaypoints on session open
- api.sessions: include note in initialWaypoints type
- SaveToJournalButton: include note in GPX waypoints on save
- Journal routes.: map and display waypoint notes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- `note` field on Waypoint type, stored in Yjs Y.Map and exported as
`<desc>` inside `<wpt>` in GPX
- Inline textarea note editing in WaypointSidebar with auto-resize,
character counter (500 max), save-on-blur, Escape cancel
- Note indicator dot on map markers; note tooltip on hover
- `useNearbyPois` hook: fetches POIs within 500m of selected waypoint
via Overpass proxy, 500ms debounce, AbortController, 60s rate-limit
suppression
- NearbyPoiMarkers component: renders POI markers on map for selected
waypoint with snap-to-POI on click
- Nearby section in WaypointSidebar: list with snap buttons, spinner,
empty/rate-limited states, "Show more" toggle (5 → all)
- Unit tests for fetchNearbyPois bbox geometry and snap-to-POI Yjs
transaction behaviour
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Waypoints snapped to OSM POIs in the Planner now carry their metadata all
the way through to the Journal:
- Extend Waypoint type with osmId and poiTags fields
- Extract osmId/poiTags from Yjs Y.Map in ExportButton and SaveToJournalButton
- Encode POI metadata as <trails:poi> extensions in GPX <wpt> elements
- Parse <trails:poi> extensions back in the GPX parser
- Display phone, website, opening hours, address on Journal route detail
- E2E test for the full roundtrip; seed endpoint now defaults to public visibility
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Vitest browser mode writes ephemeral screenshot attachments here during
test runs; only the __screenshots__/ reference snapshots belong in the repo.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Generate baseline screenshots (macOS/Chromium) for the elevation-chart
visual regression suite. The CI update-visual-snapshots workflow will
overwrite these with Linux variants on the first run.
Also fix the test:visual:update script: --update-snapshots is not a
valid Vitest 4 flag; the correct short form is -u.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add @vitest/browser-playwright; use playwright() factory (not string)
- Import page from vitest/browser (not deprecated @vitest/browser/context)
- Add afterEach(cleanup) so each test gets a fresh DOM
- Use oxc transform for JSX instead of esbuild (avoids Vite 8 warning)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extract shared fitToGpx into connected-services/fit.ts (FIT is a Garmin
open standard used by Wahoo, Coros, Garmin — not provider-specific)
- Add importActivity() to sync/imports.server.ts so providers call one
function instead of createActivity + recordImport separately; eliminates
direct dependency on activities.server.ts from provider adapters
- Update wahoo importer + webhook to use both shared helpers
- Extract useHostElection(yjs) hook from useRouting so host election is
independently testable without mounting the full routing stack
- Add setDb() to journal db.ts for module-level injection in unit tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The deps stage was missing the COPY for packages/fit/package.json, so
@trails-cool/fit was absent from the workspace at install time and
@garmin/fitsdk could not be resolved during the journal build.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The compose healthcheck for both apps was `curl -sf http://localhost:.../health`,
but node:25-slim (Debian trixie-slim) ships neither curl nor wget, so
the check always reported the containers as `unhealthy` even while they
were happily serving 200s. Install curl in both Dockerfiles.
Separately, today's flagship BRouter cleanup (PR #297) left an orphan
`brouter` container behind because cd-infra's `docker compose up -d <list>`
doesn't pass `--remove-orphans`. I had to remove it by hand. Pass the
flag in both cd-infra and cd-apps so future service deletions clean up
on the next deploy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Moving one waypoint previously re-fetched all N-1 segments of the route
through /api/route every time, because the server-side computeRoute
recomputes the whole route from scratch. Under rapid edits this piles up
on BRouter's thread pool and wastes BRouter+Planner cycles on segments
the caller already has.
This change keeps a host-local LRU segment cache in the Planner tab,
keyed by (from, to, profile, noGoHash). On each computeRoute:
- build the pair list from the current waypoints
- split into cached vs. missing by looking up each pair key
- if any missing, POST { pairs: missing, … } to the new
/api/route-segments endpoint and populate the cache with its ordered
raw BRouter responses
- merge cache-ordered segments into an EnrichedRoute client-side via
mergeGeoJsonSegments (moved to a pure, isomorphic route-merge.ts)
- write to yjs.routeData exactly as before
Typical drag of one waypoint in a 10-waypoint route drops from 9 BRouter
fetches to 2. A session re-entering a cached arrangement (undo/redo)
short-circuits the server entirely. Profile/no-go changes invalidate all
keys and trigger a full refetch, matching prior behavior.
Yjs stays out of the cache: segments are bulky and CRDT sync of large
blobs hurts more than it helps. The elected host owns the cache; on host
handover the new host pays one full recompute.
Backwards-compat: /api/route is unchanged for external callers (e2e
integration tests, the Journal demo-bot's format=gpx path). Server-side
it now delegates to the shared fetchSegments helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a user rapidly moves waypoints in the Planner, BRouter's
thread-priority watchdog kills older in-flight requests to free a thread
for newer ones, surfacing to the user as "operation killed by
thread-priority-watchdog" errors.
Two fixes:
1. Bump the BRouter thread pool on the dedicated host from 4 to 16.
`BROUTER_THREADS` is now a Dockerfile ENV (default 4 for
flagship/CI/dev) overridden to 16 in `infrastructure/brouter-host/`.
The host has 32 GB of RAM and idle cores; 4 threads was a flagship-era
constraint that no longer applies.
2. Cancel the previous in-flight `/api/route` request when a new one
starts. `use-routing.ts` now keeps an `AbortController` in a ref,
aborts it at the top of each `computeRoute`, and bails quietly on
`AbortError` so the newer call drives state.
Together these eliminate both the "real" contention (at the BRouter
level) and the wasted work (BRouter computing a route the client no
longer cares about).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands sections 3-5 of the relocate-brouter-to-dedicated-host change:
everything needed to run BRouter on the dedicated Hetzner Robot host
and have the Planner talk to it with the shared-secret header. Does
NOT flip the cutover — the flagship BRouter stays warm during soak.
## BRouter host compose (section 3)
New `infrastructure/brouter-host/` — a standalone compose project that
runs as the `trails` user on `ullrich.is`:
- `docker-compose.yml` — brouter + caddy sidecar. BRouter has no host
port; caddy binds only to `10.0.1.10:17777` (vSwitch IP). Every
service explicitly overrides the host's default Loki logging driver
to `json-file` so logs don't leak to the operator's personal Loki.
- `Caddyfile` — single-purpose reverse proxy that requires
`X-BRouter-Auth: ${BROUTER_AUTH_TOKEN}` on every request. `auto_https
off` (vSwitch-only); default access log format omits request
headers, so the token is never written to disk.
- `download-segments.sh` — crawls brouter.de, pulls planet-wide RD5
tiles via `wget -N` (incremental). Idempotent, safe to cron.
- `README.md` — one-shot provisioning + token rotation + rollback
notes.
`docker/brouter/Dockerfile` is patched to honor `JAVA_OPTS` (was
hardcoded `-Xmx1024M` in CMD). Default keeps the flagship's current
heap; compose on the dedicated host overrides to `-Xmx8g` for planet
scale on a 32 GB box.
## Planner shared-secret header (section 4)
`apps/planner/app/lib/brouter.ts`:
- Module-level guard: throws at startup in production if
`BROUTER_AUTH_TOKEN` is unset.
- `authHeaders()` helper (reads env at call time, so tests can
`vi.stubEnv` without module reset).
- Header attached on both `computeRoute` (per-segment) and
`computeSegmentGpx`.
3 new unit tests cover header attachment + the no-token path.
`infrastructure/docker-compose.yml` passes `BROUTER_AUTH_TOKEN` to
the Planner service, and makes `BROUTER_URL` overridable via SOPS so
the cutover is a one-variable flip.
## cd-brouter workflow (section 5)
Rewritten to deploy to the dedicated host:
- SSH as `trails@${BROUTER_DEPLOY_HOST}` on port
`${BROUTER_DEPLOY_SSH_PORT}` (2232) using
`${BROUTER_DEPLOY_SSH_KEY}`.
- Decrypts SOPS, extracts ONLY `BROUTER_AUTH_TOKEN` into a `.env`
file, scp'd alongside the compose project.
- `paths:` trigger now includes `infrastructure/brouter-host/**`.
- Segment download is NOT run here — first-time seed is a manual
operator step (multi-hour). Routine re-runs are cron-able on the
dedicated host.
- Grafana annotation step preserved (reaches flagship Grafana as
before).
## What's NOT here
- `brouter:` service on the flagship is intentionally left in place
(removed in section 7.5 after the 48 h soak window post-cutover).
- Observability (section 6) — Prometheus scrape + Loki shipping from
the dedicated host — comes in a follow-up PR.
- Cutover itself (section 7) — flip `BROUTER_URL`, verify, remove the
flagship brouter — is an operator action gated on first-time
provisioning + smoke testing.
## Verification
`pnpm typecheck && pnpm lint && pnpm test` all clean; planner build
passes (the CI regression from #286 was fixed in #290).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Route files in React Router only get server-code stripped from a
specific set of exports (loader/action/middleware/headers). The
overpass failover work added a `fetchWithFailover` named export to
`app/routes/api.overpass.ts` for unit tests, but that export pulls
in `metrics.server` (prom-client registry, etc.) — which then
leaks into the client bundle and fails the production build with:
[plugin react-router:dot-server]
Error: Server-only module referenced by client
'app/lib/metrics.server' imported by route 'app/routes/api.overpass.ts'
Move the upstream fetch logic and its config constants into
`app/lib/overpass.server.ts`. The `.server.ts` naming makes the
server-only boundary explicit, and tests now import from the lib
instead of from the route. The route keeps only its `action` export
(plus the per-instance LRU cache, which is internal to the module
and not exported).
Tests: 85 passing (no behavior changes). Build: planner + journal
both succeed. Typecheck + lint clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The gauge was maintained by conditional inc on "this sessionId isn't
in the in-memory `docs` Map yet" and dec on "last client for the
sessionId left." Because `docs` caches entries indefinitely — even
after every client disconnects — a reconnect found the doc still
cached, skipped the inc, then decremented again on the next
disconnect. Every reconnect-after-last-disconnect leaked one
decrement. After a day of normal reload / wifi-blip / tab-hibernate
patterns the gauge hit -182.
Replace the transitions with a `set()` derived from the live `conns`
map: count distinct session ids with at least one open WebSocket.
Self-correcting, no drift possible.
Extract `countActiveSessions()` as a pure helper + unit tests
including a regression case that simulates repeated reconnect/
disconnect cycles on one session and asserts the count never goes
negative.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Today both proxies are effectively open to anyone who can set an
Origin header for trails.cool — a third party can use us as a free
BRouter/Overpass relay. Require a live planner session on every call
so abuse traffic costs the scraper a session row (observable,
revocable) before they can issue a single query.
Server:
- New `requireSession(id)` helper — returns the session row or a 401
Response. Reused by both route handlers.
- `/api/route`: `sessionId` in body is now required and verified;
rate-limit key always falls back to the session id.
- `/api/overpass`: new `X-Trails-Session` header, verified. Header
keeps the session out of the request body so the body-keyed cache
is unaffected.
Client plumbing:
- `useRouting(yjs, sessionId)` — sessionId goes into the /api/route
body.
- `usePois(sessionId)` → `queryPois(..., sessionId)` → `X-Trails-Session`
on the proxy call.
- `PlannerMap` + `YjsDebugPanel` gain a `sessionId` prop from
`SessionView`.
Journal server-to-server:
- Demo-bot and `/api/v1/routes/compute` now POST `/api/sessions` to
mint a throwaway planner session, then cite it on the forwarded
`/api/route` call. Planner's `expire-sessions` cron cleans these up
(7d window) so nothing needs explicit teardown.
Tests:
- 5 unit tests for `requireSession` covering missing / empty /
non-string / unknown-session / valid-session cases.
- Two integration E2E tests document the 401 for missing session on
each proxy.
- Pre-existing `/api/route` integration tests updated to mint a
session first.
Caveat: existing browser tabs lose their /api/route ability until
reload (the old JS doesn't know to send sessionId). Acceptable for
an anonymous planner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Public Overpass instances go bad without warning — we just lived
through a day where overpass.private.coffee (our single upstream)
returned 504s after 60s while lz4.overpass-api.de served the same
query in 0.6s. Survive that by trying a list of upstreams in order
until one responds usefully.
Server changes:
- OVERPASS_URLS env — comma-separated list, tried in order. Falls
back to OVERPASS_URL for backward compat, then to a built-in
default pair (lz4.overpass-api.de, overpass-api.de).
- Per-upstream timeout of 10s via AbortSignal.timeout, so a saturated
instance fails over in seconds, not minutes.
- Client abort propagation via AbortSignal.any — if the browser
cancels (e.g. user panned the map), the in-flight upstream fetch is
aborted too. Stops wasting upstream capacity on requests nobody
wants.
- Rate-limited-body detection (Overpass returns 200 with a
`rate_limited` marker when throttling) triggers failover.
- Per-upstream labels on overpass_upstream_requests_total and
overpass_upstream_duration_seconds so the dashboard can break out
health + latency by instance. Histogram buckets extended to 30s.
Compose: OVERPASS_URLS passthrough with the default pair hard-wired,
overridable via SOPS.
Tests: 8 cases covering the new fetchWithFailover helper — happy
path, 5xx failover, rate_limited failover, network error failover,
timeout tagging, all-upstreams-fail, client abort stops the loop,
per-attempt latency observation.
Follow-ups left out of scope:
- Client-side debounce (UI concern).
- Moving `.observe()` after body read for accuracy in measurement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The planner is the BRouter client in this architecture — route
compute, rate limiting, and the eventual move of BRouter off-box all
live there. The journal had two places that called BRouter directly
(demo-bot and /api/v1/routes/compute) and both were broken on prod
because the journal service has no BROUTER_URL.
Fix:
- Extend the planner's /api/route with an optional `format: "gpx"`
that returns BRouter's raw GPX (for server-to-server callers that
don't need the way-tag enriched GeoJSON).
- Point the journal demo-bot at `PLANNER_URL/api/route` with
`format: "gpx"` instead of calling BRouter directly.
- Point /api/v1/routes/compute at `PLANNER_URL/api/route` instead of
BRouter directly.
Journal no longer needs BROUTER_URL at all. When BRouter moves to a
dedicated host later, only the planner changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The same-origin check compared the browser's Origin header against
new URL(request.url), but inside the Planner container request.url
is http://planner:3001/... while the browser sends Origin
https://planner.trails.cool — so production always 403'd.
Trust Caddy's X-Forwarded-Host and X-Forwarded-Proto headers when
present (both are set by Caddy's reverse_proxy directive by default)
to reconstruct the external origin the browser actually connected to.
Falls back to request.url for dev and any direct (non-proxied) access.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Near-identical viewports from different collaborators (slightly
different pan/zoom because Yjs awareness doesn't enforce pixel-exact
alignment) previously produced byte-different Overpass queries and
therefore different server-side cache keys. Each client missed cache
and triggered its own upstream fetch.
buildQuery now quantizes the bbox to a 0.01° grid (~1 km), expanding
outward (south/west floor, north/east ceil) so the viewport is always
covered, and formats coords with 3 decimals for a stable string.
Trade-offs:
- Slightly larger query bbox: ≤ one grid cell (≈ 1 km) of padding on
each side. For a zoom-12 viewport (~11 km wide) that's <10% extra
data; bounded by the existing `[maxsize:1048576]` + `out 100` limits.
- Coordinates in the query are now 3-decimal strings regardless of the
caller's precision — existing buildQuery test updated to match.
Two new tests cover the cache-alignment property and the outward-
expansion invariant. Hit ratio impact will be visible on the Grafana
"Overpass Cache Hit Ratio" stat added in the prior PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds four metrics surfaced via the existing /metrics endpoint:
- overpass_cache_events_total{result=hit|miss|coalesced}
- overpass_cache_size (gauge)
- overpass_upstream_duration_seconds (histogram)
- overpass_upstream_requests_total{status} — covers 2xx/4xx/5xx and
an explicit "error" label for network-level failures
Grafana dashboards/planner.json gets a new row:
- Upstream health (5m 2xx success ratio, red <90%, green >99%)
- Cache hit ratio
- Cache size
- Upstream p95 latency
Plus timeseries panels for cache event breakdown, upstream status
over time, and p50/p95/p99 upstream latency.
The success-rate formula uses clamp_min(..., 1) on the denominator so
it doesn't NaN when traffic is zero.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds apps/planner/app/routes/api.overpass.ts — a same-origin, rate-limited
server-side proxy that forwards Overpass QL to the upstream endpoint
configured via OVERPASS_URL (default: overpass.private.coffee).
Motivation:
- private.coffee's best-practices require a meaningful User-Agent
identifying the project. Browsers cannot set User-Agent on fetch()
(forbidden header), so the request has to originate server-side.
- Collaborative sessions commonly have N clients pan/zoom the same map,
so the same bbox query arrives multiple times within seconds.
- Setting up the proxy now lets us swap OVERPASS_URL to a self-hosted
Overpass later without client changes.
What the proxy does:
- Sets User-Agent "trails.cool Planner (https://trails.cool; legal@trails.cool)"
- Enforces same-origin via the Origin header
- Rate-limits per client IP (120 req/min)
- In-memory LRU cache of upstream responses keyed on the form-encoded body
(TTL 10 min, max 200 entries)
- Coalesces concurrent misses for the same key so N simultaneous clients
in one session incur exactly one upstream call
Client change: apps/planner/app/lib/overpass.ts POSTs to /api/overpass
instead of iterating over public Overpass endpoints. Fallback list
removed; resilience is now the proxy's responsibility.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ullrich Schäfer
dependabot[bot]
stigi