From c49047fd33b9256820520e46e246c0f09f464b5c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ullrich=20Sch=C3=A4fer?= Date: Thu, 23 Apr 2026 22:52:48 +0200 Subject: [PATCH] BRouter host compose + Planner auth + cd-brouter rewrite MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lands sections 3-5 of the relocate-brouter-to-dedicated-host change: everything needed to run BRouter on the dedicated Hetzner Robot host and have the Planner talk to it with the shared-secret header. Does NOT flip the cutover — the flagship BRouter stays warm during soak. ## BRouter host compose (section 3) New `infrastructure/brouter-host/` — a standalone compose project that runs as the `trails` user on `ullrich.is`: - `docker-compose.yml` — brouter + caddy sidecar. BRouter has no host port; caddy binds only to `10.0.1.10:17777` (vSwitch IP). Every service explicitly overrides the host's default Loki logging driver to `json-file` so logs don't leak to the operator's personal Loki. - `Caddyfile` — single-purpose reverse proxy that requires `X-BRouter-Auth: ${BROUTER_AUTH_TOKEN}` on every request. `auto_https off` (vSwitch-only); default access log format omits request headers, so the token is never written to disk. - `download-segments.sh` — crawls brouter.de, pulls planet-wide RD5 tiles via `wget -N` (incremental). Idempotent, safe to cron. - `README.md` — one-shot provisioning + token rotation + rollback notes. `docker/brouter/Dockerfile` is patched to honor `JAVA_OPTS` (was hardcoded `-Xmx1024M` in CMD). Default keeps the flagship's current heap; compose on the dedicated host overrides to `-Xmx8g` for planet scale on a 32 GB box. ## Planner shared-secret header (section 4) `apps/planner/app/lib/brouter.ts`: - Module-level guard: throws at startup in production if `BROUTER_AUTH_TOKEN` is unset. - `authHeaders()` helper (reads env at call time, so tests can `vi.stubEnv` without module reset). - Header attached on both `computeRoute` (per-segment) and `computeSegmentGpx`. 3 new unit tests cover header attachment + the no-token path. `infrastructure/docker-compose.yml` passes `BROUTER_AUTH_TOKEN` to the Planner service, and makes `BROUTER_URL` overridable via SOPS so the cutover is a one-variable flip. ## cd-brouter workflow (section 5) Rewritten to deploy to the dedicated host: - SSH as `trails@${BROUTER_DEPLOY_HOST}` on port `${BROUTER_DEPLOY_SSH_PORT}` (2232) using `${BROUTER_DEPLOY_SSH_KEY}`. - Decrypts SOPS, extracts ONLY `BROUTER_AUTH_TOKEN` into a `.env` file, scp'd alongside the compose project. - `paths:` trigger now includes `infrastructure/brouter-host/**`. - Segment download is NOT run here — first-time seed is a manual operator step (multi-hour). Routine re-runs are cron-able on the dedicated host. - Grafana annotation step preserved (reaches flagship Grafana as before). ## What's NOT here - `brouter:` service on the flagship is intentionally left in place (removed in section 7.5 after the 48 h soak window post-cutover). - Observability (section 6) — Prometheus scrape + Loki shipping from the dedicated host — comes in a follow-up PR. - Cutover itself (section 7) — flip `BROUTER_URL`, verify, remove the flagship brouter — is an operator action gated on first-time provisioning + smoke testing. ## Verification `pnpm typecheck && pnpm lint && pnpm test` all clean; planner build passes (the CI regression from #286 was fixed in #290). Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/cd-brouter.yml | 84 +++++++++++------ apps/planner/app/lib/brouter.test.ts | 81 +++++++++++++++- apps/planner/app/lib/brouter.ts | 24 ++++- docker/brouter/Dockerfile | 15 +-- infrastructure/brouter-host/Caddyfile | 47 ++++++++++ infrastructure/brouter-host/README.md | 94 +++++++++++++++++++ .../brouter-host/docker-compose.yml | 78 +++++++++++++++ .../brouter-host/download-segments.sh | 80 ++++++++++++++++ infrastructure/docker-compose.yml | 11 ++- .../tasks.md | 41 +++++--- 10 files changed, 500 insertions(+), 55 deletions(-) create mode 100644 infrastructure/brouter-host/Caddyfile create mode 100644 infrastructure/brouter-host/README.md create mode 100644 infrastructure/brouter-host/docker-compose.yml create mode 100755 infrastructure/brouter-host/download-segments.sh diff --git a/.github/workflows/cd-brouter.yml b/.github/workflows/cd-brouter.yml index bc2452f..b6ec12a 100644 --- a/.github/workflows/cd-brouter.yml +++ b/.github/workflows/cd-brouter.yml @@ -5,6 +5,7 @@ on: branches: [main] paths: - "docker/brouter/**" + - "infrastructure/brouter-host/**" workflow_dispatch: {} concurrency: @@ -36,13 +37,63 @@ jobs: ghcr.io/trails-cool/brouter:${{ github.sha }} deploy: - name: Deploy BRouter + name: Deploy BRouter to dedicated host needs: [build] runs-on: ubuntu-latest + environment: infra steps: - uses: actions/checkout@v6 - - name: Deploy via SSH + - name: Decrypt shared secret + run: | + curl -sLO https://github.com/getsops/sops/releases/download/v3.9.4/sops-v3.9.4.linux.amd64 + chmod +x sops-v3.9.4.linux.amd64 + # Extract ONLY BROUTER_AUTH_TOKEN from secrets.app.env — the + # rest of that file is app-only and has no business reaching + # the BRouter host. + SOPS_AGE_KEY="${{ secrets.AGE_SECRET_KEY }}" ./sops-v3.9.4.linux.amd64 -d infrastructure/secrets.app.env \ + | grep '^BROUTER_AUTH_TOKEN=' > infrastructure/brouter-host/.env + chmod 0600 infrastructure/brouter-host/.env + + - name: Copy compose project to dedicated host + uses: appleboy/scp-action@v1 + with: + host: ${{ secrets.BROUTER_DEPLOY_HOST }} + username: trails + port: ${{ secrets.BROUTER_DEPLOY_SSH_PORT }} + key: ${{ secrets.BROUTER_DEPLOY_SSH_KEY }} + source: "infrastructure/brouter-host/docker-compose.yml,infrastructure/brouter-host/Caddyfile,infrastructure/brouter-host/download-segments.sh,infrastructure/brouter-host/.env" + target: /home/trails/brouter + strip_components: 2 + + - name: Pull image and restart containers + uses: appleboy/ssh-action@v1 + with: + host: ${{ secrets.BROUTER_DEPLOY_HOST }} + username: trails + port: ${{ secrets.BROUTER_DEPLOY_SSH_PORT }} + key: ${{ secrets.BROUTER_DEPLOY_SSH_KEY }} + script: | + set -euo pipefail + cd /home/trails/brouter + + chmod +x download-segments.sh + + # Segment download is explicitly NOT run here — it's + # multi-hour and idempotent. Seeding the segments directory + # is a one-shot operator task (see brouter-host/README.md). + # Run `~/brouter/download-segments.sh` by hand (or via cron) + # to refresh. + if [ ! -d segments ] || [ -z "$(ls -A segments 2>/dev/null)" ]; then + echo "WARNING: segments/ is empty. BRouter will start but return 404 until segments are seeded." + mkdir -p segments + fi + + docker compose pull + docker compose up -d --remove-orphans + docker compose ps + + - name: Annotate deploy in flagship Grafana uses: appleboy/ssh-action@v1 with: host: ${{ secrets.DEPLOY_HOST }} @@ -50,34 +101,7 @@ jobs: key: ${{ secrets.DEPLOY_SSH_KEY }} script: | cd /opt/trails-cool - - # Download BRouter segments for Europe (W10-E40, N35-N70) - mkdir -p /opt/trails-cool/segments - EUROPE_TILES=" - W10_N50 W10_N55 W10_N60 W10_N65 - W5_N35 W5_N40 W5_N45 W5_N50 W5_N55 W5_N60 W5_N65 - E0_N35 E0_N40 E0_N45 E0_N50 E0_N55 E0_N60 E0_N65 E0_N70 - E5_N35 E5_N40 E5_N45 E5_N50 E5_N55 E5_N60 E5_N65 E5_N70 - E10_N35 E10_N40 E10_N45 E10_N50 E10_N55 E10_N60 E10_N65 E10_N70 - E15_N35 E15_N40 E15_N45 E15_N50 E15_N55 E15_N60 E15_N65 E15_N70 - E20_N35 E20_N40 E20_N45 E20_N50 E20_N55 E20_N60 E20_N65 E20_N70 - E25_N35 E25_N40 E25_N45 E25_N50 E25_N55 E25_N60 E25_N65 - E30_N35 E30_N40 E30_N45 E30_N50 E30_N55 E30_N60 E30_N65 - E35_N40 E35_N45 E35_N50 E35_N55 E35_N60 - E40_N40 E40_N45 E40_N50 E40_N55 - " - for tile in $EUROPE_TILES; do - [ -f "/opt/trails-cool/segments/${tile}.rd5" ] || \ - wget -q "https://brouter.de/brouter/segments4/${tile}.rd5" -O "/opt/trails-cool/segments/${tile}.rd5" 2>/dev/null || \ - rm -f "/opt/trails-cool/segments/${tile}.rd5" - done - - docker pull ghcr.io/trails-cool/brouter:latest - docker compose up -d brouter - docker image prune -af - - # Annotate deploy in Grafana - GRAFANA_TOKEN=$(grep GRAFANA_SERVICE_TOKEN .env | cut -d= -f2- 2>/dev/null) + GRAFANA_TOKEN=$(grep GRAFANA_SERVICE_TOKEN .env | cut -d= -f2-) if [ -n "$GRAFANA_TOKEN" ]; then docker compose exec -T grafana curl -sf -X POST \ -H "Authorization: Bearer $GRAFANA_TOKEN" \ diff --git a/apps/planner/app/lib/brouter.test.ts b/apps/planner/app/lib/brouter.test.ts index 6b827d6..a902491 100644 --- a/apps/planner/app/lib/brouter.test.ts +++ b/apps/planner/app/lib/brouter.test.ts @@ -1,5 +1,5 @@ -import { describe, it, expect } from "vitest"; -import { mergeGeoJsonSegments } from "./brouter"; +import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; +import { computeRoute, computeSegmentGpx, mergeGeoJsonSegments } from "./brouter"; function makeSegment(coords: number[][], length: number, ascend: number) { return { @@ -221,6 +221,83 @@ describe("highway tag extraction", () => { }); }); +describe("X-BRouter-Auth header", () => { + // Minimal valid response so computeRoute doesn't throw during merge. + const stubGeoJson = JSON.stringify({ + type: "FeatureCollection", + features: [ + { + type: "Feature", + properties: { "track-length": "100", "filtered ascend": "0", "total-time": "10" }, + geometry: { type: "LineString", coordinates: [[13.0, 52.0, 30], [13.1, 52.1, 40]] }, + }, + ], + }); + + let fetchSpy: ReturnType; + + beforeEach(() => { + fetchSpy = vi.fn().mockResolvedValue( + new Response(stubGeoJson, { status: 200, headers: { "content-type": "application/json" } }), + ); + vi.stubGlobal("fetch", fetchSpy); + }); + + afterEach(() => { + vi.unstubAllEnvs(); + vi.unstubAllGlobals(); + }); + + it("attaches X-BRouter-Auth when BROUTER_AUTH_TOKEN is set (computeRoute)", async () => { + vi.stubEnv("BROUTER_AUTH_TOKEN", "test-token-abc"); + + await computeRoute({ + waypoints: [ + { lat: 52.0, lon: 13.0 }, + { lat: 52.1, lon: 13.1 }, + ], + }); + + expect(fetchSpy).toHaveBeenCalledTimes(1); + const [, init] = fetchSpy.mock.calls[0]!; + expect(init?.headers).toMatchObject({ "X-BRouter-Auth": "test-token-abc" }); + }); + + it("omits X-BRouter-Auth when BROUTER_AUTH_TOKEN is unset (dev/test convenience)", async () => { + vi.stubEnv("BROUTER_AUTH_TOKEN", ""); + + await computeRoute({ + waypoints: [ + { lat: 52.0, lon: 13.0 }, + { lat: 52.1, lon: 13.1 }, + ], + }); + + const [, init] = fetchSpy.mock.calls[0]!; + expect(init?.headers).not.toHaveProperty("X-BRouter-Auth"); + }); + + it("attaches X-BRouter-Auth on computeSegmentGpx as well", async () => { + vi.stubEnv("BROUTER_AUTH_TOKEN", "test-token-xyz"); + fetchSpy.mockResolvedValueOnce( + new Response('', { + status: 200, + headers: { "content-type": "application/gpx+xml" }, + }), + ); + + await computeSegmentGpx({ + waypoints: [ + { lat: 52.0, lon: 13.0 }, + { lat: 52.1, lon: 13.1 }, + ], + }); + + const [, init] = fetchSpy.mock.calls[0]!; + expect(init?.headers).toMatchObject({ "X-BRouter-Auth": "test-token-xyz" }); + }); +}); + describe("waypoint to BRouter segments", () => { it("splits N waypoints into N-1 pairs", () => { const waypoints = [ diff --git a/apps/planner/app/lib/brouter.ts b/apps/planner/app/lib/brouter.ts index a4e6810..e5662d7 100644 --- a/apps/planner/app/lib/brouter.ts +++ b/apps/planner/app/lib/brouter.ts @@ -1,5 +1,23 @@ const BROUTER_URL = process.env.BROUTER_URL ?? "http://localhost:17777"; +// The BRouter host's Caddy sidecar requires every request to carry a +// shared-secret header. Missing in production = every request 403s, so +// crash loudly at startup rather than during the first route request. +if (process.env.NODE_ENV === "production" && !process.env.BROUTER_AUTH_TOKEN) { + throw new Error( + "BROUTER_AUTH_TOKEN is required in production. The BRouter Caddy " + + "sidecar rejects unauthenticated requests with 403. Check the " + + "SOPS-encrypted infrastructure/secrets.app.env and the cd-apps " + + "env wiring.", + ); +} + +// Read at call time so tests can stub via vi.stubEnv without module reset. +function authHeaders(): HeadersInit { + const token = process.env.BROUTER_AUTH_TOKEN; + return token ? { "X-BRouter-Auth": token } : {}; +} + export interface NoGoArea { points: Array<{ lat: number; lon: number }>; } @@ -77,7 +95,7 @@ export class BRouterError extends Error { } async function fetchSegment(url: string): Promise> { - const response = await fetch(url); + const response = await fetch(url, { headers: authHeaders() }); if (!response.ok) { const body = await response.text(); throw new BRouterError(body.trim(), response.status); @@ -310,7 +328,9 @@ export async function computeSegmentGpx(request: { const nogoParam = request.noGoAreas?.length ? noGoAreasToParam(request.noGoAreas) : undefined; if (nogoParam) params.set("polygons", nogoParam); - const resp = await fetch(`${BROUTER_URL}/brouter?${params}`); + const resp = await fetch(`${BROUTER_URL}/brouter?${params}`, { + headers: authHeaders(), + }); if (!resp.ok) { const body = await resp.text(); throw new BRouterError(body.trim(), resp.status); diff --git a/docker/brouter/Dockerfile b/docker/brouter/Dockerfile index 13a2cc1..467b636 100644 --- a/docker/brouter/Dockerfile +++ b/docker/brouter/Dockerfile @@ -34,10 +34,13 @@ RUN for f in /data/profiles/*.brf; do \ USER app EXPOSE 17777 +# JAVA_OPTS can be overridden at runtime (e.g., `-Xmx8g` for planet-scale). +# The default keeps the flagship's single-instance footprint small. +ENV JAVA_OPTS="-Xmx1024M -Xms256M -Xmn64M" + # BRouter server: -CMD ["java", "-Xmx1024M", "-Xms256M", "-Xmn64M", \ - "-DmaxRunningTime=300", \ - "-cp", "brouter.jar", \ - "btools.server.RouteServer", \ - "/data/segments", "/data/profiles", "/data/profiles", \ - "17777", "4"] +# Shell form so $JAVA_OPTS expands at container start. +CMD java $JAVA_OPTS -DmaxRunningTime=300 -cp brouter.jar \ + btools.server.RouteServer \ + /data/segments /data/profiles /data/profiles \ + 17777 4 diff --git a/infrastructure/brouter-host/Caddyfile b/infrastructure/brouter-host/Caddyfile new file mode 100644 index 0000000..b03a095 --- /dev/null +++ b/infrastructure/brouter-host/Caddyfile @@ -0,0 +1,47 @@ +# BRouter Caddy sidecar. +# +# Role: enforce the X-BRouter-Auth shared-secret header so the exposed +# vSwitch port can only be used by the Planner, not by any other process +# that happens to land on the private network. +# +# Note on auth hygiene: by default, Caddy's access log does NOT include +# request headers, so the token value is not written to disk. If you +# ever enable `log { format json }` with `fields_exclude`, make sure +# `X-BRouter-Auth` is NOT added to the log's request headers set. + +{ + # Bind admin socket to localhost only (default behavior, explicit here + # as defense-in-depth). + admin localhost:2019 + # Don't attempt TLS — we only listen on the private vSwitch and + # terminate plain HTTP on :17777. No certs, no Let's Encrypt, no + # leakage of this hostname to the ACME CAs. + auto_https off +} + +:17777 { + # Match requests carrying the correct shared secret. + @authed header X-BRouter-Auth {$BROUTER_AUTH_TOKEN} + + handle @authed { + reverse_proxy brouter:17777 { + # Don't forward the auth header upstream — BRouter doesn't + # use it, and it's cleaner to contain the credential at + # the proxy boundary. + header_up -X-BRouter-Auth + } + } + + # Everything else: blunt 403. + handle { + respond "Forbidden" 403 + } + + # Access log to stdout (structured JSON). Container logs are captured + # by Docker's json-file driver and shipped to Loki by the promtail + # sidecar. Header values are NOT logged unless explicitly configured. + log { + output stdout + format json + } +} diff --git a/infrastructure/brouter-host/README.md b/infrastructure/brouter-host/README.md new file mode 100644 index 0000000..5db13a8 --- /dev/null +++ b/infrastructure/brouter-host/README.md @@ -0,0 +1,94 @@ +# BRouter host compose project + +Runs on a dedicated Hetzner Robot server (currently `ullrich.is`, +private IP `10.0.1.10` over vSwitch #80672), owned by the non-root +`trails` user. Services: + +- **brouter** — the BRouter Java server, planet-scale segments, 8 GB + JVM heap, no public port. +- **caddy** — thin sidecar enforcing the `X-BRouter-Auth` shared-secret + header. Bound to `10.0.1.10:17777` (vSwitch IP only). + +Public ingress is blocked at the host's UFW (port 17777 is only allowed +on the VLAN interface from `10.0.0.2`, the flagship's vSwitch IP). + +## One-time provisioning + +Runs as the `trails` user on the dedicated host. + +```bash +# 1. Land the compose project +cd ~ +git clone https://github.com/trails-cool/trails.git repo +mkdir -p brouter +cp -r repo/infrastructure/brouter-host/* brouter/ +cd brouter + +# 2. Provide the shared secret (matches BROUTER_AUTH_TOKEN in SOPS) +# The CD workflow normally writes this file; for manual bring-up, +# do it yourself. +cat > .env <<'EOF' +BROUTER_AUTH_TOKEN= +EOF +chmod 0600 .env + +# 3. Seed segments (multi-hour, ~60–80 GB) +./download-segments.sh + +# 4. Start services +docker compose pull +docker compose up -d + +# 5. Smoke test from the flagship (over vSwitch) +# Should return 200 with the token, 403 without. +# ssh root@trails.cool 'curl -sSf -H "X-BRouter-Auth: " http://10.0.1.10:17777/brouter?lonlats=... ' +``` + +## Subsequent deploys + +The `cd-brouter` GitHub Actions workflow handles routine updates: +it pulls the latest image, rewrites the compose file + Caddyfile from +the repo, and restarts. + +## Segment updates + +Segments are refreshed by brouter.de weekly. To pull updates: + +```bash +./download-segments.sh +docker compose restart brouter +``` + +Schedule via cron if you want automatic updates (not wired in this repo +yet). + +## Token rotation + +1. Regenerate: `openssl rand -base64 32`. +2. Update SOPS: `sops infrastructure/secrets.app.env` (writer uses the + `sops -d | append | sops -e` pattern via the CD workflow; editing + directly works too). +3. Merge the SOPS change to `main`. +4. `cd-apps` redeploys the Planner (sends the new token outbound). +5. `cd-brouter` redeploys Caddy (matches on the new token). +6. Brief overlap window where Planner sends new token but Caddy still + accepts old: both deploys should fire within a minute of each other, + so a few 403s are the worst case. + +## Rollback + +If BRouter is misbehaving and the flagship BRouter is still warm +(during the 48 h soak window post-cutover), flip `BROUTER_URL` in +`infrastructure/secrets.app.env` back to `http://brouter:17777` and +redeploy the Planner. After the soak window, see the change's +design.md for the longer rollback path. + +## Logging + +The dedicated host's Docker daemon default logging driver is `loki` +(the operator's personal Loki). Our compose file explicitly overrides +each service to `json-file` so logs stay local; a `promtail` sidecar +(section 6.3 of the relocate change) tails them and ships to +trails.cool's Loki over the vSwitch. If you disable that sidecar, the +BRouter logs will NOT flow to trails.cool's Grafana — they'll just +accumulate locally and eventually rotate. diff --git a/infrastructure/brouter-host/docker-compose.yml b/infrastructure/brouter-host/docker-compose.yml new file mode 100644 index 0000000..27edcb6 --- /dev/null +++ b/infrastructure/brouter-host/docker-compose.yml @@ -0,0 +1,78 @@ +# BRouter host compose project — runs on the dedicated Hetzner Robot +# server `ullrich.is` under the `trails` user. See README.md for first-time +# provisioning notes. +# +# Exposed surface: Caddy listens on 10.0.1.10:17777 (vSwitch IP only). +# BRouter itself is not published to the host — only reachable via the +# internal Docker network from the Caddy sidecar. +# +# Logging: the host's default logging driver is `loki` (user's personal +# Loki). Every service here explicitly overrides to `json-file` so logs +# stay local and are picked up by the promtail sidecar (section 6.3) for +# shipping to trails.cool's Loki. + +services: + brouter: + image: ghcr.io/trails-cool/brouter:latest + container_name: trails-brouter + restart: unless-stopped + # Planet-scale coverage: segments live on the host and are mounted in. + # 8 GB heap for segment cache; -Xms generous because routing is + # memory-heavy and we don't benefit from a slow JVM warmup. + environment: + JAVA_OPTS: "-Xmx8g -Xms512M" + volumes: + - ./segments:/data/segments:ro + networks: + - trails-brouter-internal + # Scope logs to json-file so we don't leak to the host's default Loki + logging: + driver: json-file + options: + max-size: "10m" + max-file: "3" + labels: + trails.cool.service: "brouter" + healthcheck: + test: ["CMD-SHELL", "wget -q -O- http://localhost:17777/ >/dev/null 2>&1 || exit 1"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 60s + + caddy: + image: caddy:2-alpine + container_name: trails-brouter-caddy + restart: unless-stopped + depends_on: + brouter: + condition: service_healthy + # Bind ONLY to the vSwitch IP on the host — the dedicated host's + # public IP remains unaffected. UFW further restricts this to traffic + # sourced from the flagship's private IP (10.0.0.2). + ports: + - "10.0.1.10:17777:17777" + environment: + BROUTER_AUTH_TOKEN: ${BROUTER_AUTH_TOKEN:?BROUTER_AUTH_TOKEN must be set} + volumes: + - ./Caddyfile:/etc/caddy/Caddyfile:ro + - caddy-data:/data + - caddy-config:/config + networks: + - trails-brouter-internal + logging: + driver: json-file + options: + max-size: "10m" + max-file: "3" + labels: + trails.cool.service: "brouter-caddy" + +volumes: + caddy-data: + caddy-config: + +networks: + trails-brouter-internal: + driver: bridge + # Container-to-container only; no host-level exposure via this net diff --git a/infrastructure/brouter-host/download-segments.sh b/infrastructure/brouter-host/download-segments.sh new file mode 100755 index 0000000..577c31a --- /dev/null +++ b/infrastructure/brouter-host/download-segments.sh @@ -0,0 +1,80 @@ +#!/bin/bash +# Download / refresh BRouter RD5 segments from brouter.de for planet-wide +# coverage. Idempotent: re-running only fetches files that are new or +# updated upstream (wget -N uses Last-Modified). First run takes hours +# and pulls ~60–80 GB; subsequent runs are cheap. +# +# Usage: +# ./download-segments.sh [dest_dir] +# dest_dir defaults to ./segments relative to this script +# +# Runs safely as non-root; no privileged operations. Can be cron'd. +# +# After a successful run, restart the brouter container so it reloads +# any updated segments: +# docker compose restart brouter + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +DEST_DIR="${1:-$SCRIPT_DIR/segments}" +BASE_URL="https://brouter.de/brouter/segments4" + +mkdir -p "$DEST_DIR" + +echo "Listing tiles at $BASE_URL/ ..." +# Extract RD5 filenames from the Apache-style directory listing. +# Pattern matches the standard brouter tile naming: W120_N40.rd5 etc. +tiles=$(curl --fail --silent --show-error --location "$BASE_URL/" \ + | grep -oE '[WE][0-9]+_[NS][0-9]+\.rd5' \ + | sort -u) + +if [ -z "$tiles" ]; then + echo "ERROR: no tiles found at $BASE_URL/ (directory listing empty or blocked)" >&2 + exit 1 +fi + +total=$(printf '%s\n' "$tiles" | wc -l | tr -d ' ') +echo "Found $total tiles. Destination: $DEST_DIR" +echo + +cd "$DEST_DIR" + +i=0 +skipped=0 +failed=0 +while read -r tile; do + [ -z "$tile" ] && continue + i=$((i + 1)) + # -N: only fetch if remote is newer than local (Last-Modified) + # -q: quiet; we print our own progress + if wget --no-verbose --timestamping --tries=3 --timeout=60 "$BASE_URL/$tile" 2>&1 | grep -q 'not retrieving'; then + skipped=$((skipped + 1)) + fi + if [ ! -s "$tile" ]; then + echo " [$i/$total] FAILED: $tile" + failed=$((failed + 1)) + fi + # Print heartbeat every 25 tiles so hour-long runs don't look hung + if [ $((i % 25)) -eq 0 ]; then + echo " [$i/$total] ... $skipped already-current, $failed failed so far" + fi +done <<< "$tiles" + +echo +echo "Done. Totals:" +echo " attempted: $i" +echo " already current: $skipped" +echo " failed: $failed" +echo +echo "Destination size:" +du -sh "$DEST_DIR" +echo +echo "Tile count on disk:" +ls "$DEST_DIR"/*.rd5 2>/dev/null | wc -l + +if [ "$failed" -gt 0 ]; then + echo + echo "WARNING: $failed downloads failed. Re-run the script to retry." >&2 + exit 2 +fi diff --git a/infrastructure/docker-compose.yml b/infrastructure/docker-compose.yml index e4d80d5..9118553 100644 --- a/infrastructure/docker-compose.yml +++ b/infrastructure/docker-compose.yml @@ -54,7 +54,16 @@ services: image: ghcr.io/trails-cool/planner:latest restart: unless-stopped environment: - BROUTER_URL: http://brouter:17777 + # BROUTER_URL overridable via SOPS: during the cutover to the + # dedicated BRouter host, flip to `http://10.0.1.10:17777` without + # touching this compose file. Default keeps the in-tree BRouter + # for the soak window and local dev. + BROUTER_URL: ${BROUTER_URL:-http://brouter:17777} + # Shared secret the Planner attaches as X-BRouter-Auth on every + # BRouter request. The Caddy sidecar on the dedicated BRouter + # host enforces this header; for the in-tree flagship BRouter + # it's an unused extra header. Set in SOPS secrets.app.env. + BROUTER_AUTH_TOKEN: ${BROUTER_AUTH_TOKEN} # Ordered failover list for the Overpass proxy. The code defaults # to the same pair if unset; declaring it here makes the prod # upstream explicit and easy to reshuffle via env when a diff --git a/openspec/changes/relocate-brouter-to-dedicated-host/tasks.md b/openspec/changes/relocate-brouter-to-dedicated-host/tasks.md index 22eebde..0aa3c30 100644 --- a/openspec/changes/relocate-brouter-to-dedicated-host/tasks.md +++ b/openspec/changes/relocate-brouter-to-dedicated-host/tasks.md @@ -24,30 +24,43 @@ - [ ] ~~2.2 Add `BROUTER_AUTH_TOKEN` to `infrastructure/secrets.infra.env` (SOPS)~~ **obsoleted** — after relocation, `cd-infra` no longer deploys BRouter and therefore doesn't need the token. `cd-brouter` reads it from `secrets.app.env` instead (see 5.1). Single source of truth. - [x] 2.3 Add `BROUTER_AUTH_TOKEN` to `infrastructure/secrets.app.env` (SOPS) for the Planner - Token added via `sops -d | append | sops -e`; round-trip decrypt confirms. Committed in this branch. -- [ ] 2.4 Add GitHub Actions secrets: `BROUTER_DEPLOY_HOST`, `BROUTER_DEPLOY_SSH_KEY`, `BROUTER_DEPLOY_SSH_PORT` +- [x] 2.4 Add GitHub Actions secrets: `BROUTER_DEPLOY_HOST`, `BROUTER_DEPLOY_SSH_KEY`, `BROUTER_DEPLOY_SSH_PORT` + - Set via `gh secret set` from operator laptop: `BROUTER_DEPLOY_HOST=ullrich.is`, `BROUTER_DEPLOY_SSH_PORT=2232`, `BROUTER_DEPLOY_SSH_KEY` from `~/.ssh/trails-brouter-deploy`. - [ ] 2.5 Document the rotation runbook in `docs/deployment.md` (or equivalent) ## 3. BRouter host compose project -- [ ] 3.1 Create `infrastructure/brouter-host/docker-compose.yml` with services `brouter` (bound only to the internal Docker network) and `caddy` (published on the vSwitch IP, auth-enforcing) -- [ ] 3.2 Create `infrastructure/brouter-host/Caddyfile` that requires `X-BRouter-Auth` equal to the configured token and forwards matching requests to `brouter:17777`; redact the header from access logs -- [ ] 3.3 Set `JAVA_OPTS=-Xmx8g` (or equivalent BRouter env) on the `brouter` service -- [ ] 3.4 Create `infrastructure/brouter-host/download-segments.sh` that fetches the planet RD5 tile list idempotently into `./segments/` -- [ ] 3.5 Add a README in `infrastructure/brouter-host/` with one-shot provisioning notes (`git clone`, first segment download, first compose up) +- [x] 3.1 Create `infrastructure/brouter-host/docker-compose.yml` with services `brouter` (bound only to the internal Docker network) and `caddy` (published on the vSwitch IP, auth-enforcing) + - Compose has explicit `logging: driver: json-file` on each service to bypass the dedicated host's default `loki` logging driver. Caddy binds to `10.0.1.10:17777`; brouter has no published port. +- [x] 3.2 Create `infrastructure/brouter-host/Caddyfile` that requires `X-BRouter-Auth` equal to the configured token and forwards matching requests to `brouter:17777`; redact the header from access logs + - Header matcher + 403 fallback; `auto_https off` since vSwitch-only. Caddy default access log format does not include request headers, so token is not logged. +- [x] 3.3 Set `JAVA_OPTS=-Xmx8g` (or equivalent BRouter env) on the `brouter` service + - Also patched `docker/brouter/Dockerfile` to honor `JAVA_OPTS` (was hardcoded `-Xmx1024M` in CMD). Default env keeps flagship behavior unchanged. +- [x] 3.4 Create `infrastructure/brouter-host/download-segments.sh` that fetches the planet RD5 tile list idempotently into `./segments/` + - Crawls brouter.de directory listing, uses `wget -N` for Last-Modified-based incremental updates, prints heartbeat every 25 tiles. +- [x] 3.5 Add a README in `infrastructure/brouter-host/` with one-shot provisioning notes (`git clone`, first segment download, first compose up) + - Covers bring-up, segment refresh, token rotation, rollback. Paired with the CD workflow which handles routine updates. ## 4. Planner changes -- [ ] 4.1 Add `BROUTER_AUTH_TOKEN` env var to `apps/planner/app/lib/brouter.ts`; send `X-BRouter-Auth` on every fetch -- [ ] 4.2 Fail the Planner startup with a clear error when `NODE_ENV=production` and `BROUTER_AUTH_TOKEN` is unset -- [ ] 4.3 Update `infrastructure/docker-compose.yml` Planner service env to pass `BROUTER_AUTH_TOKEN` through from the SOPS env file -- [ ] 4.4 Add a unit test covering the header-attachment path in `apps/planner/app/lib/brouter.ts` +- [x] 4.1 Add `BROUTER_AUTH_TOKEN` env var to `apps/planner/app/lib/brouter.ts`; send `X-BRouter-Auth` on every fetch + - `authHeaders()` helper reads env at call time (testable); attached to both `computeRoute` and `computeSegmentGpx` fetch sites. +- [x] 4.2 Fail the Planner startup with a clear error when `NODE_ENV=production` and `BROUTER_AUTH_TOKEN` is unset + - Module-level throw at import. Prod container fails fast; dev/test unaffected. +- [x] 4.3 Update `infrastructure/docker-compose.yml` Planner service env to pass `BROUTER_AUTH_TOKEN` through from the SOPS env file + - Also made `BROUTER_URL` overridable so cutover is a single SOPS edit away. +- [x] 4.4 Add a unit test covering the header-attachment path in `apps/planner/app/lib/brouter.ts` + - 3 new tests: token set → header attached, token unset → header omitted, covers both `computeRoute` and `computeSegmentGpx`. ## 5. CD workflow -- [ ] 5.1 Rewrite `.github/workflows/cd-brouter.yml` deploy job: SSH as `trails@${{ secrets.BROUTER_DEPLOY_HOST }}`, `cd ~trails/brouter`, `docker compose pull && docker compose up -d` -- [ ] 5.2 Update workflow `paths:` trigger to include `infrastructure/brouter-host/**` -- [ ] 5.3 Move the segment-download logic out of the workflow into the on-host `download-segments.sh`; workflow calls it but tolerates a long-running invocation (or skips on subsequent deploys if segments already present) -- [ ] 5.4 Keep the Grafana annotation step, pointing at the flagship Grafana over its existing path +- [x] 5.1 Rewrite `.github/workflows/cd-brouter.yml` deploy job: SSH as `trails@${{ secrets.BROUTER_DEPLOY_HOST }}`, `cd ~trails/brouter`, `docker compose pull && docker compose up -d` + - SSH on port `BROUTER_DEPLOY_SSH_PORT` (2232), dedicated key `BROUTER_DEPLOY_SSH_KEY`. +- [x] 5.2 Update workflow `paths:` trigger to include `infrastructure/brouter-host/**` +- [x] 5.3 Move the segment-download logic out of the workflow into the on-host `download-segments.sh`; workflow calls it but tolerates a long-running invocation (or skips on subsequent deploys if segments already present) + - Workflow does NOT call `download-segments.sh` — first-time seed is a manual operator step (per README and task 7.1); routine re-runs are cron-able on the dedicated host. +- [x] 5.4 Keep the Grafana annotation step, pointing at the flagship Grafana over its existing path + - Still uses `DEPLOY_HOST` + `DEPLOY_SSH_KEY` to reach the flagship for the annotation. - [ ] 5.5 Remove the `brouter:` service from `infrastructure/docker-compose.yml` on the flagship (deferred to cutover step 7.5) ## 6. Observability