trails/infrastructure/brouter-host/docker-compose.yml
Ullrich Schäfer 9e598fb6a1
BRouter host observability + docs
Lands sections 6 and 8 of relocate-brouter-to-dedicated-host on top
of the host compose in #291.

## Observability (section 6)

- **Prometheus**: new `brouter-cadvisor` job scraping
  `10.0.1.10:8080` over the vSwitch, labeled `host="brouter"`.
- **cAdvisor sidecar** on the dedicated host's compose
  (`--docker_only --whitelisted_container_labels=trails.cool.service`)
  so metrics only cover trails containers, never the operator's
  unrelated workloads on the shared box.
- **Promtail sidecar** on the dedicated host, Docker SD with relabel-
  drop on missing `trails.cool.service` label, pushing to flagship
  Loki at `http://10.0.0.2:3100/loki/api/v1/push`.
- **Flagship compose**: Loki now publishes port 3100 on the vSwitch
  IP only (10.0.0.2:3100) — Hetzner Cloud firewall blocks it from
  the public internet.
- **Grafana dashboard**: `brouter.json` — scrape up/down, request
  rate (from Planner-side `brouter_request_duration_seconds`),
  p50/p95/p99, container memory/CPU, Loki logs panel.
- **Alert**: `brouter-scrape-down` fires on
  `up{job="brouter-cadvisor"} < 1 for 2m`; `noDataState: Alerting`
  so a total scrape failure still pages.

Operator needs one UFW rule on the dedicated host for the cAdvisor
port — documented in `infrastructure/brouter-host/README.md`.

## Documentation (section 8)

- `CLAUDE.md` — hosts table + updated deployment table with SSH
  targets per workflow; BRouter host SSH is `-p 2232 trails@...`,
  different key.
- `docs/architecture.md` — Hosting section rewritten to cover both
  hosts, vSwitch boundary, and the observability-scoping rationale
  for the shared dedicated host.
- `docs/deployment.md` (new) — full operator runbook: host layout,
  first-time BRouter provisioning, SOPS rotation (including the
  macOS `SOPS_AGE_KEY_FILE` gotcha), cutover procedure with
  rollback, manual workflow triggers.

Task 8.4 (infrastructure/README.md) skipped: that file doesn't
exist and the ground is covered by brouter-host/README.md +
docs/deployment.md.

## Validation

- `docker compose config` on both the flagship and brouter-host
  compose files — both validate.
- `pnpm typecheck`, `pnpm lint`, `pnpm test` — all clean (full
  turbo cache hits; no code changes in this commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 23:10:45 +02:00

130 lines
4.1 KiB
YAML

# BRouter host compose project — runs on the dedicated Hetzner Robot
# server `ullrich.is` under the `trails` user. See README.md for first-time
# provisioning notes.
#
# Exposed surface: Caddy listens on 10.0.1.10:17777 (vSwitch IP only).
# BRouter itself is not published to the host — only reachable via the
# internal Docker network from the Caddy sidecar.
#
# Logging: the host's default logging driver is `loki` (user's personal
# Loki). Every service here explicitly overrides to `json-file` so logs
# stay local and are picked up by the promtail sidecar (section 6.3) for
# shipping to trails.cool's Loki.
services:
brouter:
image: ghcr.io/trails-cool/brouter:latest
container_name: trails-brouter
restart: unless-stopped
# Planet-scale coverage: segments live on the host and are mounted in.
# 8 GB heap for segment cache; -Xms generous because routing is
# memory-heavy and we don't benefit from a slow JVM warmup.
environment:
JAVA_OPTS: "-Xmx8g -Xms512M"
volumes:
- ./segments:/data/segments:ro
networks:
- trails-brouter-internal
# Scope logs to json-file so we don't leak to the host's default Loki
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
labels:
trails.cool.service: "brouter"
healthcheck:
test: ["CMD-SHELL", "wget -q -O- http://localhost:17777/ >/dev/null 2>&1 || exit 1"]
interval: 30s
timeout: 5s
retries: 3
start_period: 60s
caddy:
image: caddy:2-alpine
container_name: trails-brouter-caddy
restart: unless-stopped
depends_on:
brouter:
condition: service_healthy
# Bind ONLY to the vSwitch IP on the host — the dedicated host's
# public IP remains unaffected. UFW further restricts this to traffic
# sourced from the flagship's private IP (10.0.0.2).
ports:
- "10.0.1.10:17777:17777"
environment:
BROUTER_AUTH_TOKEN: ${BROUTER_AUTH_TOKEN:?BROUTER_AUTH_TOKEN must be set}
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data
- caddy-config:/config
networks:
- trails-brouter-internal
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
labels:
trails.cool.service: "brouter-caddy"
# cAdvisor — container metrics scraped by flagship Prometheus over the
# vSwitch. We set container label filters so the exposed metrics only
# cover trails-labeled containers (brouter + caddy), not the operator's
# other workloads on this shared host.
cadvisor:
image: gcr.io/cadvisor/cadvisor:latest
container_name: trails-brouter-cadvisor
restart: unless-stopped
privileged: true
ports:
- "10.0.1.10:8080:8080"
volumes:
- /:/rootfs:ro
- /var/run:/var/run:ro
- /sys:/sys:ro
- /var/lib/docker:/var/lib/docker:ro
- /dev/disk/:/dev/disk:ro
command:
# Only consider containers (not the host) and only those that carry
# our label; filters out everything else on the shared host.
- --docker_only=true
- --store_container_labels=false
- --whitelisted_container_labels=trails.cool.service
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
labels:
trails.cool.service: "brouter-cadvisor"
# Promtail — tails Docker logs for trails-labeled containers only and
# pushes to flagship Loki over the vSwitch. Does NOT scrape other
# containers' logs on this host.
promtail:
image: grafana/promtail:latest
container_name: trails-brouter-promtail
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./promtail-config.yml:/etc/promtail/config.yml:ro
- promtail-positions:/tmp
command: ["-config.file=/etc/promtail/config.yml"]
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
labels:
trails.cool.service: "brouter-promtail"
volumes:
caddy-data:
caddy-config:
promtail-positions:
networks:
trails-brouter-internal:
driver: bridge
# Container-to-container only; no host-level exposure via this net