trails/infrastructure/grafana/dashboards
Ullrich Schäfer 9e598fb6a1
BRouter host observability + docs
Lands sections 6 and 8 of relocate-brouter-to-dedicated-host on top
of the host compose in #291.

## Observability (section 6)

- **Prometheus**: new `brouter-cadvisor` job scraping
  `10.0.1.10:8080` over the vSwitch, labeled `host="brouter"`.
- **cAdvisor sidecar** on the dedicated host's compose
  (`--docker_only --whitelisted_container_labels=trails.cool.service`)
  so metrics only cover trails containers, never the operator's
  unrelated workloads on the shared box.
- **Promtail sidecar** on the dedicated host, Docker SD with relabel-
  drop on missing `trails.cool.service` label, pushing to flagship
  Loki at `http://10.0.0.2:3100/loki/api/v1/push`.
- **Flagship compose**: Loki now publishes port 3100 on the vSwitch
  IP only (10.0.0.2:3100) — Hetzner Cloud firewall blocks it from
  the public internet.
- **Grafana dashboard**: `brouter.json` — scrape up/down, request
  rate (from Planner-side `brouter_request_duration_seconds`),
  p50/p95/p99, container memory/CPU, Loki logs panel.
- **Alert**: `brouter-scrape-down` fires on
  `up{job="brouter-cadvisor"} < 1 for 2m`; `noDataState: Alerting`
  so a total scrape failure still pages.

Operator needs one UFW rule on the dedicated host for the cAdvisor
port — documented in `infrastructure/brouter-host/README.md`.

## Documentation (section 8)

- `CLAUDE.md` — hosts table + updated deployment table with SSH
  targets per workflow; BRouter host SSH is `-p 2232 trails@...`,
  different key.
- `docs/architecture.md` — Hosting section rewritten to cover both
  hosts, vSwitch boundary, and the observability-scoping rationale
  for the shared dedicated host.
- `docs/deployment.md` (new) — full operator runbook: host layout,
  first-time BRouter provisioning, SOPS rotation (including the
  macOS `SOPS_AGE_KEY_FILE` gotcha), cutover procedure with
  rollback, manual workflow triggers.

Task 8.4 (infrastructure/README.md) skipped: that file doesn't
exist and the ground is covered by brouter-host/README.md +
docs/deployment.md.

## Validation

- `docker compose config` on both the flagship and brouter-host
  compose files — both validate.
- `pnpm typecheck`, `pnpm lint`, `pnpm test` — all clean (full
  turbo cache hits; no code changes in this commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 23:10:45 +02:00
..
brouter.json BRouter host observability + docs 2026-04-23 23:10:45 +02:00
business.json Remove type:dashboard from annotation config 2026-03-29 21:57:39 +02:00
demo-bot.json Add Bruno (demo-bot) Grafana dashboard 2026-04-19 11:25:17 +02:00
infrastructure.json Unwrap dashboard JSON for Grafana provisioning 2026-03-29 22:18:06 +02:00
journal.json Add Journal dashboard, fix Caddy metrics, and improve alerts 2026-04-12 14:00:00 +02:00
overview.json Add Journal dashboard, fix Caddy metrics, and improve alerts 2026-04-12 14:00:00 +02:00
planner.json Fix Overpass health + cache-hit stat panels: vector(0) not clamp_min 2026-04-18 02:42:50 +02:00
service-health.json Update pgboss column names for v12 (snake_case) 2026-04-20 16:49:43 +02:00