Lands sections 6 and 8 of relocate-brouter-to-dedicated-host on top of the host compose in #291. ## Observability (section 6) - **Prometheus**: new `brouter-cadvisor` job scraping `10.0.1.10:8080` over the vSwitch, labeled `host="brouter"`. - **cAdvisor sidecar** on the dedicated host's compose (`--docker_only --whitelisted_container_labels=trails.cool.service`) so metrics only cover trails containers, never the operator's unrelated workloads on the shared box. - **Promtail sidecar** on the dedicated host, Docker SD with relabel- drop on missing `trails.cool.service` label, pushing to flagship Loki at `http://10.0.0.2:3100/loki/api/v1/push`. - **Flagship compose**: Loki now publishes port 3100 on the vSwitch IP only (10.0.0.2:3100) — Hetzner Cloud firewall blocks it from the public internet. - **Grafana dashboard**: `brouter.json` — scrape up/down, request rate (from Planner-side `brouter_request_duration_seconds`), p50/p95/p99, container memory/CPU, Loki logs panel. - **Alert**: `brouter-scrape-down` fires on `up{job="brouter-cadvisor"} < 1 for 2m`; `noDataState: Alerting` so a total scrape failure still pages. Operator needs one UFW rule on the dedicated host for the cAdvisor port — documented in `infrastructure/brouter-host/README.md`. ## Documentation (section 8) - `CLAUDE.md` — hosts table + updated deployment table with SSH targets per workflow; BRouter host SSH is `-p 2232 trails@...`, different key. - `docs/architecture.md` — Hosting section rewritten to cover both hosts, vSwitch boundary, and the observability-scoping rationale for the shared dedicated host. - `docs/deployment.md` (new) — full operator runbook: host layout, first-time BRouter provisioning, SOPS rotation (including the macOS `SOPS_AGE_KEY_FILE` gotcha), cutover procedure with rollback, manual workflow triggers. Task 8.4 (infrastructure/README.md) skipped: that file doesn't exist and the ground is covered by brouter-host/README.md + docs/deployment.md. ## Validation - `docker compose config` on both the flagship and brouter-host compose files — both validate. - `pnpm typecheck`, `pnpm lint`, `pnpm test` — all clean (full turbo cache hits; no code changes in this commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
111 lines
3.9 KiB
YAML
111 lines
3.9 KiB
YAML
name: CD BRouter
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths:
|
|
- "docker/brouter/**"
|
|
- "infrastructure/brouter-host/**"
|
|
workflow_dispatch: {}
|
|
|
|
concurrency:
|
|
group: deploy-brouter
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
build:
|
|
name: Build & Push BRouter Image
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: docker/login-action@v4
|
|
with:
|
|
registry: ghcr.io
|
|
username: ${{ github.actor }}
|
|
password: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- uses: docker/build-push-action@v7
|
|
with:
|
|
context: docker/brouter
|
|
push: true
|
|
tags: |
|
|
ghcr.io/trails-cool/brouter:latest
|
|
ghcr.io/trails-cool/brouter:${{ github.sha }}
|
|
|
|
deploy:
|
|
name: Deploy BRouter to dedicated host
|
|
needs: [build]
|
|
runs-on: ubuntu-latest
|
|
environment: infra
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Decrypt shared secret
|
|
run: |
|
|
curl -sLO https://github.com/getsops/sops/releases/download/v3.9.4/sops-v3.9.4.linux.amd64
|
|
chmod +x sops-v3.9.4.linux.amd64
|
|
# Extract ONLY BROUTER_AUTH_TOKEN from secrets.app.env — the
|
|
# rest of that file is app-only and has no business reaching
|
|
# the BRouter host.
|
|
SOPS_AGE_KEY="${{ secrets.AGE_SECRET_KEY }}" ./sops-v3.9.4.linux.amd64 -d infrastructure/secrets.app.env \
|
|
| grep '^BROUTER_AUTH_TOKEN=' > infrastructure/brouter-host/.env
|
|
chmod 0600 infrastructure/brouter-host/.env
|
|
|
|
- name: Copy compose project to dedicated host
|
|
uses: appleboy/scp-action@v1
|
|
with:
|
|
host: ${{ secrets.BROUTER_DEPLOY_HOST }}
|
|
username: trails
|
|
port: ${{ secrets.BROUTER_DEPLOY_SSH_PORT }}
|
|
key: ${{ secrets.BROUTER_DEPLOY_SSH_KEY }}
|
|
source: "infrastructure/brouter-host/docker-compose.yml,infrastructure/brouter-host/Caddyfile,infrastructure/brouter-host/promtail-config.yml,infrastructure/brouter-host/download-segments.sh,infrastructure/brouter-host/.env"
|
|
target: /home/trails/brouter
|
|
strip_components: 2
|
|
|
|
- name: Pull image and restart containers
|
|
uses: appleboy/ssh-action@v1
|
|
with:
|
|
host: ${{ secrets.BROUTER_DEPLOY_HOST }}
|
|
username: trails
|
|
port: ${{ secrets.BROUTER_DEPLOY_SSH_PORT }}
|
|
key: ${{ secrets.BROUTER_DEPLOY_SSH_KEY }}
|
|
script: |
|
|
set -euo pipefail
|
|
cd /home/trails/brouter
|
|
|
|
chmod +x download-segments.sh
|
|
|
|
# Segment download is explicitly NOT run here — it's
|
|
# multi-hour and idempotent. Seeding the segments directory
|
|
# is a one-shot operator task (see brouter-host/README.md).
|
|
# Run `~/brouter/download-segments.sh` by hand (or via cron)
|
|
# to refresh.
|
|
if [ ! -d segments ] || [ -z "$(ls -A segments 2>/dev/null)" ]; then
|
|
echo "WARNING: segments/ is empty. BRouter will start but return 404 until segments are seeded."
|
|
mkdir -p segments
|
|
fi
|
|
|
|
docker compose pull
|
|
docker compose up -d --remove-orphans
|
|
docker compose ps
|
|
|
|
- name: Annotate deploy in flagship Grafana
|
|
uses: appleboy/ssh-action@v1
|
|
with:
|
|
host: ${{ secrets.DEPLOY_HOST }}
|
|
username: root
|
|
key: ${{ secrets.DEPLOY_SSH_KEY }}
|
|
script: |
|
|
cd /opt/trails-cool
|
|
GRAFANA_TOKEN=$(grep GRAFANA_SERVICE_TOKEN .env | cut -d= -f2-)
|
|
if [ -n "$GRAFANA_TOKEN" ]; then
|
|
docker compose exec -T grafana curl -sf -X POST \
|
|
-H "Authorization: Bearer $GRAFANA_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"text":"Deploy brouter ${{ github.sha }}","tags":["deploy","brouter"]}' \
|
|
http://localhost:3000/api/annotations || true
|
|
fi
|