## Purpose

Server provisioning on Hetzner, Docker Compose deployment, CI/CD pipelines, database and BRouter management, TLS, Sentry, Grafana, and monitoring stack for the flagship instance.
## Requirements
### Requirement: Terraform Hetzner provisioning
Infrastructure SHALL be provisioned on Hetzner Cloud using Terraform with the Hetzner provider.

#### Scenario: Provision server
- **WHEN** `terraform apply` is run
- **THEN** a Hetzner cx23 server (2 vCPU, 4 GB RAM, 40 GB SSD) is created with Docker installed

### Requirement: Docker Compose deployment
All services SHALL be deployed via Docker Compose, including Grafana, Prometheus, and Loki for the flagship instance.

#### Scenario: Monitoring stack starts
- **WHEN** `docker compose up -d` is run
- **THEN** Grafana, Prometheus, Loki, Promtail, postgres-exporter, node-exporter, and cAdvisor containers start alongside the application containers

### Requirement: Service configuration
Each service SHALL be configured via environment variables defined in Docker Compose, with security best practices including non-root execution and security headers.

#### Scenario: Journal configuration
- **WHEN** the Journal container starts
- **THEN** it reads DOMAIN, DATABASE_URL, PLANNER_URL, JWT_SECRET, SESSION_SECRET, INTEGRATION_SECRET, WAHOO_* credentials, and SENTRY_DSN from environment variables

#### Scenario: Planner configuration
- **WHEN** the Planner container starts
- **THEN** it reads BROUTER_URL, DATABASE_URL, and INTEGRATION_SECRET from environment variables

#### Scenario: Caddy security headers
- **WHEN** Caddy proxies a request
- **THEN** it adds HSTS, X-Content-Type-Options, X-Frame-Options, Referrer-Policy, and Permissions-Policy headers

#### Scenario: Caddy scanner blocking
- **WHEN** a request matches known scanner paths (.env, .git, wp-config, etc.)
- **THEN** Caddy returns 403 without forwarding to the application

### Requirement: PostgreSQL with PostGIS
The database SHALL be PostgreSQL with the PostGIS extension for spatial queries.

#### Scenario: PostGIS available
- **WHEN** the PostgreSQL container starts
- **THEN** the PostGIS extension is available and can be enabled with `CREATE EXTENSION postgis`

### Requirement: BRouter segment management
The infrastructure SHALL support downloading and updating planet-wide RD5 segments from brouter.de to the dedicated BRouter host. Segment files SHALL live under `~trails/brouter/segments/` on the dedicated host and SHALL be owned by the `trails` user.

#### Scenario: Download segments
- **WHEN** the segment download script runs on the dedicated host as the `trails` user
- **THEN** all planet-wide RD5 files referenced by the tile list are downloaded to `~trails/brouter/segments/`, skipping files that already exist

#### Scenario: Segment update
- **WHEN** an operator re-runs the segment download script
- **THEN** outdated or missing RD5 files are re-fetched from brouter.de and the BRouter container is restarted

### Requirement: CI/CD pipeline
GitHub Actions SHALL use separate workflows for app deployment, infrastructure deployment, and BRouter deployment, with secrets decrypted from a SOPS-encrypted file.

#### Scenario: App deployment
- **WHEN** code changes are pushed to main in apps/ or packages/
- **THEN** the cd-apps workflow builds Docker images, pushes to ghcr.io, and deploys app containers to the flagship host

#### Scenario: Infrastructure deployment
- **WHEN** changes are pushed to main in infrastructure/
- **THEN** the cd-infra workflow copies configs and restarts infrastructure services on the flagship host without rebuilding app images and without touching the BRouter host

#### Scenario: BRouter deployment
- **WHEN** changes are pushed to main in docker/brouter/ or the BRouter host compose config
- **THEN** the cd-brouter workflow SSHes as the `trails` user into the dedicated BRouter host using `BROUTER_DEPLOY_HOST` / `BROUTER_DEPLOY_SSH_KEY` and runs `docker compose up -d` in `~trails/brouter/`

#### Scenario: Secret decryption at deploy time
- **WHEN** `cd-apps.yml` runs
- **THEN** `secrets.app.env` is decrypted and injected into the Journal and Planner containers
- **WHEN** `cd-infra.yml` runs
- **THEN** both `secrets.app.env` and `secrets.infra.env` are decrypted and merged for infrastructure services

#### Scenario: Gitleaks scan
- **WHEN** a PR is opened
- **THEN** gitleaks scans for committed secrets

#### Scenario: Dependency audit
- **WHEN** CI runs
- **THEN** pnpm audit checks for high/critical vulnerabilities

### Requirement: Backup strategy
The infrastructure SHALL include daily backups of the PostgreSQL database.

#### Scenario: Daily backup
- **WHEN** the daily backup cron runs
- **THEN** a PostgreSQL dump is uploaded to the Hetzner Storage Box

### Requirement: Domain and TLS
The infrastructure SHALL configure DNS and TLS for trails.cool and planner.trails.cool.

#### Scenario: HTTPS access
- **WHEN** a user navigates to https://trails.cool
- **THEN** the connection is secured with a valid TLS certificate

### Requirement: Sentry error tracking
The system SHALL enrich Sentry events with user and session context, use route-aware tracing, and prevent source maps from being served to clients.

#### Scenario: Journal error includes user context
- **WHEN** an authenticated Journal user triggers an error
- **THEN** the Sentry event SHALL include the user's ID and username

#### Scenario: Journal error without user context
- **WHEN** an unauthenticated visitor triggers an error
- **THEN** the Sentry event SHALL have no user context (Sentry.setUser(null))

#### Scenario: Planner error includes session ID
- **WHEN** an error occurs during a Planner session
- **THEN** the Sentry event SHALL include a `session_id` tag with the active session ID

#### Scenario: Route-level performance traces
- **WHEN** a user navigates between routes in either app
- **THEN** Sentry SHALL create a transaction span named after the route pattern (e.g., `/routes/:id`)

#### Scenario: Source maps not served to clients
- **WHEN** a client requests a `.map` file from the production server
- **THEN** the server SHALL return 404 (source maps are uploaded to Sentry during build, not shipped in the bundle)

### Requirement: Grafana authentication
Grafana SHALL authenticate users via GitHub OAuth, restricted to the trails-cool GitHub organization.

#### Scenario: GitHub OAuth login
- **WHEN** a user navigates to grafana.internal.trails.cool
- **THEN** they are redirected to GitHub for authentication and granted access if they are a member of the trails-cool organization

#### Scenario: No password-based login
- **WHEN** Grafana is deployed
- **THEN** the login form is disabled and only GitHub OAuth is available

### Requirement: Grafana database access
The `grafana_reader` PostgreSQL role SHALL have SELECT access to the `pgboss` schema for job queue observability.

#### Scenario: Grant access on deploy
- **WHEN** the infrastructure deploy runs
- **THEN** `grafana_reader` is granted `USAGE` on the `pgboss` schema and `SELECT` on all tables in it

### Requirement: Monitoring stack
The Grafana Service Health dashboard SHALL include a job queue health panel.

#### Scenario: Job queue panel displays metrics
- **WHEN** a user views the Service Health dashboard
- **THEN** they see a panel showing job queue depth, completed jobs per hour, and failed jobs
- **AND** failed jobs are highlighted for investigation

### Requirement: Metrics collection
Prometheus SHALL scrape metrics from all application and infrastructure services.

#### Scenario: Exporter targets
- **WHEN** Prometheus is running
- **THEN** it scrapes metrics from journal (/api/metrics), planner (/metrics), postgres-exporter, node-exporter, cAdvisor, and Caddy (:2019)

#### Scenario: pg_stat_statements
- **WHEN** postgres-exporter scrapes PostgreSQL
- **THEN** slow query metrics from pg_stat_statements are exposed with query text via a custom queries config

### Requirement: Container log shipping
Promtail SHALL scrape all Docker container logs and push them to Loki for querying in Grafana.

#### Scenario: Logs visible in Grafana
- **WHEN** a container writes to stdout or stderr
- **THEN** the log line is available in Grafana Explore via Loki with container name labels

### Requirement: Caddy access logging
Caddy SHALL emit structured JSON access logs for all requests.

#### Scenario: Access log emitted
- **WHEN** any HTTP request passes through Caddy
- **THEN** a JSON log line with remote IP, method, path, status, and duration is written to stdout

### Requirement: Private network between flagship and BRouter hosts
The flagship host and the dedicated BRouter host SHALL be joined on a Hetzner vSwitch in the same datacenter. All traffic between Planner and BRouter SHALL traverse this private network.

#### Scenario: vSwitch reachability
- **WHEN** the flagship host issues a request to the BRouter host's vSwitch IP on the BRouter service port with a valid `X-BRouter-Auth` header
- **THEN** the request succeeds over the private network without traversing the public internet

#### Scenario: No public BRouter exposure
- **WHEN** Hetzner Cloud firewall rules or equivalent host firewall rules are inspected
- **THEN** no rule allows inbound traffic to the BRouter service port from any public IP

### Requirement: Non-root deploy user on the BRouter host
The BRouter host SHALL be administered by the trails.cool project through a non-root `trails` user that is a member of the `docker` group. The CD workflow SHALL NOT require sudo or root SSH access on this host.

#### Scenario: Deploy with trails user
- **WHEN** the cd-brouter workflow connects to the BRouter host
- **THEN** it authenticates as `trails` and successfully runs `docker compose` commands without invoking sudo

#### Scenario: Scoped ownership
- **WHEN** files are created by the deploy or segment-download scripts
- **THEN** they live under `~trails/brouter/` and are owned by `trails:trails`