Archive observability change, sync specs to main

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-03-27 21:44:07 +01:00
parent 5915907656
commit 81d0feffdd
No known key found for this signature in database
GPG key ID: A32FF691A0F752D9
8 changed files with 81 additions and 0 deletions

View file

@ -121,3 +121,17 @@ Grafana SHALL authenticate users via GitHub OAuth, restricted to the trails-cool
#### Scenario: No password-based login
- **WHEN** Grafana is deployed
- **THEN** the login form is disabled and only GitHub OAuth is available
### Requirement: Docker Compose deployment
All services SHALL be deployed via Docker Compose, including Grafana, Prometheus, and Loki for the flagship instance.
#### Scenario: Monitoring stack starts
- **WHEN** `docker compose up -d` is run
- **THEN** Grafana, Prometheus, and Loki containers start alongside the application containers
### Requirement: Caddy access logging
Caddy SHALL emit structured JSON access logs for all requests.
#### Scenario: Access log emitted
- **WHEN** any HTTP request passes through Caddy
- **THEN** a JSON log line with remote IP, method, path, status, and duration is written to stdout

View file

@ -0,0 +1,67 @@
## Requirements
### Requirement: Health endpoints
Both apps SHALL expose a `/health` endpoint returning service and database status.
#### Scenario: Healthy service
- **WHEN** the app is running and the database is reachable
- **THEN** `GET /health` returns `{ "status": "ok", "db": "connected" }` with HTTP 200
#### Scenario: Degraded service
- **WHEN** the app is running but the database is unreachable
- **THEN** `GET /health` returns `{ "status": "degraded", "db": "unreachable" }` with HTTP 503
### Requirement: Prometheus metrics
Both apps SHALL expose a `/metrics` endpoint with Prometheus-formatted metrics.
#### Scenario: Default Node.js metrics
- **WHEN** Prometheus scrapes `/metrics`
- **THEN** it receives event loop lag, heap usage, and GC metrics
#### Scenario: HTTP request metrics
- **WHEN** requests are served
- **THEN** `http_request_duration_seconds` histogram is updated with route, method, and status labels
#### Scenario: Planner-specific metrics
- **WHEN** the Planner is running
- **THEN** `planner_active_sessions` and `planner_connected_clients` gauges reflect current state
### Requirement: Structured logging
Both apps SHALL output structured JSON logs in production.
#### Scenario: Request logging
- **WHEN** an HTTP request is served in production
- **THEN** a JSON log line is emitted with method, path, status, duration, and timestamp
#### Scenario: Dev mode pretty printing
- **WHEN** running in development
- **THEN** logs are human-readable (pretty-printed)
### Requirement: Grafana dashboards
The flagship instance SHALL have pre-configured Grafana dashboards.
#### Scenario: Overview dashboard
- **WHEN** an operator opens Grafana
- **THEN** they see request rate, error rate, and latency percentiles
#### Scenario: Infrastructure dashboard
- **WHEN** an operator checks infrastructure health
- **THEN** they see CPU, memory, disk usage, and DB connection pool stats
### Requirement: Alerting
Grafana SHALL alert on critical conditions.
#### Scenario: Disk usage alert
- **WHEN** disk usage exceeds 80% for 15 minutes
- **THEN** an alert fires
#### Scenario: App down alert
- **WHEN** an app returns zero healthy responses for 2 minutes
- **THEN** an alert fires
### Requirement: Log aggregation
Loki SHALL collect and index logs from all Docker containers.
#### Scenario: Search logs
- **WHEN** an operator searches logs in Grafana
- **THEN** they can filter by container name, log level, and time range