Add observability: health endpoints, structured logging, metrics, Grafana stack

Health endpoints:
- /api/health (Journal) and /health (Planner) with DB connectivity check
- Docker healthchecks updated to use app health endpoints

Structured logging:
- Pino with JSON output in production, pretty-print in dev
- Request logging middleware in Planner (method, path, status, duration)
- Replaced console.log/error with structured logger in email and auth flows

Prometheus metrics:
- prom-client with default Node.js metrics + custom histograms/gauges
- /metrics endpoints on both apps
- http_request_duration, planner_active_sessions, brouter_request_duration

Monitoring stack:
- Prometheus, Loki, Grafana containers in docker-compose
- Grafana provisioned with datasources, dashboards, and alert rules
- Caddy access logging (JSON to stdout for Loki)
- grafana.trails.cool with basic auth via Caddy

Dashboards and alerting:
- Overview: request rate, error rate, latency p50/p95/p99
- Planner: active sessions, connected clients, BRouter latency
- Infrastructure: memory, CPU, event loop lag
- Alerts: disk >80%, app down 2min, error rate >5%

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-03-26 22:59:44 +01:00
parent ce964cae96
commit 49aadd04a9
No known key found for this signature in database
GPG key ID: A32FF691A0F752D9
24 changed files with 770 additions and 48 deletions

View file

@ -17,6 +17,10 @@
{$DOMAIN:trails.cool} {
import security_headers
import block_scanners
log {
output stdout
format json
}
reverse_proxy journal:3000
}
@ -24,8 +28,19 @@ www.{$DOMAIN:trails.cool} {
redir https://{$DOMAIN:trails.cool}{uri} permanent
}
grafana.{$DOMAIN:trails.cool} {
basicauth {
{$GRAFANA_USER:admin} {$GRAFANA_PASSWORD_HASH}
}
reverse_proxy grafana:3000
}
planner.{$DOMAIN:trails.cool} {
import security_headers
import block_scanners
log {
output stdout
format json
}
reverse_proxy planner:3001
}