49aadd04a9
Health endpoints: - /api/health (Journal) and /health (Planner) with DB connectivity check - Docker healthchecks updated to use app health endpoints Structured logging: - Pino with JSON output in production, pretty-print in dev - Request logging middleware in Planner (method, path, status, duration) - Replaced console.log/error with structured logger in email and auth flows Prometheus metrics: - prom-client with default Node.js metrics + custom histograms/gauges - /metrics endpoints on both apps - http_request_duration, planner_active_sessions, brouter_request_duration Monitoring stack: - Prometheus, Loki, Grafana containers in docker-compose - Grafana provisioned with datasources, dashboards, and alert rules - Caddy access logging (JSON to stdout for Loki) - grafana.trails.cool with basic auth via Caddy Dashboards and alerting: - Overview: request rate, error rate, latency p50/p95/p99 - Planner: active sessions, connected clients, BRouter latency - Infrastructure: memory, CPU, event loop lag - Alerts: disk >80%, app down 2min, error rate >5% Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3.1 KiB
3.1 KiB
1. Health Endpoints
- 1.1 Add
/healthAPI route to Journal — checks DB with a simple query, returns status JSON - 1.2 Add
/healthendpoint to Planner server.ts — checks DB connectivity - 1.3 Update Docker healthcheck in docker-compose.yml to use
/healthinstead ofpg_isready - 1.4 Add health routes to route configs (journal routes.ts)
2. Structured Logging
- 2.1 Add
pinodependency to both apps - 2.2 Create
apps/journal/app/lib/logger.server.ts— Pino instance, JSON in prod, pretty in dev - 2.3 Create
apps/planner/app/lib/logger.server.ts— same pattern - 2.4 Replace key
console.log/console.errorcalls with logger (auth, DB errors, session lifecycle) - 2.5 Add request logging middleware to Planner server.ts (method, path, status, duration)
3. Prometheus Metrics
- 3.1 Add
prom-clientdependency to both apps - 3.2 Create metrics utility for Journal — default metrics + http_request_duration_seconds histogram
- 3.3 Create metrics utility for Planner — default metrics + http histogram + planner_active_sessions + planner_connected_clients gauges + brouter_request_duration_seconds histogram
- 3.4 Add
/metricsendpoint to Journal (API route) - 3.5 Add
/metricsendpoint to Planner server.ts - 3.6 Wire request duration tracking into request handlers
4. Caddy Access Logs
- 4.1 Enable Caddy structured JSON access logging in Caddyfile (log directive)
5. Monitoring Stack (Docker Compose)
- 5.1 Add Prometheus container to docker-compose.yml with scrape config for journal:3000 and planner:3001
- 5.2 Add Loki container to docker-compose.yml with Docker logging driver config
- 5.3 Add Grafana container to docker-compose.yml with provisioned data sources (Prometheus + Loki)
- 5.4 Create
infrastructure/prometheus/prometheus.yml— scrape config - 5.5 Create
infrastructure/loki/loki-config.yml— retention, storage - 5.6 Create
infrastructure/grafana/provisioning/— data sources + dashboard provider config - 5.7 Expose Grafana via Caddy on grafana.trails.cool with basic auth
6. Dashboards
- 6.1 Create overview dashboard JSON — request rate, error rate, latency p50/p95/p99
- 6.2 Create planner dashboard JSON — active sessions, connected clients, BRouter latency
- 6.3 Create infrastructure dashboard JSON — node_exporter or cAdvisor metrics (CPU, memory, disk)
7. Alerting
- 7.1 Configure Grafana alert rule: disk usage > 80%
- 7.2 Configure Grafana alert rule: app health check failing for 2 min
- 7.3 Configure Grafana alert rule: error rate > 5% for 5 min
- 7.4 Set up alert notification channel (email or webhook)
8. DNS & Terraform
- 8.1 Add grafana.trails.cool DNS record
- 8.2 Add Grafana basic auth credentials to deploy secrets
9. Verify
- 9.1 Test /health endpoints locally — verify OK and degraded responses
- 9.2 Test /metrics endpoints — verify Prometheus format output
- 9.3 Test Grafana dashboards load with data after deploy
- 9.4 Test alert fires when simulating disk full condition