trails/infrastructure/grafana/dashboards/infrastructure.json
Ullrich Schäfer 49aadd04a9
Add observability: health endpoints, structured logging, metrics, Grafana stack
Health endpoints:
- /api/health (Journal) and /health (Planner) with DB connectivity check
- Docker healthchecks updated to use app health endpoints

Structured logging:
- Pino with JSON output in production, pretty-print in dev
- Request logging middleware in Planner (method, path, status, duration)
- Replaced console.log/error with structured logger in email and auth flows

Prometheus metrics:
- prom-client with default Node.js metrics + custom histograms/gauges
- /metrics endpoints on both apps
- http_request_duration, planner_active_sessions, brouter_request_duration

Monitoring stack:
- Prometheus, Loki, Grafana containers in docker-compose
- Grafana provisioned with datasources, dashboards, and alert rules
- Caddy access logging (JSON to stdout for Loki)
- grafana.trails.cool with basic auth via Caddy

Dashboards and alerting:
- Overview: request rate, error rate, latency p50/p95/p99
- Planner: active sessions, connected clients, BRouter latency
- Infrastructure: memory, CPU, event loop lag
- Alerts: disk >80%, app down 2min, error rate >5%

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:59:44 +01:00

45 lines
1.3 KiB
JSON

{
"dashboard": {
"title": "Infrastructure",
"uid": "trails-infra",
"timezone": "browser",
"refresh": "30s",
"panels": [
{
"title": "Process Memory (RSS)",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
"targets": [
{ "expr": "process_resident_memory_bytes", "legendFormat": "{{job}}" }
],
"fieldConfig": { "defaults": { "unit": "bytes" } }
},
{
"title": "CPU Usage",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 0 },
"targets": [
{ "expr": "rate(process_cpu_seconds_total[5m])", "legendFormat": "{{job}}" }
],
"fieldConfig": { "defaults": { "unit": "percentunit" } }
},
{
"title": "Event Loop Lag",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
"targets": [
{ "expr": "nodejs_eventloop_lag_seconds", "legendFormat": "{{job}}" }
],
"fieldConfig": { "defaults": { "unit": "s" } }
},
{
"title": "Open File Descriptors",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
"targets": [
{ "expr": "process_open_fds", "legendFormat": "{{job}}" }
]
}
]
}
}