Add service health monitoring: postgres, node, cAdvisor exporters + dashboard

Exporters:
- postgres_exporter: DB connections, transactions, cache hit ratio, query stats
- node_exporter: host CPU, memory, disk, network
- cAdvisor: per-container CPU and memory usage

PostgreSQL:
- Enable pg_stat_statements for query-level performance tracking
- Track index scans vs sequential scans, cache hit ratio

Dashboard (service-health.json):
- DB: connections, size, transactions/s, slow queries, cache hit ratio, index usage
- Host: disk gauge, CPU, memory, network I/O, disk I/O
- BRouter: request latency p50/p95/p99, container CPU + memory
- All containers: CPU and memory comparison

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-03-27 18:44:14 +01:00
parent f87aca6188
commit 424e692ee0
No known key found for this signature in database
GPG key ID: A32FF691A0F752D9
14 changed files with 326 additions and 12 deletions

View file

@ -4,7 +4,7 @@
- [x] 1.2 Create `.sops.yaml` at repo root with age encryption rule for `secrets.*.env`
- [x] 1.3 Create `infrastructure/secrets.app.env` with app secrets (POSTGRES_PASSWORD, JWT_SECRET, SESSION_SECRET, SMTP_URL, SMTP_FROM, SENTRY_AUTH_TOKEN, DEPLOY_GHCR_TOKEN), encrypt with `sops -e`
- [x] 1.4 Create `infrastructure/secrets.infra.env` with infra secrets (GF_AUTH_GITHUB_CLIENT_ID, GF_AUTH_GITHUB_CLIENT_SECRET), encrypt with `sops -e`
- [ ] 1.5 Remove migrated secrets from GitHub Actions (keep only AGE_SECRET_KEY, DEPLOY_SSH_KEY, DEPLOY_HOST)
- [x] 1.5 Remove migrated secrets from GitHub Actions (keep only AGE_SECRET_KEY, DEPLOY_SSH_KEY, DEPLOY_HOST)
## 2. GitHub OAuth for Grafana
@ -30,7 +30,7 @@
## 4. Verify
- [x] 4.1 Test sops encrypt/decrypt cycle locally
- [ ] 4.2 Test cd-apps deploys only on app changes (not infra)
- [ ] 4.3 Test cd-infra deploys only on infra changes (not apps)
- [ ] 4.4 Test Grafana GitHub OAuth login
- [ ] 4.5 Verify old GitHub secrets can be removed after migration
- [x] 4.2 Test cd-apps deploys only on app changes (not infra)
- [x] 4.3 Test cd-infra deploys only on infra changes (not apps)
- [x] 4.4 Test Grafana GitHub OAuth login
- [x] 4.5 Verify old GitHub secrets can be removed after migration