trails-cool/trails
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
trails/openspec/changes/archive/2026-03-27-sops-age-split-cd/proposal.md
Ullrich Schäfer 424e692ee0
Add service health monitoring: postgres, node, cAdvisor exporters + dashboard 
Exporters:
- postgres_exporter: DB connections, transactions, cache hit ratio, query stats
- node_exporter: host CPU, memory, disk, network
- cAdvisor: per-container CPU and memory usage

PostgreSQL:
- Enable pg_stat_statements for query-level performance tracking
- Track index scans vs sequential scans, cache hit ratio

Dashboard (service-health.json):
- DB: connections, size, transactions/s, slow queries, cache hit ratio, index usage
- Host: disk gauge, CPU, memory, network I/O, disk I/O
- BRouter: request latency p50/p95/p99, container CPU + memory
- All containers: CPU and memory comparison

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:50:42 +01:00

1.9 KiB
Raw Blame History

Why

Secrets are scattered across GitHub Actions secrets with no version control, no audit trail, and painful manual management (the Grafana password hash saga). The CD pipeline is monolithic — changing a Grafana dashboard rebuilds both app Docker images. And Grafana authentication requires managing bcrypt hashes and basic auth layers.

What Changes

  • SOPS + age for secrets: Encrypt a .env.production file in the repo. CD decrypts at deploy time with a single age private key stored as one GitHub secret. All other secrets move from GitHub Actions secrets into the encrypted file — version-controlled, diffable, auditable.
  • Split CD into apps vs infra: Two workflows triggered by path filters. App changes (apps/, packages/) build Docker images and deploy. Infra changes (infrastructure/) copy configs and restart services. No unnecessary rebuilds.
  • GitHub OAuth for Grafana: Replace Caddy basic auth + Grafana login with GitHub OAuth. One login, restricted to the trails-cool GitHub org. Remove GRAFANA_PASSWORD_HASH, GRAFANA_USER, GRAFANA_PASSWORD secrets entirely.

Capabilities

New Capabilities

  • secret-management: SOPS + age encrypted secrets in the repository with CD decryption

Modified Capabilities

  • infrastructure: Split CD workflows, GitHub OAuth for Grafana, remove Caddy basic auth for Grafana

Impact

  • Files: New .env.production.enc (encrypted), .sops.yaml config, split cd-apps.yml and cd-infra.yml workflows, updated docker-compose.yml and Caddyfile
  • Dependencies: sops and age CLI tools in CD runner (install step)
  • GitHub secrets: Reduced from ~10 secrets to 2 (AGE_SECRET_KEY + DEPLOY_SSH_KEY). Everything else moves into the encrypted env file.
  • Grafana: GitHub OAuth app registration needed (Client ID + Secret go into the SOPS-encrypted file)
  • Caddy: Remove basic auth block for grafana.internal, just proxy through