SOPS+age secrets, split CD workflows, GitHub OAuth for Grafana
Secrets: - Add .sops.yaml with age encryption config - Add encrypted secrets.app.env (app secrets) and secrets.infra.env (Grafana OAuth) - CD decrypts at deploy time with AGE_SECRET_KEY — all other secrets move out of GitHub Actions into version-controlled encrypted files Split CD: - cd-apps.yml: triggered by apps/packages changes, builds Docker images, deploys apps - cd-infra.yml: triggered by infrastructure/ changes, copies configs, restarts services - Remove monolithic cd.yml Grafana auth: - GitHub OAuth (trails-cool org), disable login form - Remove Caddy basic_auth block and all GRAFANA_* env vars/secrets Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
206f6ac941
commit
57094323d2
15 changed files with 541 additions and 138 deletions
126
openspec/changes/sops-age-split-cd/design.md
Normal file
126
openspec/changes/sops-age-split-cd/design.md
Normal file
|
|
@ -0,0 +1,126 @@
|
|||
## Context
|
||||
|
||||
The current CD workflow has ~10 GitHub Actions secrets passed via `export`
|
||||
statements in a deploy script. Some secrets contain `$` characters that get
|
||||
mangled. Grafana authentication required managing a bcrypt hash, a plaintext
|
||||
password, and a username across three different systems (Caddy, Grafana, CD).
|
||||
The single CD workflow rebuilds Docker images even for config-only changes.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- All secrets in one encrypted file in the repo (SOPS + age)
|
||||
- Only one GitHub secret needed for decryption (AGE_SECRET_KEY)
|
||||
- App deploys (Docker build + push) separate from infra deploys (config copy)
|
||||
- Grafana login via GitHub OAuth (no passwords)
|
||||
- Remove Caddy basic auth for Grafana
|
||||
|
||||
**Non-Goals:**
|
||||
- Secret rotation automation (manual for now)
|
||||
- Per-environment secret files (only production)
|
||||
- Grafana RBAC / team-based access (just org membership check)
|
||||
- Moving DEPLOY_SSH_KEY or GITHUB_TOKEN into SOPS (these are GitHub-native)
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1: SOPS + age file-based encryption
|
||||
|
||||
Create `.sops.yaml` at the repo root defining age as the encryption method.
|
||||
Two encrypted secret files, split by deployment scope:
|
||||
|
||||
- **`infrastructure/secrets.app.env`** — app secrets needed by journal/planner.
|
||||
Used by both cd-apps (production) and future staging deploys.
|
||||
- **`infrastructure/secrets.infra.env`** — monitoring/Grafana secrets.
|
||||
Used only by cd-infra (production). Staging doesn't run Grafana.
|
||||
|
||||
```yaml
|
||||
# .sops.yaml
|
||||
creation_rules:
|
||||
- path_regex: secrets\..*\.env$
|
||||
age: <public-key>
|
||||
```
|
||||
|
||||
Workflow:
|
||||
- **Edit secrets**: `sops infrastructure/secrets.app.env` (decrypts in editor,
|
||||
re-encrypts on save)
|
||||
- **CD decrypts**: Install sops + age, decrypt the relevant file(s) to a temp
|
||||
`.env`, pass to `docker compose --env-file`
|
||||
- **Only one GitHub secret**: `AGE_SECRET_KEY` (the private key)
|
||||
|
||||
**secrets.app.env** contents (used by cd-apps + cd-infra):
|
||||
- POSTGRES_PASSWORD, JWT_SECRET, SESSION_SECRET
|
||||
- SMTP_URL, SMTP_FROM
|
||||
- SENTRY_AUTH_TOKEN
|
||||
- DEPLOY_GHCR_TOKEN
|
||||
|
||||
**secrets.infra.env** contents (used by cd-infra only):
|
||||
- GF_AUTH_GITHUB_CLIENT_ID, GF_AUTH_GITHUB_CLIENT_SECRET
|
||||
|
||||
Secrets that stay as GitHub Actions secrets:
|
||||
- DEPLOY_SSH_KEY (used by SCP/SSH actions, not by docker-compose)
|
||||
- AGE_SECRET_KEY (chicken-and-egg: can't encrypt the decryption key)
|
||||
- DEPLOY_HOST (not really secret, but convenient)
|
||||
|
||||
### D2: Split CD into two workflows
|
||||
|
||||
**cd-apps.yml** — triggered by changes to `apps/`, `packages/`, `pnpm-lock.yaml`:
|
||||
1. Build and push Docker images (journal, planner, brouter)
|
||||
2. SSH to server, pull images, run migrations, restart app containers
|
||||
|
||||
**cd-infra.yml** — triggered by changes to `infrastructure/`:
|
||||
1. Copy config files (docker-compose, Caddyfile, Prometheus, Loki, Grafana)
|
||||
2. SSH to server, decrypt secrets, `docker compose up -d`
|
||||
|
||||
Both workflows also trigger on `workflow_dispatch` for manual runs.
|
||||
Both share the decrypt-secrets step.
|
||||
|
||||
### D3: GitHub OAuth for Grafana
|
||||
|
||||
Register a GitHub OAuth app at github.com/settings/applications:
|
||||
- Callback URL: `https://grafana.internal.trails.cool/login/github`
|
||||
- Homepage: `https://grafana.internal.trails.cool`
|
||||
|
||||
Grafana config via environment variables:
|
||||
```yaml
|
||||
GF_AUTH_GITHUB_ENABLED: "true"
|
||||
GF_AUTH_GITHUB_CLIENT_ID: <from-sops>
|
||||
GF_AUTH_GITHUB_CLIENT_SECRET: <from-sops>
|
||||
GF_AUTH_GITHUB_ALLOWED_ORGANIZATIONS: trails-cool
|
||||
GF_AUTH_GITHUB_SCOPES: user:email,read:org
|
||||
GF_AUTH_DISABLE_LOGIN_FORM: "true"
|
||||
```
|
||||
|
||||
Remove from Caddyfile:
|
||||
- `basic_auth` block for grafana.internal
|
||||
- GRAFANA_USER, GRAFANA_PASSWORD, GRAFANA_PASSWORD_HASH env vars
|
||||
|
||||
Remove from docker-compose.yml:
|
||||
- GF_SECURITY_ADMIN_USER, GF_SECURITY_ADMIN_PASSWORD
|
||||
- Caddy GRAFANA_USER, GRAFANA_PASSWORD_HASH env vars
|
||||
|
||||
Remove from CD:
|
||||
- grafana cli password reset step
|
||||
- All GRAFANA_* secret exports
|
||||
|
||||
### D4: Server-side secret decryption
|
||||
|
||||
The CD deploy step:
|
||||
1. Install sops + age on the runner
|
||||
2. Decrypt: `SOPS_AGE_KEY=${{ secrets.AGE_SECRET_KEY }} sops -d infrastructure/secrets.env > /tmp/secrets.env`
|
||||
3. SCP the decrypted `.env` to server as `/opt/trails-cool/.env`
|
||||
4. `docker compose --env-file .env up -d`
|
||||
5. Clean up `/tmp/secrets.env` from runner
|
||||
|
||||
The server never has the age private key — secrets arrive as a plain `.env`
|
||||
file via SCP (same security as current `export` approach, but now
|
||||
version-controlled and auditable).
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **age key loss** → If the AGE_SECRET_KEY is lost, secrets can't be decrypted.
|
||||
Mitigate: store a backup of the age key outside GitHub (e.g., password
|
||||
manager).
|
||||
- **Encrypted file merge conflicts** → SOPS encrypted files don't merge well.
|
||||
Mitigate: only one person edits secrets at a time (fine for solo/small team).
|
||||
- **GitHub OAuth requires internet** → If GitHub is down, Grafana login fails.
|
||||
Acceptable for a monitoring dashboard.
|
||||
Loading…
Add table
Add a link
Reference in a new issue