Add staging + PR-preview environments on the flagship
Implements the staging-environments OpenSpec change. Persistent staging at staging.trails.cool / planner.staging.trails.cool deploys from main; PR opens get a journal-only preview at pr-<N>.staging.trails.cool that shares the persistent planner. cd-staging.yml builds tagged images, manages per-PR Postgres databases and Caddyfile snippets, evicts the oldest preview at the cap of 3, and tears everything down on PR close. staging-cleanup.yml runs weekly to sweep orphaned previews. DNS records (staging + *.staging A/AAAA) already applied to production via tofu. Caddy approach: per-PR Caddyfile snippets imported from /etc/caddy/sites/ and reloaded on each PR event — no wildcard / on-demand TLS, no router service. Production compose gains a trails-shared network for the staging project to reach Postgres, and host.docker.internal on Caddy so it can reverse-proxy staging containers published on the host loopback. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
795ec2215c
commit
c8a7a0b253
11 changed files with 818 additions and 29 deletions
|
|
@ -27,10 +27,10 @@ The server has headroom for lightweight additional containers. Caddy handles aut
|
|||
**Rationale**: A separate Compose project gives clean namespace isolation (container names, volumes) without a second server. Sharing BRouter and monitoring avoids duplicating heavy services.
|
||||
**Alternative considered**: Docker Compose profiles — simpler but risks accidental cross-contamination between production and staging in the same project.
|
||||
|
||||
### 2. Caddy on-demand TLS with wildcard routing
|
||||
**Choice**: Use Caddy's `on_demand_tls` with a wildcard site block for `*.staging.trails.cool`. A small validation endpoint confirms which subdomains are active before Caddy obtains a certificate.
|
||||
**Rationale**: Avoids pre-configuring Caddy for each PR. Caddy automatically provisions TLS certificates on first request. The validation endpoint prevents abuse (random subdomains triggering cert issuance).
|
||||
**Alternative considered**: Wildcard certificate via DNS challenge — requires DNS API credentials and more complex setup.
|
||||
### 2. Per-PR Caddyfile snippets with reload
|
||||
**Choice**: The main Caddyfile uses `import sites/*.caddyfile`. The cd-staging workflow writes a snippet per PR (e.g. `sites/pr-123.caddyfile`) on PR open and removes it on PR close, then reloads Caddy in-place. Standard automatic HTTPS issues a per-host cert.
|
||||
**Rationale**: Caddy can't compute upstream ports from a regex capture (the spec's `3200 + 2N` formula), so a wildcard block would need a sidecar router service. A graceful Caddy reload is fast and idempotent, and the existing cd-apps workflow already reloads Caddy on every deploy — extending that pattern is simpler than adding a new service.
|
||||
**Alternatives considered**: (a) Wildcard `*.staging.trails.cool` block with on-demand TLS plus a small Node router that handles Caddy's `ask` check and wildcard host→port proxying — more moving parts. (b) DNS-challenge wildcard certificate — requires DNS API credentials in Caddy and more complex setup.
|
||||
|
||||
### 3. Per-PR databases in shared PostgreSQL
|
||||
**Choice**: PR previews use per-PR databases (`trails_pr_123`) in the production PostgreSQL instance. Staging uses `trails_staging`. Created by the workflow, dropped on PR close.
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Caddy reverse proxy routing
|
||||
Caddy SHALL route requests to staging and PR preview containers via wildcard subdomain matching, in addition to the existing production routing.
|
||||
Caddy SHALL route requests to staging and PR preview containers via per-host site blocks, in addition to the existing production routing.
|
||||
|
||||
#### Scenario: Staging subdomain routing
|
||||
- **WHEN** a request arrives for `staging.trails.cool`
|
||||
|
|
@ -13,12 +13,14 @@ Caddy SHALL route requests to staging and PR preview containers via wildcard sub
|
|||
|
||||
#### Scenario: PR preview routing
|
||||
- **WHEN** a request arrives for `pr-123.staging.trails.cool`
|
||||
- **THEN** Caddy proxies it to the PR 123 journal container on the correct dynamically assigned port
|
||||
- **THEN** Caddy proxies it to the PR 123 journal container on its assigned port (`3200 + 2N`)
|
||||
|
||||
#### Scenario: On-demand TLS for staging subdomains
|
||||
- **WHEN** a first request arrives for a new staging subdomain
|
||||
- **THEN** Caddy automatically provisions a TLS certificate via Let's Encrypt
|
||||
- **AND** a validation endpoint confirms the subdomain is an active staging/preview environment before certificate issuance
|
||||
#### Scenario: Per-PR Caddyfile snippet lifecycle
|
||||
- **WHEN** a PR preview is deployed
|
||||
- **THEN** the cd-staging workflow writes a Caddyfile snippet at `sites/pr-<N>.caddyfile` and reloads Caddy
|
||||
- **WHEN** a PR is closed
|
||||
- **THEN** the workflow removes the snippet and reloads Caddy
|
||||
- **AND** standard automatic HTTPS issues / retains the per-host certificate via Let's Encrypt
|
||||
|
||||
### Requirement: Docker Compose deployment
|
||||
The staging environment SHALL be deployed as a separate Docker Compose project alongside production on the same server.
|
||||
|
|
|
|||
|
|
@ -1,38 +1,38 @@
|
|||
## 1. DNS & TLS Setup
|
||||
|
||||
- [ ] 1.1 Add wildcard DNS record `*.staging.trails.cool` pointing to the Hetzner server IP
|
||||
- [ ] 1.2 Add `staging.trails.cool` and `planner.staging.trails.cool` DNS A records
|
||||
- [x] 1.1 Add wildcard DNS record `*.staging.trails.cool` pointing to the Hetzner server IP
|
||||
- [x] 1.2 Add `staging.trails.cool` and `planner.staging.trails.cool` DNS A records (planner.staging covered by the wildcard)
|
||||
|
||||
## 2. Docker Compose Staging Configuration
|
||||
|
||||
- [ ] 2.1 Create `infrastructure/docker-compose.staging.yml` with staging journal (port 3100), planner (port 3101), memory limits (256MB), and `trails_staging` database URL
|
||||
- [ ] 2.2 Create `infrastructure/staging.env.template` documenting required staging environment variables (DOMAIN, DATABASE_URL, JWT_SECRET, SESSION_SECRET)
|
||||
- [ ] 2.3 Add a shared Docker network (`trails-shared`) to production `docker-compose.yml` so staging can reach BRouter and PostgreSQL
|
||||
- [x] 2.1 Create `infrastructure/docker-compose.staging.yml` with staging journal (port 3100), planner (port 3101), memory limits (256MB), and `trails_staging` database URL
|
||||
- [x] 2.2 Create `infrastructure/staging.env.template` documenting required staging environment variables (DOMAIN, DATABASE_URL, JWT_SECRET, SESSION_SECRET)
|
||||
- [x] 2.3 Add a shared Docker network (`trails-shared`) to production `docker-compose.yml` so staging can reach BRouter and PostgreSQL
|
||||
- [ ] 2.4 Verify staging containers start with `docker compose -f docker-compose.staging.yml -p trails-staging up -d` on the server
|
||||
|
||||
## 3. Caddy Wildcard Routing
|
||||
|
||||
- [ ] 3.1 Add `staging.trails.cool` site block proxying to journal on port 3100
|
||||
- [ ] 3.2 Add `planner.staging.trails.cool` site block proxying to planner on port 3101
|
||||
- [ ] 3.3 Add `*.staging.trails.cool` wildcard site block with on-demand TLS for PR previews — extract PR number from subdomain, proxy to `localhost:3200 + (PR * 2)`
|
||||
- [ ] 3.4 Create a TLS validation endpoint (small script or Caddy matcher) that checks if the requested subdomain corresponds to a running container
|
||||
- [x] 3.1 Add `staging.trails.cool` site block proxying to journal on port 3100
|
||||
- [x] 3.2 Add `planner.staging.trails.cool` site block proxying to planner on port 3101
|
||||
- [x] 3.3 Add `import sites/*.caddyfile` to the main Caddyfile so per-PR site blocks can be dropped in and picked up on reload
|
||||
- [x] 3.4 Define the per-PR Caddyfile snippet template the cd-staging workflow writes for each preview (host = `pr-<N>.staging.trails.cool`, upstream = `host.docker.internal:<port>`, port = `3200 + 2N`)
|
||||
- [ ] 3.5 Reload Caddy and verify staging routes work with `curl -sf https://staging.trails.cool/api/health`
|
||||
|
||||
## 4. GitHub Actions Workflow
|
||||
|
||||
- [ ] 4.1 Create `.github/workflows/cd-staging.yml` triggered on push to main (paths: `apps/`, `packages/`) and on PR open/synchronize/close (same paths)
|
||||
- [ ] 4.2 Implement the **staging deploy** job: build images, SSH to server, `docker compose -f docker-compose.staging.yml -p trails-staging pull && up -d`, run Drizzle push against `trails_staging`
|
||||
- [ ] 4.3 Implement the **PR preview deploy** job: compute ports from PR number, create `trails_pr_<number>` database if not exists, build images tagged with PR number, deploy containers, post preview URL as PR comment
|
||||
- [ ] 4.4 Implement the **PR preview teardown** job: stop and remove PR containers, drop `trails_pr_<number>` database, delete PR comment
|
||||
- [ ] 4.5 Add the concurrent preview limit check: if >3 active previews, tear down the oldest before deploying a new one
|
||||
- [x] 4.1 Create `.github/workflows/cd-staging.yml` triggered on push to main (paths: `apps/`, `packages/`) and on PR open/synchronize/close (same paths)
|
||||
- [x] 4.2 Implement the **staging deploy** job: build images, SSH to server, `docker compose -f docker-compose.staging.yml -p trails-staging pull && up -d`, run Drizzle push against `trails_staging`
|
||||
- [x] 4.3 Implement the **PR preview deploy** job: compute ports from PR number, create `trails_pr_<number>` database if not exists, build images tagged with PR number, deploy containers, post preview URL as PR comment
|
||||
- [x] 4.4 Implement the **PR preview teardown** job: stop and remove PR containers, drop `trails_pr_<number>` database, delete PR comment
|
||||
- [x] 4.5 Add the concurrent preview limit check: if >3 active previews, tear down the oldest before deploying a new one
|
||||
|
||||
## 5. Cleanup & Safety
|
||||
|
||||
- [ ] 5.1 Create a scheduled cleanup job (weekly cron in GitHub Actions or pg-boss on the server) that lists running `trails-pr-*` containers, checks PR status via `gh pr view`, and tears down orphans
|
||||
- [ ] 5.2 Add memory limits (`deploy.resources.limits.memory: 256m`) to staging containers in the compose override
|
||||
- [x] 5.1 Create a scheduled cleanup job (weekly cron in GitHub Actions or pg-boss on the server) that lists running `trails-pr-*` containers, checks PR status via `gh pr view`, and tears down orphans
|
||||
- [x] 5.2 Add memory limits (`deploy.resources.limits.memory: 256m`) to staging containers in the compose override
|
||||
- [ ] 5.3 Test full lifecycle: open a test PR → verify preview deploys → push a commit → verify preview updates → close PR → verify teardown
|
||||
|
||||
## 6. Documentation
|
||||
|
||||
- [ ] 6.1 Add a "Staging & Previews" section to CLAUDE.md documenting the staging URL, PR preview URL pattern, port scheme, and how to debug staging issues
|
||||
- [ ] 6.2 Update the Deployment table in CLAUDE.md with the new `cd-staging.yml` workflow
|
||||
- [x] 6.1 Add a "Staging & Previews" section to CLAUDE.md documenting the staging URL, PR preview URL pattern, port scheme, and how to debug staging issues
|
||||
- [x] 6.2 Update the Deployment table in CLAUDE.md with the new `cd-staging.yml` workflow
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue