From 39ebaa9841cf8022a5de8bc09024f99203cc8dcd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ullrich=20Sch=C3=A4fer?= Date: Tue, 14 Apr 2026 09:02:23 +0200 Subject: [PATCH] Add OpenSpec proposal for staging environments Proposal, design, specs, and tasks for persistent staging instance and ephemeral PR preview environments on the existing Hetzner server. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../staging-environments/.openspec.yaml | 2 + .../changes/staging-environments/design.md | 55 +++++++++++++++++++ .../changes/staging-environments/proposal.md | 29 ++++++++++ .../specs/infrastructure/spec.md | 32 +++++++++++ .../specs/staging-environment/spec.md | 55 +++++++++++++++++++ .../changes/staging-environments/tasks.md | 38 +++++++++++++ 6 files changed, 211 insertions(+) create mode 100644 openspec/changes/staging-environments/.openspec.yaml create mode 100644 openspec/changes/staging-environments/design.md create mode 100644 openspec/changes/staging-environments/proposal.md create mode 100644 openspec/changes/staging-environments/specs/infrastructure/spec.md create mode 100644 openspec/changes/staging-environments/specs/staging-environment/spec.md create mode 100644 openspec/changes/staging-environments/tasks.md diff --git a/openspec/changes/staging-environments/.openspec.yaml b/openspec/changes/staging-environments/.openspec.yaml new file mode 100644 index 0000000..76a85e8 --- /dev/null +++ b/openspec/changes/staging-environments/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-04-14 diff --git a/openspec/changes/staging-environments/design.md b/openspec/changes/staging-environments/design.md new file mode 100644 index 0000000..23b1a5e --- /dev/null +++ b/openspec/changes/staging-environments/design.md @@ -0,0 +1,55 @@ +## Context + +trails.cool runs on a single Hetzner cx23 server (2 vCPU, 4 GB RAM) with Docker Compose. Production uses Caddy as reverse proxy, PostgreSQL + PostGIS, and three CD workflows (apps, infra, brouter). There is currently no staging or preview environment — changes go straight to production after CI passes. + +The server has headroom for lightweight additional containers. Caddy handles automatic TLS via Let's Encrypt and supports on-demand TLS for wildcard subdomains. + +## Goals / Non-Goals + +**Goals:** +- Persistent staging instance at `staging.trails.cool` that auto-deploys from main +- Ephemeral PR preview environments at `pr-.staging.trails.cool` that spin up on PR open and tear down on PR close +- Full database isolation between production, staging, and each PR preview +- Minimal resource overhead — share BRouter and infrastructure services (Prometheus, Grafana, Loki) with production +- PR previews accessible to anyone with the URL (no auth required for the preview itself) + +**Non-Goals:** +- Staging federation (ActivityPub) — staging instances don't need to federate +- Staging email delivery — use log transport or discard +- Load testing or performance parity with production +- PR previews for infrastructure-only changes +- Separate server provisioning + +## Decisions + +### 1. Shared server, separate Docker Compose projects +**Choice**: Run staging as a separate Docker Compose project (`trails-staging`) on the same server, sharing the host network and BRouter/monitoring containers with production. +**Rationale**: A separate Compose project gives clean namespace isolation (container names, volumes) without a second server. Sharing BRouter and monitoring avoids duplicating heavy services. +**Alternative considered**: Docker Compose profiles — simpler but risks accidental cross-contamination between production and staging in the same project. + +### 2. Caddy on-demand TLS with wildcard routing +**Choice**: Use Caddy's `on_demand_tls` with a wildcard site block for `*.staging.trails.cool`. A small validation endpoint confirms which subdomains are active before Caddy obtains a certificate. +**Rationale**: Avoids pre-configuring Caddy for each PR. Caddy automatically provisions TLS certificates on first request. The validation endpoint prevents abuse (random subdomains triggering cert issuance). +**Alternative considered**: Wildcard certificate via DNS challenge — requires DNS API credentials and more complex setup. + +### 3. Per-PR databases in shared PostgreSQL +**Choice**: PR previews use per-PR databases (`trails_pr_123`) in the production PostgreSQL instance. Staging uses `trails_staging`. Created by the workflow, dropped on PR close. +**Rationale**: PostgreSQL handles multiple databases efficiently. No need for a separate PostgreSQL container per preview. Drizzle Kit `push` handles schema setup. +**Alternative considered**: Separate PostgreSQL container per preview — full isolation but heavy resource cost. + +### 4. Port allocation scheme +**Choice**: Staging journal on port 3100, planner on 3101. PR previews on ports `3200 + (PR number * 2)` for journal and `3201 + (PR number * 2)` for planner. +**Rationale**: Deterministic port mapping from PR number. Production stays on 3000/3001. Port collisions are practically impossible (would need 50+ concurrent PRs). +**Alternative considered**: Docker DNS-based routing — would require a custom network resolver setup. + +### 5. GitHub Actions workflow +**Choice**: Single `cd-staging.yml` workflow handling both staging deploys (on main push) and PR preview lifecycle (on PR open/sync/close). +**Rationale**: Keeps staging logic in one place. Uses `github.event.action` to distinguish between deploy, update, and teardown. + +## Risks / Trade-offs + +- **Resource contention**: Staging and PR previews share CPU/memory with production. → Mitigation: Limit concurrent PR previews (e.g., max 3). Add memory limits to staging containers. Monitor via existing Grafana/cAdvisor. +- **Port exhaustion**: Many concurrent PRs could exhaust the port range. → Mitigation: Port scheme supports ~50 concurrent PRs, far more than needed. Cleanup job runs on PR close. +- **Database isolation**: PR databases share the same PostgreSQL instance as production. → Mitigation: Use separate database names and credentials. PR databases are disposable — created and dropped by the workflow. +- **Stale PR previews**: If a workflow fails to clean up, containers and databases linger. → Mitigation: Add a scheduled cleanup job that checks for closed PRs and removes their resources. +- **TLS rate limits**: Let's Encrypt has rate limits (50 certs/week per registered domain). → Mitigation: `*.staging.trails.cool` previews are subdomains of a single domain, counting as one. On-demand TLS with validation prevents abuse. diff --git a/openspec/changes/staging-environments/proposal.md b/openspec/changes/staging-environments/proposal.md new file mode 100644 index 0000000..9efb4f1 --- /dev/null +++ b/openspec/changes/staging-environments/proposal.md @@ -0,0 +1,29 @@ +## Why + +There is no way to test changes in a production-like environment before merging. The only testing options are local dev (`pnpm dev:full`) or deploying directly to production. This makes it risky to test features that depend on real infrastructure (Caddy TLS, Docker networking, PostgreSQL migrations, OAuth callbacks) and impossible for non-developers (e.g., design reviewers) to preview PRs. A staging environment would catch integration issues earlier and give PR reviewers a live URL to test against. + +## What Changes + +- Add a **persistent staging instance** (`staging.trails.cool` / `planner.staging.trails.cool`) running on the same Hetzner server as production, using a separate Docker Compose project with its own PostgreSQL database, port range, and Caddy configuration +- Add **ephemeral PR preview environments** that spin up automatically when a PR is opened, serve at `pr-.staging.trails.cool`, and tear down when the PR is merged or closed +- Add a **GitHub Actions workflow** (`cd-staging.yml`) that deploys the staging instance on pushes to main and manages PR preview lifecycle +- Add a **Docker Compose override** (`docker-compose.staging.yml`) for staging-specific configuration (ports, database, domain) +- Add **Caddy wildcard routing** for `*.staging.trails.cool` to dynamically route to the correct preview or staging container +- Add a **cleanup job** to tear down PR preview containers and their databases when PRs close + +## Capabilities + +### New Capabilities +- `staging-environment`: Persistent staging instance configuration, PR preview lifecycle, Docker Compose staging overrides, Caddy routing, GitHub Actions integration, database isolation, and cleanup + +### Modified Capabilities +- `infrastructure`: Caddy gains wildcard subdomain routing for staging; Docker Compose gains staging profiles and a staging-specific override file + +## Impact + +- **DNS**: Requires a wildcard DNS record `*.staging.trails.cool` pointing to the same server +- **TLS**: Caddy handles automatic TLS for staging subdomains via Let's Encrypt +- **Disk/memory**: Each PR preview runs journal + planner + a shared staging PostgreSQL instance on the production server. Resource usage scales with active PRs. +- **Database**: Staging uses a separate PostgreSQL database (`trails_staging`). PR previews use per-PR databases (`trails_pr_`), created and dropped by the workflow. +- **Secrets**: Staging shares the same SOPS-encrypted secrets as production (same server), with staging-specific overrides for domain and database. +- **GitHub Actions**: New workflow triggered on PR open/sync/close and main push. diff --git a/openspec/changes/staging-environments/specs/infrastructure/spec.md b/openspec/changes/staging-environments/specs/infrastructure/spec.md new file mode 100644 index 0000000..b670ca0 --- /dev/null +++ b/openspec/changes/staging-environments/specs/infrastructure/spec.md @@ -0,0 +1,32 @@ +## MODIFIED Requirements + +### Requirement: Caddy reverse proxy routing +Caddy SHALL route requests to staging and PR preview containers via wildcard subdomain matching, in addition to the existing production routing. + +#### Scenario: Staging subdomain routing +- **WHEN** a request arrives for `staging.trails.cool` +- **THEN** Caddy proxies it to the staging journal container on port 3100 + +#### Scenario: Planner staging routing +- **WHEN** a request arrives for `planner.staging.trails.cool` +- **THEN** Caddy proxies it to the staging planner container on port 3101 + +#### Scenario: PR preview routing +- **WHEN** a request arrives for `pr-123.staging.trails.cool` +- **THEN** Caddy proxies it to the PR 123 journal container on the correct dynamically assigned port + +#### Scenario: On-demand TLS for staging subdomains +- **WHEN** a first request arrives for a new staging subdomain +- **THEN** Caddy automatically provisions a TLS certificate via Let's Encrypt +- **AND** a validation endpoint confirms the subdomain is an active staging/preview environment before certificate issuance + +### Requirement: Docker Compose deployment +The staging environment SHALL be deployed as a separate Docker Compose project alongside production on the same server. + +#### Scenario: Staging compose project +- **WHEN** the staging deployment runs +- **THEN** it creates containers in the `trails-staging` project namespace, separate from the `trails-cool` production project + +#### Scenario: Shared services +- **WHEN** staging containers need BRouter routing +- **THEN** they connect to the production BRouter container via Docker network, not a duplicate instance diff --git a/openspec/changes/staging-environments/specs/staging-environment/spec.md b/openspec/changes/staging-environments/specs/staging-environment/spec.md new file mode 100644 index 0000000..bd53e20 --- /dev/null +++ b/openspec/changes/staging-environments/specs/staging-environment/spec.md @@ -0,0 +1,55 @@ +## ADDED Requirements + +### Requirement: Persistent staging instance +A persistent staging instance SHALL run at `staging.trails.cool` (journal) and `planner.staging.trails.cool` (planner), auto-deploying from main on every push. + +#### Scenario: Deploy staging on main push +- **WHEN** a commit is pushed to main that changes `apps/` or `packages/` +- **THEN** the staging journal and planner containers are rebuilt and redeployed with the latest main + +#### Scenario: Staging uses isolated database +- **WHEN** the staging instance is running +- **THEN** it uses the `trails_staging` database, separate from the production `trails` database + +#### Scenario: Staging is accessible +- **WHEN** a user navigates to `https://staging.trails.cool` +- **THEN** they see the journal app served over HTTPS with a valid TLS certificate + +### Requirement: PR preview environments +Ephemeral preview environments SHALL be created for each PR that changes app or package code. + +#### Scenario: Preview created on PR open +- **WHEN** a PR is opened that changes files in `apps/` or `packages/` +- **THEN** a preview environment is deployed at `pr-.staging.trails.cool` within 5 minutes +- **AND** a comment is posted on the PR with the preview URL + +#### Scenario: Preview updated on PR push +- **WHEN** new commits are pushed to a PR branch with an active preview +- **THEN** the preview containers are rebuilt and redeployed with the latest branch code + +#### Scenario: Preview torn down on PR close +- **WHEN** a PR is merged or closed +- **THEN** its preview containers are stopped and removed +- **AND** its database (`trails_pr_`) is dropped + +#### Scenario: Preview database isolation +- **WHEN** a PR preview is running +- **THEN** it uses a dedicated database `trails_pr_` with schema applied via Drizzle Kit push + +### Requirement: PR preview cleanup +A scheduled cleanup job SHALL remove orphaned preview resources from closed PRs. + +#### Scenario: Stale preview cleanup +- **WHEN** the cleanup job runs +- **THEN** any preview containers or databases belonging to closed/merged PRs are removed + +### Requirement: Resource limits +Staging and preview containers SHALL have resource limits to protect production. + +#### Scenario: Memory limits enforced +- **WHEN** a staging or preview container is running +- **THEN** it has a memory limit of 256MB per app container + +#### Scenario: Concurrent preview limit +- **WHEN** more than 3 PR previews are active +- **THEN** the oldest preview is torn down before the new one is created diff --git a/openspec/changes/staging-environments/tasks.md b/openspec/changes/staging-environments/tasks.md new file mode 100644 index 0000000..9c0d1cd --- /dev/null +++ b/openspec/changes/staging-environments/tasks.md @@ -0,0 +1,38 @@ +## 1. DNS & TLS Setup + +- [ ] 1.1 Add wildcard DNS record `*.staging.trails.cool` pointing to the Hetzner server IP +- [ ] 1.2 Add `staging.trails.cool` and `planner.staging.trails.cool` DNS A records + +## 2. Docker Compose Staging Configuration + +- [ ] 2.1 Create `infrastructure/docker-compose.staging.yml` with staging journal (port 3100), planner (port 3101), memory limits (256MB), and `trails_staging` database URL +- [ ] 2.2 Create `infrastructure/staging.env.template` documenting required staging environment variables (DOMAIN, DATABASE_URL, JWT_SECRET, SESSION_SECRET) +- [ ] 2.3 Add a shared Docker network (`trails-shared`) to production `docker-compose.yml` so staging can reach BRouter and PostgreSQL +- [ ] 2.4 Verify staging containers start with `docker compose -f docker-compose.staging.yml -p trails-staging up -d` on the server + +## 3. Caddy Wildcard Routing + +- [ ] 3.1 Add `staging.trails.cool` site block proxying to journal on port 3100 +- [ ] 3.2 Add `planner.staging.trails.cool` site block proxying to planner on port 3101 +- [ ] 3.3 Add `*.staging.trails.cool` wildcard site block with on-demand TLS for PR previews — extract PR number from subdomain, proxy to `localhost:3200 + (PR * 2)` +- [ ] 3.4 Create a TLS validation endpoint (small script or Caddy matcher) that checks if the requested subdomain corresponds to a running container +- [ ] 3.5 Reload Caddy and verify staging routes work with `curl -sf https://staging.trails.cool/api/health` + +## 4. GitHub Actions Workflow + +- [ ] 4.1 Create `.github/workflows/cd-staging.yml` triggered on push to main (paths: `apps/`, `packages/`) and on PR open/synchronize/close (same paths) +- [ ] 4.2 Implement the **staging deploy** job: build images, SSH to server, `docker compose -f docker-compose.staging.yml -p trails-staging pull && up -d`, run Drizzle push against `trails_staging` +- [ ] 4.3 Implement the **PR preview deploy** job: compute ports from PR number, create `trails_pr_` database if not exists, build images tagged with PR number, deploy containers, post preview URL as PR comment +- [ ] 4.4 Implement the **PR preview teardown** job: stop and remove PR containers, drop `trails_pr_` database, delete PR comment +- [ ] 4.5 Add the concurrent preview limit check: if >3 active previews, tear down the oldest before deploying a new one + +## 5. Cleanup & Safety + +- [ ] 5.1 Create a scheduled cleanup job (weekly cron in GitHub Actions or pg-boss on the server) that lists running `trails-pr-*` containers, checks PR status via `gh pr view`, and tears down orphans +- [ ] 5.2 Add memory limits (`deploy.resources.limits.memory: 256m`) to staging containers in the compose override +- [ ] 5.3 Test full lifecycle: open a test PR → verify preview deploys → push a commit → verify preview updates → close PR → verify teardown + +## 6. Documentation + +- [ ] 6.1 Add a "Staging & Previews" section to CLAUDE.md documenting the staging URL, PR preview URL pattern, port scheme, and how to debug staging issues +- [ ] 6.2 Update the Deployment table in CLAUDE.md with the new `cd-staging.yml` workflow