Archive local-dev-stack change; sync delta spec to main

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 23:10:58 +02:00 · 2026-05-17 23:10:58 +02:00 · 970e0a0755
commit 970e0a0755
parent 03791e981d
6 changed files with 28 additions and 0 deletions
--- a/openspec/changes/archive/2026-05-17-local-dev-stack/.openspec.yaml
+++ b/openspec/changes/archive/2026-05-17-local-dev-stack/.openspec.yaml
@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-03-28
--- a/openspec/changes/archive/2026-05-17-local-dev-stack/design.md
+++ b/openspec/changes/archive/2026-05-17-local-dev-stack/design.md
@ -0,0 +1,151 @@
+## Context
+
+trails.cool runs `docker-compose.dev.yml` for local dev (PostgreSQL + BRouter)
+and `infrastructure/docker-compose.yml` for production (full stack with Caddy,
+Prometheus, Grafana, Loki, exporters). There is no staging environment. CI runs
+E2E tests with a manually started PostgreSQL container and a separately
+downloaded BRouter, not reusing the dev compose file. The gap between local dev
+and production causes issues:
+
+1. Monitoring changes (alert rules, dashboards) are untestable before deploy
+2. CI's PostgreSQL setup diverges from both dev and prod (no PostGIS extensions
+   preloaded, no pg_stat_statements, no init scripts)
+3. The dev PostgreSQL lacks `pg_stat_statements` which means queries that depend
+   on it (like Grafana datasource queries) fail locally
+
+## Goals / Non-Goals
+
+**Goals:**
+- Local dev PostgreSQL matches production config (pg_stat_statements, init
+  scripts)
+- Optional monitoring stack available locally via a compose profile
+- CI E2E tests use the same compose file as local dev
+- One-command reset for a clean dev environment
+- Seed data available for both local dev and CI
+
+**Non-Goals:**
+- Replicating the production Caddy reverse proxy locally (apps run natively
+  with Vite, no TLS needed for local dev)
+- Running S3/Garage locally (media storage is a future concern)
+- Federation testing (ActivityPub requires publicly reachable endpoints)
+- Matching exact production image versions (dev uses source builds, prod uses
+  GHCR images)
+
+## Decisions
+
+### D1: Extend docker-compose.dev.yml with profiles, don't create a new file
+
+Add monitoring services to the existing `docker-compose.dev.yml` using Docker
+Compose profiles. The core services (postgres, brouter) have no profile
+assigned and always start. Monitoring services get the `monitoring` profile
+and only start when explicitly requested.
+
+```bash
+pnpm dev:services                              # postgres + brouter (default)
+docker compose -f docker-compose.dev.yml --profile monitoring up -d  # + monitoring
+```
+
+**Alternative**: Separate `docker-compose.monitoring.yml` with `extends`.
+Rejected — profiles are the standard Docker Compose mechanism for this, and a
+single file is simpler to maintain.
+
+### D2: Service profiles — core always runs, monitoring is opt-in
+
+Two logical groups:
+
+| Profile | Services | When |
+|---------|----------|------|
+| *(none)* | postgres, brouter | Always — required for app development |
+| `monitoring` | prometheus, grafana, loki | Opt-in — for testing observability changes |
+
+Production-only services NOT included locally: Caddy (apps run natively),
+node-exporter (host metrics not useful in Docker Desktop), cadvisor (container
+metrics not useful locally), postgres-exporter (can add later if needed).
+
+Grafana runs with anonymous auth locally (no GitHub OAuth), connecting to
+the local Prometheus and Loki instances.
+
+### D3: Database initialization — auto-push schema, seed script for test data
+
+On `pnpm dev:full`, the `scripts/dev.sh` script already runs `pnpm db:push`.
+Add a seed script (`scripts/seed.ts`) that inserts test data:
+
+- A test user account in Journal
+- A sample route with waypoints (Berlin area, matching the BRouter segment)
+- A sample activity linked to the route
+
+The seed script is idempotent (uses `ON CONFLICT DO NOTHING`). It runs
+automatically in `dev:full` but can be run standalone with `pnpm db:seed`.
+
+For CI, the seed script runs after `db:push` and before E2E tests, ensuring
+tests have consistent data to work with.
+
+### D4: CI uses compose file for services
+
+Replace the manual `docker run` and BRouter download steps in `.github/
+workflows/ci.yml` with:
+
+```yaml
+- name: Start services
+  run: docker compose -f docker-compose.dev.yml up -d --wait
+```
+
+The `--wait` flag blocks until health checks pass, replacing the manual
+`pg_isready` loops. BRouter still builds from the local Dockerfile in
+`docker/brouter/` and uses the same segment download mechanism.
+
+Benefits:
+- CI and local dev use identical service configuration
+- Health check logic is defined once (in compose) not twice (compose + CI)
+- Simpler CI workflow with fewer steps
+
+Trade-off: Docker Compose in CI adds ~5s overhead for compose parsing. The
+PostGIS image is already cached. BRouter segment download is already cached.
+Net time should be similar or faster due to parallel health checks.
+
+### D5: dev.sh improvements — health checks, error messages, monitoring flag
+
+Improve `scripts/dev.sh`:
+
+1. **Health check with timeout**: Use `docker compose up -d --wait` instead
+   of manual `pg_isready` loop. This respects the healthcheck config in the
+   compose file and has a built-in timeout.
+2. **Error messages**: If Docker is not running, print a clear message instead
+   of a cryptic error. Check for Docker before anything else.
+3. **Monitoring flag**: `pnpm dev:full -- --monitoring` starts the monitoring
+   profile alongside core services.
+4. **Seed data**: Run seed script after schema push.
+
+### D6: Environment — .env.development template with sensible defaults
+
+Create `.env.development` (gitignored) from `.env.development.example`
+(committed). All values have working defaults so local dev works with zero
+configuration:
+
+```env
+DATABASE_URL=postgres://trails:trails@localhost:5432/trails
+BROUTER_URL=http://localhost:17777
+JWT_SECRET=dev-secret-not-for-production
+SESSION_SECRET=dev-secret-not-for-production
+```
+
+The apps already read `DATABASE_URL` from environment. The `.env.development`
+file is for documentation and convenience — `scripts/dev.sh` sets these
+values if not already present.
+
+## Risks / Trade-offs
+
+**[Monitoring profile adds image pulls]** → First `--profile monitoring` run
+downloads Prometheus, Grafana, Loki images (~500MB). Mitigation: one-time
+cost, cached by Docker.
+
+**[Compose in CI needs Docker Compose v2]** → GitHub Actions ubuntu-latest
+includes Docker Compose v2. No action needed.
+
+**[Seed data can drift from schema]** → If schema changes, seed script may
+break. Mitigation: seed script uses Drizzle ORM (not raw SQL), so TypeScript
+catches drift at compile time.
+
+**[BRouter segment download in CI]** → The compose file builds BRouter from
+Dockerfile but doesn't include segments. CI still needs the segment download
+step (cached). The compose file mounts a local directory for segments.
--- a/openspec/changes/archive/2026-05-17-local-dev-stack/proposal.md
+++ b/openspec/changes/archive/2026-05-17-local-dev-stack/proposal.md
@ -0,0 +1,46 @@
+## Why
+
+Infra changes (Prometheus alerts, Caddy config, Grafana dashboards) can only be
+validated against staging or production — there is no way to test them locally
+first. Meanwhile, CI E2E tests skip ~15 of 20 tests because there is no
+PostgreSQL service in the workflow, and BRouter was only recently added to CI
+with manual setup instead of reusing the dev compose file. The existing
+`docker-compose.dev.yml` covers PostgreSQL and BRouter but nothing else from the
+production stack.
+
+## What Changes
+
+- Extend `docker-compose.dev.yml` with an optional monitoring profile
+  (Prometheus, Grafana, Loki) so developers can test observability changes
+  locally before deploying to production
+- Add `pg_stat_statements` and initialization scripts to the dev PostgreSQL
+  to match production config
+- Improve `scripts/dev.sh` with proper health checks and better error output
+- Simplify CI by using the same compose file instead of ad-hoc `docker run`
+  commands
+- Add a database seed script for consistent test data
+- Add a `pnpm dev:reset` command to tear down and recreate the local stack
+
+## Capabilities
+
+### New Capabilities
+
+- `local-monitoring`: Optional local Prometheus + Grafana + Loki stack via
+  `--profile monitoring`, matching production monitoring configuration
+- `dev-reset`: One command to wipe and recreate the local dev environment
+
+### Modified Capabilities
+
+- `local-dev-environment`: PostgreSQL config aligned with production
+  (pg_stat_statements, init scripts), improved health checks, seed data
+
+## Impact
+
+- **docker-compose.dev.yml**: Add monitoring services behind a profile, update
+  postgres config to match production
+- **.github/workflows/ci.yml**: Replace manual `docker run` with compose-based
+  service startup
+- **scripts/dev.sh**: Better health checks, error messages, optional monitoring
+- **scripts/reset-dev.sh**: New script to wipe volumes and restart
+- **scripts/seed.ts**: Test data for local development and E2E tests
+- **Dependencies**: None new (all Docker images already used in production)
--- a/openspec/changes/archive/2026-05-17-local-dev-stack/specs/local-dev-environment/spec.md
+++ b/openspec/changes/archive/2026-05-17-local-dev-stack/specs/local-dev-environment/spec.md
@ -0,0 +1,29 @@
+## MODIFIED Requirements
+
+### Requirement: Optional local monitoring stack
+The dev environment SHALL support an optional monitoring profile matching the production stack.
+
+#### Scenario: Start with monitoring
+- **WHEN** a developer runs `pnpm dev:full` with `--profile monitoring`
+- **THEN** Prometheus, Grafana, and Loki start alongside the app services
+
+### Requirement: Production-aligned PostgreSQL config
+The dev PostgreSQL SHALL match production configuration including pg_stat_statements.
+
+#### Scenario: pg_stat_statements available
+- **WHEN** the dev PostgreSQL container starts
+- **THEN** pg_stat_statements is enabled via initialization scripts
+
+### Requirement: Database seed script
+The dev environment SHALL provide a seed script for consistent test data.
+
+#### Scenario: Seed database
+- **WHEN** a developer runs the seed script
+- **THEN** test users, routes, and activities are created in the local database
+
+### Requirement: Dev environment reset
+The dev environment SHALL provide a command to tear down and recreate the local stack.
+
+#### Scenario: Reset dev environment
+- **WHEN** a developer runs `pnpm dev:reset`
+- **THEN** all Docker volumes are removed, containers are recreated, and the database is re-seeded
--- a/openspec/changes/archive/2026-05-17-local-dev-stack/tasks.md
+++ b/openspec/changes/archive/2026-05-17-local-dev-stack/tasks.md
@ -0,0 +1,34 @@
+## 1. Docker Compose
+
+- [x] 1.1 Update dev PostgreSQL to match production: add `shared_preload_libraries=pg_stat_statements` command and mount `infrastructure/postgres/init-grafana-user.sql` as init script
+- [x] 1.2 Add Prometheus service with `monitoring` profile, mounting `infrastructure/prometheus/prometheus.yml`
+- [x] 1.3 Add Grafana service with `monitoring` profile, anonymous auth enabled, provisioned with local Prometheus and Loki datasources
+- [x] 1.4 Add Loki service with `monitoring` profile, mounting `infrastructure/loki/loki-config.yml`
+
+## 2. Database
+
+- [x] 2.1 Create `scripts/seed.ts` with idempotent test data: user account, sample route (Berlin area), sample activity. Use Drizzle ORM, `ON CONFLICT DO NOTHING`
+- [x] 2.2 Add `pnpm db:seed` script to root package.json that runs `scripts/seed.ts` with `--experimental-strip-types`
+
+## 3. CI Integration
+
+- [x] 3.1 Replace manual `docker run` PostgreSQL setup in `ci.yml` e2e job with `docker compose -f docker-compose.dev.yml up -d --wait`
+- [x] 3.2 Replace manual BRouter download/start in `ci.yml` with the compose BRouter service plus cached segment mount
+- [x] 3.3 Add `pnpm db:seed` step after `pnpm db:push` in CI e2e job
+- [x] 3.4 Remove any E2E test skip workarounds that check for DB availability (the `withDb()` 503 skip pattern)
+
+## 4. Scripts
+
+- [x] 4.1 Improve `scripts/dev.sh`: check Docker is running first, use `docker compose up -d --wait`, add `--monitoring` flag to start monitoring profile, run seed after schema push
+- [x] 4.2 Create `scripts/reset-dev.sh`: stop containers, remove volumes, restart — add as `pnpm dev:reset` in package.json
+
+## 5. Environment
+
+- [x] 5.1 Create `.env.development.example` with documented defaults (DATABASE_URL, BROUTER_URL, JWT_SECRET, SESSION_SECRET) and add `.env.development` to `.gitignore`
+
+## 6. Verify
+
+- [x] 6.1 Test full local dev flow: `pnpm dev:full` starts services, pushes schema, seeds data, launches apps — create session, compute route, verify seeded data visible
+- [x] 6.2 Test monitoring profile: `pnpm dev:full -- --monitoring` starts Prometheus + Grafana + Loki, verify Grafana dashboards load at localhost:3002
+- [x] 6.3 Test CI flow: push branch, verify e2e job uses compose, all ~20 E2E tests run (none skipped)
+- [x] 6.4 Test reset: `pnpm dev:reset` wipes volumes and restarts cleanly