Commit graph

1500 commits

Author SHA1 Message Date
Ullrich Schäfer
424eecd274
Add PostgreSQL datasource to Grafana with read-only user
- Create grafana_reader role with SELECT-only access on all schemas
- Init script runs on postgres first boot (docker-entrypoint-initdb.d)
- Grafana PostgreSQL datasource provisioned with read-only credentials
- Enables SQL queries in dashboards (user count, routes, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:07:27 +01:00
Ullrich Schäfer
5e564e7562
Merge pull request #106 from trails-cool/simplify-restart-all
Simplify restart-all to workflow_dispatch only
2026-03-27 19:18:42 +01:00
Ullrich Schäfer
9a7ecd8911
Simplify restart-all: workflow_dispatch only
Remove commit message [restart-all] detection — doesn't work with
GitHub merge commits. Keep only the explicit workflow_dispatch input.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 19:15:22 +01:00
Ullrich Schäfer
daebb63d1e
Merge pull request #105 from trails-cool/service-health-dashboard
Add service health monitoring: DB, host, container exporters + dashboard
2026-03-27 18:53:26 +01:00
Ullrich Schäfer
424e692ee0
Add service health monitoring: postgres, node, cAdvisor exporters + dashboard
Exporters:
- postgres_exporter: DB connections, transactions, cache hit ratio, query stats
- node_exporter: host CPU, memory, disk, network
- cAdvisor: per-container CPU and memory usage

PostgreSQL:
- Enable pg_stat_statements for query-level performance tracking
- Track index scans vs sequential scans, cache hit ratio

Dashboard (service-health.json):
- DB: connections, size, transactions/s, slow queries, cache hit ratio, index usage
- Host: disk gauge, CPU, memory, network I/O, disk I/O
- BRouter: request latency p50/p95/p99, container CPU + memory
- All containers: CPU and memory comparison

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:50:42 +01:00
Ullrich Schäfer
f87aca6188
Merge pull request #104 from trails-cool/fix-docker-ca-certs
Install ca-certificates in Docker images for Sentry uploads
2026-03-27 18:35:51 +01:00
Ullrich Schäfer
48fa94e51b
Install ca-certificates in Docker base image for Sentry uploads
node:25-slim doesn't include CA certificates, causing sentry-cli
SSL errors: "unable to get local issuer certificate". Also removes
debug logging from previous troubleshooting.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:35:46 +01:00
Ullrich Schäfer
f8957902d9
Merge pull request #103 from trails-cool/fix-sentry-token-newline
Fix Sentry auth token trailing newline
2026-03-27 18:23:29 +01:00
Ullrich Schäfer
318f0c93c1
Strip trailing newline from Sentry auth token
The token file may have a trailing newline that causes sentry-cli
API requests to fail with "API request failed".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:23:20 +01:00
Ullrich Schäfer
d121dba2c1
Merge pull request #102 from trails-cool/fix-sentry-build
Fix Sentry build: telemetry, release version, secret passing
2026-03-27 18:18:16 +01:00
Ullrich Schäfer
5d7b8c82e8
Fix Sentry build: disable telemetry, fix release version passing
- Disable Sentry Vite plugin telemetry (telemetry: false)
- Pass SENTRY_RELEASE as build-arg (not secret) since it's just the git sha
- Keep only SENTRY_AUTH_TOKEN as Docker secret

The release version was showing as *** because Docker secrets get masked
by GitHub Actions. Git sha is not sensitive — safe as a build arg.

The sourcemap warning from react-router is a known upstream issue (ships
without sourcemaps) and is harmless.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:18:00 +01:00
Ullrich Schäfer
cdfd23abc2
Merge pull request #101 from trails-cool/fix-pnpm-build-scripts
Fix pnpm build scripts: use onlyBuiltDependencies
2026-03-27 18:09:51 +01:00
Ullrich Schäfer
6b147d04ec
Use pnpm.onlyBuiltDependencies for @sentry/cli and esbuild
pnpm approve-builds in Dockerfile ran after install, so the approval
wasn't picked up. Use onlyBuiltDependencies in package.json instead —
this is read during install and allows build scripts to run.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:07:25 +01:00
Ullrich Schäfer
b2bf7cba28
Merge pull request #100 from trails-cool/fix-docker-sentry
Fix Sentry source map upload in Docker builds
2026-03-27 18:02:34 +01:00
Ullrich Schäfer
5e200c8b62
Fix Sentry source map upload in Docker builds
- Approve @sentry/cli and esbuild build scripts so postinstall runs
- Use --mount=type=secret instead of ARG for SENTRY_AUTH_TOKEN
  (fixes Docker warning about secrets in build args)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:01:54 +01:00
Ullrich Schäfer
87d01b4b0d
Merge pull request #99 from trails-cool/fix-cd-apps-source
Fix cd-apps: don't source env file with angle brackets
2026-03-27 17:53:22 +01:00
Ullrich Schäfer
06916868fe
Fix cd-apps: extract SENTRY_AUTH_TOKEN instead of sourcing env file
The decrypted secrets.app.env contains values with angle brackets
(SMTP_FROM) that bash interprets as redirects. The build job only
needs SENTRY_AUTH_TOKEN, so extract it with grep instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:52:41 +01:00
Ullrich Schäfer
fe87545744
Merge pull request #98 from trails-cool/fix-scp-paths
Fix SCP paths for decrypted secrets in CD workflows
2026-03-27 17:46:40 +01:00
Ullrich Schäfer
e880e3e9d9
Fix SCP path for decrypted secrets in CD workflows
Write decrypted env files to infrastructure/ instead of /tmp/ so
strip_components: 1 correctly places them on the server.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:43:30 +01:00
Ullrich Schäfer
6e39d23dd4
Merge pull request #97 from trails-cool/fix-cd-environments
Add GitHub environments to CD workflows
2026-03-27 17:36:51 +01:00
Ullrich Schäfer
9673496211
Add GitHub environments to CD workflows
- cd-apps: uses `production` environment for AGE_SECRET_KEY
- cd-infra: uses `infra` environment for AGE_SECRET_KEY

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:34:28 +01:00
Ullrich Schäfer
c3e80c2dad
Merge pull request #96 from trails-cool/sops-age-split-cd
SOPS+age secrets, split CD, GitHub OAuth for Grafana
2026-03-27 17:30:26 +01:00
Ullrich Schäfer
57094323d2
SOPS+age secrets, split CD workflows, GitHub OAuth for Grafana
Secrets:
- Add .sops.yaml with age encryption config
- Add encrypted secrets.app.env (app secrets) and secrets.infra.env (Grafana OAuth)
- CD decrypts at deploy time with AGE_SECRET_KEY — all other secrets
  move out of GitHub Actions into version-controlled encrypted files

Split CD:
- cd-apps.yml: triggered by apps/packages changes, builds Docker images, deploys apps
- cd-infra.yml: triggered by infrastructure/ changes, copies configs, restarts services
- Remove monolithic cd.yml

Grafana auth:
- GitHub OAuth (trails-cool org), disable login form
- Remove Caddy basic_auth block and all GRAFANA_* env vars/secrets

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:28:04 +01:00
Ullrich Schäfer
206f6ac941
Merge pull request #95 from trails-cool/fix-grafana-cd-password
Sync Grafana admin password on deploy
2026-03-27 15:13:37 +01:00
Ullrich Schäfer
b01624b31a
Sync Grafana admin password on each deploy
GF_SECURITY_ADMIN_PASSWORD only works on first boot. Reset the admin
password via grafana-cli after each deploy to stay in sync with secrets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 14:53:54 +01:00
Ullrich Schäfer
65f30b952a
Merge pull request #94 from trails-cool/fix-caddy-basicauth
Fix deprecated basicauth directive in Caddyfile
2026-03-27 14:35:03 +01:00
Ullrich Schäfer
73edb1bfed
Use basic_auth instead of deprecated basicauth in Caddyfile
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 14:32:57 +01:00
Ullrich Schäfer
a8ec0a9509
Merge pull request #93 from trails-cool/fix-grafana-datasource-uids
Add explicit UIDs to Grafana datasources
2026-03-27 13:10:35 +00:00
Ullrich Schäfer
83243cea31
Add explicit UIDs to Grafana datasources
Alert rules reference datasourceUid: prometheus/loki which requires
explicit UIDs in the provisioning config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 14:08:21 +01:00
Ullrich Schäfer
5b07e68364
Merge pull request #92 from trails-cool/fix-grafana-alerts
Fix Grafana alert email config
2026-03-27 13:06:07 +00:00
Ullrich Schäfer
c47426213b
Fix Grafana alert provisioning: hardcode email address
Grafana provisioning YAML doesn't support env var substitution.
Replace ${ALERT_EMAIL} with hardcoded admin@trails.cool.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 14:03:48 +01:00
Ullrich Schäfer
4751176da9
Merge pull request #91 from trails-cool/fix-caddy-env
Fix Caddy env vars and Loki compactor config
2026-03-27 13:55:49 +01:00
Ullrich Schäfer
ed33fcf9ae
Deploy monitoring configs (Prometheus, Loki, Grafana) via CD
Add prometheus.yml, loki-config.yml, and grafana provisioning/dashboards
to the SCP step so they're copied to the server on each deploy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 13:55:09 +01:00
Ullrich Schäfer
2ecfe81c08
Pass Grafana env vars to Caddy container, fix Loki compactor config
- Add DOMAIN, GRAFANA_USER, GRAFANA_PASSWORD_HASH as environment vars
  on the Caddy container so Caddyfile can reference them
- Fix Loki compactor: add delete_request_store required when retention is enabled

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 13:53:56 +01:00
Ullrich Schäfer
1e970ae475
Merge pull request #90 from trails-cool/internal-wildcard-dns
Use *.internal.trails.cool wildcard for internal services
2026-03-27 12:22:10 +00:00
Ullrich Schäfer
559f53d8fc
Use *.internal.trails.cool wildcard for internal services
Add wildcard DNS record for *.internal.trails.cool (A + AAAA) so
internal services don't leak hostnames in individual DNS entries.
Move Grafana from grafana.trails.cool to grafana.internal.trails.cool.

Future internal services just need a Caddy block — no DNS changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 13:19:47 +01:00
Ullrich Schäfer
bff9ff04a4
Merge pull request #89 from trails-cool/fix-grafana-deploy
Wire Grafana secrets through CD pipeline
2026-03-27 07:41:54 +00:00
Ullrich Schäfer
e105382c8f
Wire Grafana secrets through CD pipeline
Pass GRAFANA_USER, GRAFANA_PASSWORD, and GRAFANA_PASSWORD_HASH
from GitHub Actions secrets to the deploy script.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 08:39:25 +01:00
Ullrich Schäfer
d9b7a150ca
Merge pull request #88 from trails-cool/observability
Add observability: health, logging, metrics, Grafana
2026-03-27 06:59:05 +00:00
Ullrich Schäfer
49aadd04a9
Add observability: health endpoints, structured logging, metrics, Grafana stack
Health endpoints:
- /api/health (Journal) and /health (Planner) with DB connectivity check
- Docker healthchecks updated to use app health endpoints

Structured logging:
- Pino with JSON output in production, pretty-print in dev
- Request logging middleware in Planner (method, path, status, duration)
- Replaced console.log/error with structured logger in email and auth flows

Prometheus metrics:
- prom-client with default Node.js metrics + custom histograms/gauges
- /metrics endpoints on both apps
- http_request_duration, planner_active_sessions, brouter_request_duration

Monitoring stack:
- Prometheus, Loki, Grafana containers in docker-compose
- Grafana provisioned with datasources, dashboards, and alert rules
- Caddy access logging (JSON to stdout for Loki)
- grafana.trails.cool with basic auth via Caddy

Dashboards and alerting:
- Overview: request rate, error rate, latency p50/p95/p99
- Planner: active sessions, connected clients, BRouter latency
- Infrastructure: memory, CPU, event loop lag
- Alerts: disk >80%, app down 2min, error rate >5%

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:59:44 +01:00
Ullrich Schäfer
ce964cae96
Merge pull request #87 from trails-cool/planner-route-interactions
Update OpenSpec artifacts for route interactions
2026-03-26 21:38:52 +00:00
Ullrich Schäfer
ba044610b7
Add route interactions: click-to-split, drag-to-reshape, colored route rendering
- Enrich BRouter response with per-point 3D coordinates and segment boundary
  tracking (EnrichedRoute interface)
- ColoredRoute component: plain, elevation gradient (green→yellow→red), and
  surface color modes with invisible wide polyline for click targeting
- Click-to-split: click on route polyline inserts waypoint at nearest point,
  mapped to correct segment via boundary indices
- MidpointHandles: draggable CircleMarkers at route segment midpoints for
  reshaping, hidden below zoom 12, opaque on hover
- Color mode toggle (select) synced via Yjs routeData
- i18n keys for color mode labels (en + de)
- Unit tests for segment boundary tracking (13 tests)
- E2E tests for enriched route response and color mode toggle

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:36:42 +01:00
Ullrich Schäfer
5aef99b5ce
Merge pull request #86 from trails-cool/planner-route-interactions
Add route interactions: split, reshape, and colored rendering
2026-03-26 22:18:27 +01:00
Ullrich Schäfer
b080a15fb1
Add route interactions: click-to-split, drag-to-reshape, colored route rendering
- Enrich BRouter response with per-point 3D coordinates and segment boundary
  tracking (EnrichedRoute interface)
- ColoredRoute component: plain, elevation gradient (green→yellow→red), and
  surface color modes with invisible wide polyline for click targeting
- Click-to-split: click on route polyline inserts waypoint at nearest point,
  mapped to correct segment via boundary indices
- MidpointHandles: draggable CircleMarkers at route segment midpoints for
  reshaping, hidden below zoom 12, opaque on hover
- Color mode toggle (select) synced via Yjs routeData
- i18n keys for color mode labels (en + de)
- Unit tests for segment boundary tracking (13 tests)
- E2E tests for enriched route response and color mode toggle

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 21:15:55 +01:00
Ullrich Schäfer
14a39dc7cc
Merge pull request #84 from trails-cool/merge-queue
Replace automerge action with GitHub merge queue
2026-03-26 18:28:23 +00:00
Ullrich Schäfer
ed4757382a
Use forked gitleaks-action with merge_group support
The upstream gitleaks/gitleaks-action@v2 errors on merge_group events.
Use our fork which includes https://github.com/gitleaks/gitleaks-action/pull/186.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 19:26:18 +01:00
Ullrich Schäfer
cce2ddea5c
Fix: gh pr merge --auto requires --merge flag
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 19:09:08 +01:00
Ullrich Schäfer
aed99ba423
Merge pull request #85 from trails-cool/fix-merge-queue-docs
Fix CLAUDE.md merge queue docs to use --auto
2026-03-26 19:02:46 +01:00
Ullrich Schäfer
eccc274652
Fix CLAUDE.md: use gh pr merge --auto, not --merge-queue
The --merge-queue flag doesn't exist in gh CLI. The correct flag is
--auto, which routes through the merge queue when it's enabled.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 19:02:27 +01:00
Ullrich Schäfer
746b09276c
Replace automerge action with GitHub native merge queue
The automerge GitHub Action (pascalgn/automerge-action) is removed in
favor of GitHub's built-in merge queue, which is now enabled on the
main branch ruleset.

- Delete automerge.yml workflow
- Add merge_group trigger to CI so it runs on queue candidates
- Remove workflow_run trigger for Automerge from CD
- Update CLAUDE.md to document gh pr merge --merge-queue

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:40:11 +01:00