From ae23d31a5fd0648a43f3d3c29f92485195952c9f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ullrich=20Sch=C3=A4fer?= Date: Sat, 23 May 2026 21:10:08 +0200 Subject: [PATCH] Wire INTEGRATION_SECRET into docker-compose and CI; archive komoot-import MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add INTEGRATION_SECRET to journal service in docker-compose.yml with :? guard so a missing value fails loudly at compose-up time - Add INTEGRATION_SECRET to E2E test step in ci.yml via GitHub secret (unit tests already set their own value in the test file) - Archive openspec/changes/komoot-import → archive/2026-05-23-komoot-import - Sync delta specs: new openspec/specs/komoot-import/spec.md, updated openspec/specs/route-management/spec.md Co-Authored-By: Claude Sonnet 4.6 --- .github/workflows/ci.yml | 1 + infrastructure/docker-compose.yml | 1 + .../2026-05-23-komoot-import/.openspec.yaml | 2 + .../2026-05-23-komoot-import/design.md | 100 ++++++++++++++++++ .../2026-05-23-komoot-import/proposal.md | 42 ++++++++ .../specs/komoot-import/spec.md | 91 ++++++++++++++++ .../specs/route-management/spec.md | 12 +++ .../archive/2026-05-23-komoot-import/tasks.md | 42 ++++++++ openspec/specs/komoot-import/spec.md | 95 +++++++++++++++++ openspec/specs/route-management/spec.md | 11 ++ 10 files changed, 397 insertions(+) create mode 100644 openspec/changes/archive/2026-05-23-komoot-import/.openspec.yaml create mode 100644 openspec/changes/archive/2026-05-23-komoot-import/design.md create mode 100644 openspec/changes/archive/2026-05-23-komoot-import/proposal.md create mode 100644 openspec/changes/archive/2026-05-23-komoot-import/specs/komoot-import/spec.md create mode 100644 openspec/changes/archive/2026-05-23-komoot-import/specs/route-management/spec.md create mode 100644 openspec/changes/archive/2026-05-23-komoot-import/tasks.md create mode 100644 openspec/specs/komoot-import/spec.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0494470..1237944 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -256,6 +256,7 @@ jobs: env: BROUTER_URL: http://localhost:17777 E2E: "true" + INTEGRATION_SECRET: ${{ secrets.INTEGRATION_SECRET }} - name: Playwright job summary if: ${{ !cancelled() }} diff --git a/infrastructure/docker-compose.yml b/infrastructure/docker-compose.yml index 4614663..01c1c04 100644 --- a/infrastructure/docker-compose.yml +++ b/infrastructure/docker-compose.yml @@ -48,6 +48,7 @@ services: WAHOO_CLIENT_ID: ${WAHOO_CLIENT_ID:-} WAHOO_CLIENT_SECRET: ${WAHOO_CLIENT_SECRET:-} WAHOO_WEBHOOK_TOKEN: ${WAHOO_WEBHOOK_TOKEN:-} + INTEGRATION_SECRET: ${INTEGRATION_SECRET:?INTEGRATION_SECRET must be set in SOPS secrets.app.env} # Demo-activity-bot. Disabled by default everywhere; flip to "true" # only in prod. When unset, DEMO_BOT_PERSONA uses the built-in # Bruno/Berlin persona. See docs/demo-persona.md for the schema. diff --git a/openspec/changes/archive/2026-05-23-komoot-import/.openspec.yaml b/openspec/changes/archive/2026-05-23-komoot-import/.openspec.yaml new file mode 100644 index 0000000..40c5540 --- /dev/null +++ b/openspec/changes/archive/2026-05-23-komoot-import/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-03-25 diff --git a/openspec/changes/archive/2026-05-23-komoot-import/design.md b/openspec/changes/archive/2026-05-23-komoot-import/design.md new file mode 100644 index 0000000..c9c4e06 --- /dev/null +++ b/openspec/changes/archive/2026-05-23-komoot-import/design.md @@ -0,0 +1,100 @@ +## Context + +trails.cool Journal supports manual route creation and GPX import. Users coming +from Komoot have hundreds of tours they'd lose by switching. The Komoot public +API (`api.komoot.de/v007`) exposes public tours and user profiles without +authentication. Profile fields (`content_text`, `content_link`) are readable +unauthenticated after a short cache delay (~minutes). + +> **Note (added 2026-05-08, post `deepen-connected-services`)**: +> Earlier drafts of this design proposed a separate `journal.integrations` +> table for Komoot credentials. That has been **superseded** by the +> connected-services architecture introduced in +> `openspec/changes/deepen-connected-services/`. When this change is +> revisited, Komoot must implement: +> +> - A row in `journal.connected_services` with `credential_kind = 'web-login'` +> (authenticated mode) or `credential_kind = 'public'` (public mode). +> - A `KomootImporter` in +> `apps/journal/app/lib/connected-services/providers/komoot/importer.ts` +> that branches on credential kind. +> - A manifest at `providers/komoot/manifest.ts` registered via +> `providers/index.ts`. +> +> Don't add a `journal.integrations` table. The user-facing "Connected +> Services" list at `/settings/connections` should show Komoot alongside Wahoo. + +## Goals / Non-Goals + +**Goals:** +- Public import: verify Komoot profile ownership via bio field, import public tours, store no passwords +- Authenticated import: connect via email + password, import all tours including private +- Import all tours (paginated) as activities + routes +- Track import progress with batch status +- Deduplicate on re-import (same tour never imported twice) +- Fetch GPX geometry per tour (not just metadata) +- Cross-link: store Komoot username on the connection so the Journal profile can show "also on Komoot" + +**Non-Goals:** +- OAuth flow +- Real-time sync or webhook-based updates +- Other providers (future iteration) + +## Import Modes + +### Public mode + +No credentials stored. Ownership is verified by checking that the user's +trails.cool profile URL appears in their Komoot `content_text` (bio/"Über dich") +field, which is readable via the unauthenticated public API. + +Verification flow: +1. User enters their Komoot profile URL (e.g. `komoot.com/user/27595800585`) +2. trails.cool shows: *"Add your trails.cool profile link (`https://trails.cool/users/ullrich`) to your Komoot bio ('Über dich'), then click Verify"* +3. trails.cool fetches `api.komoot.de/v007/users/{id}/` and checks `content_text` contains their trails.cool profile URL +4. On success: store Komoot username in `connected_services` with `credential_kind = 'public'`, no password +5. Fetch `api.komoot.de/v007/users/{id}/tours/?status=public` (paginated), import + +### Authenticated mode + +User provides email + password. Credentials validated and stored AES-256-GCM +encrypted. All tours (public and private) imported. + +## Decisions + +### D1: Two connection modes in `connected_services` + +A `mode` column (`'public' | 'authenticated'`) on the connection row controls +which import path runs. Public mode rows have no `encryptedCredentials`. + +### D2: Bio field for ownership verification + +`api.komoot.de/v007/users/{id}/` returns `content_text` (bio) unauthenticated. +Checking for the user's own trails.cool profile URL proves they control the +Komoot account without any credential exchange. The profile link stays in the +bio permanently, providing cross-platform discovery. + +### D3: Import batches for progress tracking + +Each import creates a batch row tracking: status, total found, imported count, +duplicate count, error message. UI polls for live progress. + +### D4: Deduplication via dedupe key + +Activities get a `dedupeKey` of `komoot:{tourId}`. Unique constraint on +`(ownerId, dedupeKey)` prevents duplicates on re-import. + +### D5: AES-256-GCM for credential encryption (authenticated mode only) + +`INTEGRATION_SECRET` env var → scrypt-derived key. Decrypt on use, never log. + +## Risks / Trade-offs + +- **Komoot API is undocumented** → Could change without notice. All calls + wrapped in error handling; connection marked as needing reauth on 401. +- **Public mode: only public tours importable** → Expected and documented. + Users who want private tours use authenticated mode. +- **Bio field cache delay** → Verification shows a clear "allow a few minutes + for changes to propagate" message and a retry button. +- **Authenticated mode: storing third-party passwords** → AES-256-GCM, + separate `INTEGRATION_SECRET`, documented in privacy manifest. diff --git a/openspec/changes/archive/2026-05-23-komoot-import/proposal.md b/openspec/changes/archive/2026-05-23-komoot-import/proposal.md new file mode 100644 index 0000000..de25d91 --- /dev/null +++ b/openspec/changes/archive/2026-05-23-komoot-import/proposal.md @@ -0,0 +1,42 @@ +## Why + +Users switching to trails.cool have years of tours on Komoot. Without import, +they start with an empty Journal — no history, no motivation to switch. A +Komoot import lets users bring their existing tours and start using +trails.cool immediately. + +## What Changes + +- **Public import** (no credentials): user proves ownership of their Komoot + profile by placing their trails.cool profile URL in their Komoot bio field. + trails.cool verifies via the public Komoot API, then imports all public tours. +- **Authenticated import** (email + password): imports all tours including + private ones. Credentials stored encrypted. +- Both modes share the same batch import engine, progress tracking, and + deduplication logic. + +## Capabilities + +### New Capabilities + +- `komoot-import`: Two-mode Komoot import — public (bio verification, no + credential storage) and authenticated (email + password, all tours including + private). Both track batch progress and deduplicate on re-import. + +### Modified Capabilities + +- `route-management`: Routes can now be created via import (not just manual + creation), with an external source tracking field. + +## Impact + +- **Database**: New tables for integration connections and import batches in + `journal` schema. Connections can be credential-free (public mode). +- **Files**: New routes (`/integrations`, `/api/integrations/*`), new server + utilities for Komoot API, new DB schema tables +- **Dependencies**: No new packages — uses fetch for Komoot API, existing + Drizzle for DB +- **Privacy**: Only authenticated mode stores credentials (encrypted). Public + mode stores only the Komoot username. Both documented in privacy manifest. +- **External**: Komoot public API (`api.komoot.de`) — unauthenticated for + public tours and profile lookup; basic auth for authenticated mode diff --git a/openspec/changes/archive/2026-05-23-komoot-import/specs/komoot-import/spec.md b/openspec/changes/archive/2026-05-23-komoot-import/specs/komoot-import/spec.md new file mode 100644 index 0000000..ab0570e --- /dev/null +++ b/openspec/changes/archive/2026-05-23-komoot-import/specs/komoot-import/spec.md @@ -0,0 +1,91 @@ +## ADDED Requirements + +### Requirement: Connect Komoot account — public mode +Users SHALL be able to connect their Komoot account without providing credentials +by verifying ownership via their Komoot bio field. + +#### Scenario: Initiate public connection +- **WHEN** a user enters their Komoot profile URL on the integrations page +- **THEN** the system displays their trails.cool profile URL and instructs them to add it to their Komoot bio ("Über dich" field) + +#### Scenario: Verify public ownership +- **WHEN** the user clicks Verify after updating their Komoot bio +- **THEN** the system fetches the public Komoot profile, checks that `content_text` contains the user's trails.cool profile URL, and on success marks the connection as active with mode "public" + +#### Scenario: Bio not yet updated +- **WHEN** verification is attempted but the trails.cool URL is not found in the Komoot bio +- **THEN** the system shows an error explaining that changes may take a few minutes to propagate, with a retry button + +#### Scenario: Disconnect public account +- **WHEN** a user disconnects their public Komoot integration +- **THEN** the stored Komoot username is removed and no further imports can occur + +### Requirement: Connect Komoot account — authenticated mode +Users SHALL be able to connect their Komoot account by providing email and +password to access all tours including private ones. + +#### Scenario: Successful authenticated connection +- **WHEN** a user enters valid Komoot email and password +- **THEN** the system validates credentials via the Komoot API, stores them encrypted, and shows the connection as active with mode "authenticated" + +#### Scenario: Invalid credentials +- **WHEN** a user enters incorrect Komoot credentials +- **THEN** the system shows an error and does not store credentials + +#### Scenario: Disconnect authenticated account +- **WHEN** a user disconnects their authenticated Komoot integration +- **THEN** stored credentials are deleted and no further imports can occur + +### Requirement: Import Komoot tours +Users SHALL be able to import their Komoot tours as Journal activities with +linked routes. Public mode imports public tours only; authenticated mode imports +all tours. + +#### Scenario: Import public tours +- **WHEN** a user with a public-mode connection triggers import +- **THEN** the system fetches all public tours from the Komoot API without credentials, creates an activity and route for each, and shows progress + +#### Scenario: Import all tours (authenticated) +- **WHEN** a user with an authenticated-mode connection triggers import +- **THEN** the system fetches all tours (public and private, paginated) using stored credentials, creates an activity and route for each, and shows progress + +#### Scenario: Tour with GPX geometry +- **WHEN** a Komoot tour is imported +- **THEN** the system fetches the tour's GPX and creates a route with the full track data + +#### Scenario: Import progress +- **WHEN** an import is running +- **THEN** the UI shows: total tours found, imported so far, duplicates skipped, and current status + +### Requirement: Deduplication +The system SHALL NOT create duplicate activities when re-importing tours that +were previously imported. + +#### Scenario: Re-import skips existing tours +- **WHEN** a user runs import and a tour was already imported previously +- **THEN** the tour is skipped and counted as a duplicate in the import summary + +### Requirement: Import batch tracking +Each import run SHALL be tracked as a batch with status and statistics. + +#### Scenario: Batch lifecycle +- **WHEN** an import starts +- **THEN** a batch record is created with status "running", updated to "completed" or "failed" when done + +#### Scenario: Batch statistics +- **WHEN** an import completes +- **THEN** the batch shows: totalFound, importedCount, duplicateCount, and duration + +### Requirement: Credential encryption (authenticated mode) +Komoot credentials SHALL be encrypted at rest using AES-256-GCM. + +#### Scenario: Credentials stored securely +- **WHEN** a user connects via authenticated mode +- **THEN** the password is encrypted before storage and only decrypted when making API calls + +### Requirement: Privacy disclosure +The Komoot integration SHALL be documented in the privacy manifest. + +#### Scenario: Privacy manifest updated +- **WHEN** Komoot import is available +- **THEN** the /privacy page documents: that public mode stores only the Komoot username, that authenticated mode stores an encrypted password, and what tour data is imported diff --git a/openspec/changes/archive/2026-05-23-komoot-import/specs/route-management/spec.md b/openspec/changes/archive/2026-05-23-komoot-import/specs/route-management/spec.md new file mode 100644 index 0000000..b22d5f0 --- /dev/null +++ b/openspec/changes/archive/2026-05-23-komoot-import/specs/route-management/spec.md @@ -0,0 +1,12 @@ +## MODIFIED Requirements + +### Requirement: Route creation +The system SHALL support creating routes via import from external services, in addition to manual creation and GPX upload. + +#### Scenario: Route created from import +- **WHEN** a Komoot tour is imported +- **THEN** a route is created with name, distance, elevation, GPX geometry, and a source field indicating "komoot" + +#### Scenario: Route links to activity +- **WHEN** a tour is imported as an activity +- **THEN** the activity is linked to the created route diff --git a/openspec/changes/archive/2026-05-23-komoot-import/tasks.md b/openspec/changes/archive/2026-05-23-komoot-import/tasks.md new file mode 100644 index 0000000..045459e --- /dev/null +++ b/openspec/changes/archive/2026-05-23-komoot-import/tasks.md @@ -0,0 +1,42 @@ +## 1. Komoot API Client + +- [x] 1.1 Add `fetchPublicProfile(komootUserId)` to `komoot.server.ts` — unauthenticated GET of `api.komoot.de/v007/users/{id}/`, returns `{ displayName, contentText, contentLink }` +- [x] 1.2 Add `fetchPublicTours(komootUserId, page)` to `komoot.server.ts` — unauthenticated GET with `?status=public&limit=50` +- [x] 1.3 Add `fetchPublicTourGpx(tourId)` — unauthenticated GPX fetch for public tours +- [x] 1.4 Update `KomootCredentials` type to be a discriminated union: `{ mode: 'public'; komootUserId: string } | { mode: 'authenticated'; email: string; encryptedPassword: string; komootUserId: string }` +- [x] 1.5 Update `fetchTours` / `fetchTourGpx` to branch on credential mode (use auth header only in authenticated mode) +- [x] 1.6 Add unit tests for `fetchPublicProfile` response parsing and bio verification logic + +## 2. Verification Logic + +- [x] 2.1 Add `verifyKomootOwnership(komootUserId, trailsProfileUrl)` in `komoot.server.ts` — fetches public profile, checks `content_text` contains the trails.cool URL (case-insensitive, trimmed) +- [x] 2.2 Write unit tests for verification: match, no match, null bio, trailing slash variations + +## 3. Database Schema + +- [x] 3.1 Add `mode` column (`'public' | 'authenticated'`) to `journal.sync_connections` (or connected_services table per the connected-services architecture) +- [x] 3.2 Make `encryptedCredentials` nullable (public mode has none) +- [x] 3.3 Run `pnpm db:push` and verify schema locally + +## 4. API Routes + +- [x] 4.1 Create `POST /api/sync/komoot/verify` — accepts `{ komootProfileUrl }`, calls `verifyKomootOwnership`, on success stores public connection (no credentials) +- [x] 4.2 Create `POST /api/sync/komoot/connect` — authenticated mode; accepts `{ email, password }` +- [x] 4.3 `sync.import.komoot.tsx` — branches on connection mode to use public or authenticated fetch path +- [x] 4.4 Register new routes in `apps/journal/app/routes.ts` + +## 5. Import Logic + +- [x] 5.1 `importer.ts` accepts the discriminated union credential type and routes to public or authenticated fetch functions accordingly +- [x] 5.2 Credentials stored as discriminated union in connected_services; mode determined from stored credentials + +## 6. UI + +- [x] 6.1 Add public import section to `/settings/connections/komoot` above the authenticated form: input for Komoot profile URL, instructions showing the user's trails.cool profile URL to copy, Verify button +- [x] 6.2 Show verification state: pending instructions → verifying spinner → success (connected, public) or error with retry +- [x] 6.3 Show mode badge ("Public tours only" vs "All tours") on the connected state UI +- [x] 6.4 Add i18n keys for all new public-mode strings (en + de) + +## 7. Privacy + +- [x] 7.1 Update `/privacy` page to document both modes: public (stores Komoot username only) and authenticated (stores encrypted password) diff --git a/openspec/specs/komoot-import/spec.md b/openspec/specs/komoot-import/spec.md new file mode 100644 index 0000000..3cde115 --- /dev/null +++ b/openspec/specs/komoot-import/spec.md @@ -0,0 +1,95 @@ +## Purpose + +Komoot integration for the Journal app: connect a Komoot account (public bio-verification or authenticated with credentials) and import tours as Journal activities with linked routes. + +## Requirements + +### Requirement: Connect Komoot account — public mode +Users SHALL be able to connect their Komoot account without providing credentials +by verifying ownership via their Komoot bio field. + +#### Scenario: Initiate public connection +- **WHEN** a user enters their Komoot profile URL on the integrations page +- **THEN** the system displays their trails.cool profile URL and instructs them to add it to their Komoot bio ("Über dich" field) + +#### Scenario: Verify public ownership +- **WHEN** the user clicks Verify after updating their Komoot bio +- **THEN** the system fetches the public Komoot profile, checks that `content_text` contains the user's trails.cool profile URL, and on success marks the connection as active with mode "public" + +#### Scenario: Bio not yet updated +- **WHEN** verification is attempted but the trails.cool URL is not found in the Komoot bio +- **THEN** the system shows an error explaining that changes may take a few minutes to propagate, with a retry button + +#### Scenario: Disconnect public account +- **WHEN** a user disconnects their public Komoot integration +- **THEN** the stored Komoot username is removed and no further imports can occur + +### Requirement: Connect Komoot account — authenticated mode +Users SHALL be able to connect their Komoot account by providing email and +password to access all tours including private ones. + +#### Scenario: Successful authenticated connection +- **WHEN** a user enters valid Komoot email and password +- **THEN** the system validates credentials via the Komoot API, stores them encrypted, and shows the connection as active with mode "authenticated" + +#### Scenario: Invalid credentials +- **WHEN** a user enters incorrect Komoot credentials +- **THEN** the system shows an error and does not store credentials + +#### Scenario: Disconnect authenticated account +- **WHEN** a user disconnects their authenticated Komoot integration +- **THEN** stored credentials are deleted and no further imports can occur + +### Requirement: Import Komoot tours +Users SHALL be able to import their Komoot tours as Journal activities with +linked routes. Public mode imports public tours only; authenticated mode imports +all tours. + +#### Scenario: Import public tours +- **WHEN** a user with a public-mode connection triggers import +- **THEN** the system fetches all public tours from the Komoot API without credentials, creates an activity and route for each, and shows progress + +#### Scenario: Import all tours (authenticated) +- **WHEN** a user with an authenticated-mode connection triggers import +- **THEN** the system fetches all tours (public and private, paginated) using stored credentials, creates an activity and route for each, and shows progress + +#### Scenario: Tour with GPX geometry +- **WHEN** a Komoot tour is imported +- **THEN** the system fetches the tour's GPX and creates a route with the full track data + +#### Scenario: Import progress +- **WHEN** an import is running +- **THEN** the UI shows: total tours found, imported so far, duplicates skipped, and current status + +### Requirement: Deduplication +The system SHALL NOT create duplicate activities when re-importing tours that +were previously imported. + +#### Scenario: Re-import skips existing tours +- **WHEN** a user runs import and a tour was already imported previously +- **THEN** the tour is skipped and counted as a duplicate in the import summary + +### Requirement: Import batch tracking +Each import run SHALL be tracked as a batch with status and statistics. + +#### Scenario: Batch lifecycle +- **WHEN** an import starts +- **THEN** a batch record is created with status "running", updated to "completed" or "failed" when done + +#### Scenario: Batch statistics +- **WHEN** an import completes +- **THEN** the batch shows: totalFound, importedCount, duplicateCount, and duration + +### Requirement: Credential encryption (authenticated mode) +Komoot credentials SHALL be encrypted at rest using AES-256-GCM. + +#### Scenario: Credentials stored securely +- **WHEN** a user connects via authenticated mode +- **THEN** the password is encrypted before storage and only decrypted when making API calls + +### Requirement: Privacy disclosure +The Komoot integration SHALL be documented in the privacy manifest. + +#### Scenario: Privacy manifest updated +- **WHEN** Komoot import is available +- **THEN** the /privacy page documents: that public mode stores only the Komoot username, that authenticated mode stores an encrypted password, and what tour data is imported diff --git a/openspec/specs/route-management/spec.md b/openspec/specs/route-management/spec.md index e9301de..6a60ea9 100644 --- a/openspec/specs/route-management/spec.md +++ b/openspec/specs/route-management/spec.md @@ -130,6 +130,17 @@ Any listing that exposes routes beyond the owner's own dashboard SHALL only incl - **WHEN** a logged-in user views their own routes list at `/routes` - **THEN** the list includes all of their own routes regardless of visibility +### Requirement: Route creation via import +The system SHALL support creating routes via import from external services, in addition to manual creation and GPX upload. + +#### Scenario: Route created from import +- **WHEN** a Komoot tour is imported +- **THEN** a route is created with name, distance, elevation, GPX geometry, and a source field indicating "komoot" + +#### Scenario: Route links to activity +- **WHEN** a tour is imported as an activity +- **THEN** the activity is linked to the created route + ### Requirement: Synthetic route flag The Journal SHALL persist a `synthetic` boolean on every route so automated / demo content can be distinguished from user-created content.