diff --git a/openspec/changes/komoot-import/design.md b/openspec/changes/komoot-import/design.md index 7db6b9f..c9c4e06 100644 --- a/openspec/changes/komoot-import/design.md +++ b/openspec/changes/komoot-import/design.md @@ -1,9 +1,10 @@ ## Context trails.cool Journal supports manual route creation and GPX import. Users coming -from Komoot have hundreds of tours they'd lose by switching. The old trails -project had a working Komoot integration using basic auth against Komoot's -undocumented API (`api.komoot.de`). +from Komoot have hundreds of tours they'd lose by switching. The Komoot public +API (`api.komoot.de/v007`) exposes public tours and user profiles without +authentication. Profile fields (`content_text`, `content_link`) are readable +unauthenticated after a short cache delay (~minutes). > **Note (added 2026-05-08, post `deepen-connected-services`)**: > Earlier drafts of this design proposed a separate `journal.integrations` @@ -13,87 +14,87 @@ undocumented API (`api.komoot.de`). > revisited, Komoot must implement: > > - A row in `journal.connected_services` with `credential_kind = 'web-login'` -> and a `credentials` JSONB blob carrying `{ email, encrypted_password, -> session_jar }`. -> - A `web-login` `CredentialAdapter` at -> `apps/journal/app/lib/connected-services/credential-adapters/web-login.ts` -> implementing `relogin(creds) → creds | InvalidCredentials`. Web-login -> breakage (form changes, captcha, password rotation) surfaces at the -> import layer, not the credential layer (see ADR-0001 / CONTEXT.md). +> (authenticated mode) or `credential_kind = 'public'` (public mode). > - A `KomootImporter` in > `apps/journal/app/lib/connected-services/providers/komoot/importer.ts` -> that goes through `ctx.withFreshCredentials` like the Wahoo importer. -> Komoot does not have webhooks or push, so its manifest declares only -> the `Importer` capability — no `routePusher`, no `webhookReceiver`. +> that branches on credential kind. > - A manifest at `providers/komoot/manifest.ts` registered via > `providers/index.ts`. > > Don't add a `journal.integrations` table. The user-facing "Connected -> Services" list at `/settings/connections` should show Komoot alongside -> Wahoo, which only works if both share `connected_services`. +> Services" list at `/settings/connections` should show Komoot alongside Wahoo. ## Goals / Non-Goals **Goals:** -- Connect Komoot account via email + password +- Public import: verify Komoot profile ownership via bio field, import public tours, store no passwords +- Authenticated import: connect via email + password, import all tours including private - Import all tours (paginated) as activities + routes - Track import progress with batch status - Deduplicate on re-import (same tour never imported twice) - Fetch GPX geometry per tour (not just metadata) +- Cross-link: store Komoot username on the connection so the Journal profile can show "also on Komoot" **Non-Goals:** -- OAuth flow (Komoot has no public OAuth — basic auth only) +- OAuth flow - Real-time sync or webhook-based updates -- Strava/other providers (future iteration, but design the schema generically) -- Background job queue like Inngest (keep it simple — synchronous import with progress) +- Other providers (future iteration) + +## Import Modes + +### Public mode + +No credentials stored. Ownership is verified by checking that the user's +trails.cool profile URL appears in their Komoot `content_text` (bio/"Über dich") +field, which is readable via the unauthenticated public API. + +Verification flow: +1. User enters their Komoot profile URL (e.g. `komoot.com/user/27595800585`) +2. trails.cool shows: *"Add your trails.cool profile link (`https://trails.cool/users/ullrich`) to your Komoot bio ('Über dich'), then click Verify"* +3. trails.cool fetches `api.komoot.de/v007/users/{id}/` and checks `content_text` contains their trails.cool profile URL +4. On success: store Komoot username in `connected_services` with `credential_kind = 'public'`, no password +5. Fetch `api.komoot.de/v007/users/{id}/tours/?status=public` (paginated), import + +### Authenticated mode + +User provides email + password. Credentials validated and stored AES-256-GCM +encrypted. All tours (public and private) imported. ## Decisions -### D1: Generic integrations table with provider column +### D1: Two connection modes in `connected_services` -Store connections in a `journal.integrations` table with a `provider` enum -(`komoot`, and later `strava` etc). Credentials encrypted at rest. This avoids -a separate table per provider. +A `mode` column (`'public' | 'authenticated'`) on the connection row controls +which import path runs. Public mode rows have no `encryptedCredentials`. -### D2: Import batches for progress tracking +### D2: Bio field for ownership verification -Each import creates an `import_batches` row tracking: status (running, -completed, failed), total found, imported count, duplicate count, error message. -The UI polls this for progress. +`api.komoot.de/v007/users/{id}/` returns `content_text` (bio) unauthenticated. +Checking for the user's own trails.cool profile URL proves they control the +Komoot account without any credential exchange. The profile link stays in the +bio permanently, providing cross-platform discovery. -### D3: Deduplication via composite key +### D3: Import batches for progress tracking -Activities get a `dedupeKey` column. For Komoot: `komoot:{tourId}`. Combined -with `ownerId`, a unique constraint prevents duplicates. Insert uses -`onConflictDoNothing`. +Each import creates a batch row tracking: status, total found, imported count, +duplicate count, error message. UI polls for live progress. -### D4: Synchronous import with streaming progress +### D4: Deduplication via dedupe key -No background job queue. The import runs in an API route action that: -1. Fetches all tour pages from Komoot -2. For each tour, fetches GPX and creates activity + route -3. Updates the batch row with progress -4. Client polls `/api/integrations/komoot/import-status` for live updates +Activities get a `dedupeKey` of `komoot:{tourId}`. Unique constraint on +`(ownerId, dedupeKey)` prevents duplicates on re-import. -This keeps the architecture simple. If imports are too slow (>100 tours), we -can add a background worker later. +### D5: AES-256-GCM for credential encryption (authenticated mode only) -### D5: Encrypt credentials with AES-256-GCM - -Use Node's `crypto.createCipheriv` with a server-side key derived from -`INTEGRATION_SECRET` env var. Decrypt on use, never log plaintext. - -**Alternative considered**: Store only the API token (not email+password). -Rejected because the token may expire and re-auth requires the original -credentials. +`INTEGRATION_SECRET` env var → scrypt-derived key. Decrypt on use, never log. ## Risks / Trade-offs -- **Komoot API is undocumented** → Could break without notice. Mitigation: wrap - all API calls in error handling, mark integration as "needs reauth" on 401. -- **Storing user passwords for third-party service** → Security risk. - Mitigation: AES-256-GCM encryption, separate `INTEGRATION_SECRET`, document - in privacy manifest. -- **Synchronous import may timeout for large accounts** → Mitigation: paginate - and commit per-page. If a page fails, the batch is marked partial and can be - resumed. +- **Komoot API is undocumented** → Could change without notice. All calls + wrapped in error handling; connection marked as needing reauth on 401. +- **Public mode: only public tours importable** → Expected and documented. + Users who want private tours use authenticated mode. +- **Bio field cache delay** → Verification shows a clear "allow a few minutes + for changes to propagate" message and a retry button. +- **Authenticated mode: storing third-party passwords** → AES-256-GCM, + separate `INTEGRATION_SECRET`, documented in privacy manifest. diff --git a/openspec/changes/komoot-import/proposal.md b/openspec/changes/komoot-import/proposal.md index 7a50275..de25d91 100644 --- a/openspec/changes/komoot-import/proposal.md +++ b/openspec/changes/komoot-import/proposal.md @@ -2,31 +2,41 @@ Users switching to trails.cool have years of tours on Komoot. Without import, they start with an empty Journal — no history, no motivation to switch. A -one-click Komoot import lets users bring their existing tours and start using +Komoot import lets users bring their existing tours and start using trails.cool immediately. ## What Changes -- Add Komoot account connection flow (email + password → API credentials) -- Import Komoot tours as Journal activities with route data -- Batch import with progress tracking, deduplication, and error handling -- Integration settings page to manage connected accounts and trigger imports -- Fetch full tour GPX (not just metadata) so imported routes have geometry +- **Public import** (no credentials): user proves ownership of their Komoot + profile by placing their trails.cool profile URL in their Komoot bio field. + trails.cool verifies via the public Komoot API, then imports all public tours. +- **Authenticated import** (email + password): imports all tours including + private ones. Credentials stored encrypted. +- Both modes share the same batch import engine, progress tracking, and + deduplication logic. ## Capabilities ### New Capabilities -- `komoot-import`: Connect a Komoot account, import tours as activities with routes, track import progress, deduplicate on re-import +- `komoot-import`: Two-mode Komoot import — public (bio verification, no + credential storage) and authenticated (email + password, all tours including + private). Both track batch progress and deduplicate on re-import. ### Modified Capabilities -- `route-management`: Routes can now be created via import (not just manual creation), with an external source tracking field +- `route-management`: Routes can now be created via import (not just manual + creation), with an external source tracking field. ## Impact -- **Database**: New tables for integration connections and import batches in `journal` schema -- **Files**: New routes (`/integrations`, `/api/integrations/*`), new server utilities for Komoot API, new DB schema tables -- **Dependencies**: No new packages needed — uses fetch for Komoot API, existing Drizzle for DB -- **Privacy**: Komoot credentials stored encrypted. Must be documented in privacy manifest. -- **External**: Komoot API (api.komoot.de) — basic auth, no official developer program +- **Database**: New tables for integration connections and import batches in + `journal` schema. Connections can be credential-free (public mode). +- **Files**: New routes (`/integrations`, `/api/integrations/*`), new server + utilities for Komoot API, new DB schema tables +- **Dependencies**: No new packages — uses fetch for Komoot API, existing + Drizzle for DB +- **Privacy**: Only authenticated mode stores credentials (encrypted). Public + mode stores only the Komoot username. Both documented in privacy manifest. +- **External**: Komoot public API (`api.komoot.de`) — unauthenticated for + public tours and profile lookup; basic auth for authenticated mode diff --git a/openspec/changes/komoot-import/specs/komoot-import/spec.md b/openspec/changes/komoot-import/specs/komoot-import/spec.md index 9f68410..ab0570e 100644 --- a/openspec/changes/komoot-import/specs/komoot-import/spec.md +++ b/openspec/changes/komoot-import/specs/komoot-import/spec.md @@ -1,40 +1,68 @@ ## ADDED Requirements -### Requirement: Connect Komoot account -Users SHALL be able to connect their Komoot account by providing email and password. Credentials are validated against the Komoot API and stored encrypted. +### Requirement: Connect Komoot account — public mode +Users SHALL be able to connect their Komoot account without providing credentials +by verifying ownership via their Komoot bio field. -#### Scenario: Successful connection -- **WHEN** user enters valid Komoot email and password on the integrations page -- **THEN** the system validates credentials via `api.komoot.de`, stores them encrypted, and shows the connection as active +#### Scenario: Initiate public connection +- **WHEN** a user enters their Komoot profile URL on the integrations page +- **THEN** the system displays their trails.cool profile URL and instructs them to add it to their Komoot bio ("Über dich" field) + +#### Scenario: Verify public ownership +- **WHEN** the user clicks Verify after updating their Komoot bio +- **THEN** the system fetches the public Komoot profile, checks that `content_text` contains the user's trails.cool profile URL, and on success marks the connection as active with mode "public" + +#### Scenario: Bio not yet updated +- **WHEN** verification is attempted but the trails.cool URL is not found in the Komoot bio +- **THEN** the system shows an error explaining that changes may take a few minutes to propagate, with a retry button + +#### Scenario: Disconnect public account +- **WHEN** a user disconnects their public Komoot integration +- **THEN** the stored Komoot username is removed and no further imports can occur + +### Requirement: Connect Komoot account — authenticated mode +Users SHALL be able to connect their Komoot account by providing email and +password to access all tours including private ones. + +#### Scenario: Successful authenticated connection +- **WHEN** a user enters valid Komoot email and password +- **THEN** the system validates credentials via the Komoot API, stores them encrypted, and shows the connection as active with mode "authenticated" #### Scenario: Invalid credentials -- **WHEN** user enters incorrect Komoot credentials -- **THEN** the system shows an error message and does not store credentials +- **WHEN** a user enters incorrect Komoot credentials +- **THEN** the system shows an error and does not store credentials -#### Scenario: Disconnect account -- **WHEN** user disconnects their Komoot integration +#### Scenario: Disconnect authenticated account +- **WHEN** a user disconnects their authenticated Komoot integration - **THEN** stored credentials are deleted and no further imports can occur ### Requirement: Import Komoot tours -Users SHALL be able to import all their Komoot tours as Journal activities with linked routes. +Users SHALL be able to import their Komoot tours as Journal activities with +linked routes. Public mode imports public tours only; authenticated mode imports +all tours. -#### Scenario: Import all tours -- **WHEN** user clicks "Import" on the integrations page -- **THEN** the system fetches all tours (paginated), creates an activity and route for each, and shows progress +#### Scenario: Import public tours +- **WHEN** a user with a public-mode connection triggers import +- **THEN** the system fetches all public tours from the Komoot API without credentials, creates an activity and route for each, and shows progress + +#### Scenario: Import all tours (authenticated) +- **WHEN** a user with an authenticated-mode connection triggers import +- **THEN** the system fetches all tours (public and private, paginated) using stored credentials, creates an activity and route for each, and shows progress #### Scenario: Tour with GPX geometry - **WHEN** a Komoot tour is imported -- **THEN** the system fetches the tour's GPX/geometry and creates a route with the full track data +- **THEN** the system fetches the tour's GPX and creates a route with the full track data #### Scenario: Import progress - **WHEN** an import is running - **THEN** the UI shows: total tours found, imported so far, duplicates skipped, and current status ### Requirement: Deduplication -The system SHALL NOT create duplicate activities when re-importing tours that were previously imported. +The system SHALL NOT create duplicate activities when re-importing tours that +were previously imported. #### Scenario: Re-import skips existing tours -- **WHEN** user runs import and a tour was already imported previously +- **WHEN** a user runs import and a tour was already imported previously - **THEN** the tour is skipped and counted as a duplicate in the import summary ### Requirement: Import batch tracking @@ -42,17 +70,17 @@ Each import run SHALL be tracked as a batch with status and statistics. #### Scenario: Batch lifecycle - **WHEN** an import starts -- **THEN** a batch record is created with status "running", and updated to "completed" or "failed" when done +- **THEN** a batch record is created with status "running", updated to "completed" or "failed" when done #### Scenario: Batch statistics - **WHEN** an import completes - **THEN** the batch shows: totalFound, importedCount, duplicateCount, and duration -### Requirement: Credential encryption +### Requirement: Credential encryption (authenticated mode) Komoot credentials SHALL be encrypted at rest using AES-256-GCM. #### Scenario: Credentials stored securely -- **WHEN** a user connects their Komoot account +- **WHEN** a user connects via authenticated mode - **THEN** the password is encrypted before storage and only decrypted when making API calls ### Requirement: Privacy disclosure @@ -60,4 +88,4 @@ The Komoot integration SHALL be documented in the privacy manifest. #### Scenario: Privacy manifest updated - **WHEN** Komoot import is available -- **THEN** the /privacy page documents: what credentials are stored, what data is imported, and that credentials are encrypted +- **THEN** the /privacy page documents: that public mode stores only the Komoot username, that authenticated mode stores an encrypted password, and what tour data is imported diff --git a/openspec/changes/komoot-import/tasks.md b/openspec/changes/komoot-import/tasks.md index 6890a79..fe47908 100644 --- a/openspec/changes/komoot-import/tasks.md +++ b/openspec/changes/komoot-import/tasks.md @@ -1,51 +1,42 @@ -## 1. Database Schema +## 1. Komoot API Client -- [ ] 1.1 Add `journal.integrations` table (id, userId, provider enum, encryptedCredentials, apiUsername, status, lastSyncedAt, createdAt) -- [ ] 1.2 Add `journal.import_batches` table (id, userId, integrationId, status enum, totalFound, importedCount, duplicateCount, errorMessage, startedAt, completedAt) -- [ ] 1.3 Add `dedupeKey` column to `journal.activities` table with unique constraint on (ownerId, dedupeKey) -- [ ] 1.4 Add `source` column to `journal.routes` table (nullable, e.g. "komoot", "manual", "gpx-upload") -- [ ] 1.5 Run `pnpm db:push` and verify schema locally +- [ ] 1.1 Add `fetchPublicProfile(komootUserId)` to `komoot.server.ts` — unauthenticated GET of `api.komoot.de/v007/users/{id}/`, returns `{ displayName, contentText, contentLink }` +- [ ] 1.2 Add `fetchPublicTours(komootUserId, page)` to `komoot.server.ts` — unauthenticated GET with `?status=public&limit=50` +- [ ] 1.3 Add `fetchPublicTourGpx(tourId)` — unauthenticated GPX fetch for public tours +- [ ] 1.4 Update `KomootCredentials` type to be a discriminated union: `{ mode: 'public'; username: string } | { mode: 'authenticated'; email: string; password: string; username: string; token: string }` +- [ ] 1.5 Update `fetchTours` / `fetchTourGpx` to branch on credential mode (use auth header only in authenticated mode) +- [ ] 1.6 Add unit tests for `fetchPublicProfile` response parsing and bio verification logic -## 2. Credential Encryption +## 2. Verification Logic -- [ ] 2.1 Create `apps/journal/app/lib/crypto.server.ts` with AES-256-GCM encrypt/decrypt using `INTEGRATION_SECRET` env var -- [ ] 2.2 Write unit tests for encrypt/decrypt roundtrip +- [ ] 2.1 Add `verifyKomootOwnership(komootUserId, trailsProfileUrl)` in `komoot.server.ts` — fetches public profile, checks `content_text` contains the trails.cool URL (case-insensitive, trimmed) +- [ ] 2.2 Write unit tests for verification: match, no match, null bio, trailing slash variations -## 3. Komoot API Client +## 3. Database Schema -- [ ] 3.1 Create `apps/journal/app/lib/komoot.server.ts` with login function (email + password → username + token) -- [ ] 3.2 Add fetchTours function (paginated, fetches all pages) -- [ ] 3.3 Add fetchTourGpx function (fetch GPX geometry for a single tour) -- [ ] 3.4 Write unit tests for API response parsing (mock fetch) +- [ ] 3.1 Add `mode` column (`'public' | 'authenticated'`) to `journal.sync_connections` (or connected_services table per the connected-services architecture) +- [ ] 3.2 Make `encryptedCredentials` nullable (public mode has none) +- [ ] 3.3 Run `pnpm db:push` and verify schema locally -## 4. Import Logic +## 4. API Routes -- [ ] 4.1 Create `apps/journal/app/lib/import.server.ts` with importKomootTours function that: creates batch, pages through tours, fetches GPX, creates activities + routes, deduplicates, updates batch -- [ ] 4.2 Write integration test for import with mock Komoot responses +- [ ] 4.1 Create `POST /api/integrations/komoot/verify` — accepts `{ komootProfileUrl }`, calls `verifyKomootOwnership`, on success stores public connection (no credentials) +- [ ] 4.2 Update `POST /api/integrations/komoot/connect` — now handles authenticated mode only; accepts `{ email, password }` +- [ ] 4.3 Update `POST /api/integrations/komoot/import` — branches on connection mode to use public or authenticated fetch path +- [ ] 4.4 Register new verify route in `apps/journal/app/routes.ts` -## 5. API Routes +## 5. Import Logic -- [ ] 5.1 Create `POST /api/integrations/komoot/connect` — validate credentials, store encrypted -- [ ] 5.2 Create `POST /api/integrations/komoot/disconnect` — delete credentials -- [ ] 5.3 Create `POST /api/integrations/komoot/import` — trigger import, return batch ID -- [ ] 5.4 Create `GET /api/integrations/komoot/import-status` — return current batch progress +- [ ] 5.1 Update `importKomootTours` in `komoot-import.server.ts` to accept the discriminated union credential type and route to public or authenticated fetch functions accordingly +- [ ] 5.2 Update `komootImportJob` background job handler to reconstruct credentials as `{ mode: 'public', username }` when `encryptedCredentials` is null ## 6. UI -- [ ] 6.1 Create `/integrations` route with Komoot connection form (email + password) -- [ ] 6.2 Show connected status, last sync time, and import button when connected -- [ ] 6.3 Import progress UI — poll import-status, show counts (found, imported, duplicated) -- [ ] 6.4 Add link to integrations page from Journal navigation -- [ ] 6.5 Add i18n keys for all integration strings (en + de) +- [ ] 6.1 Add public import section to `/integrations` page above the authenticated form: input for Komoot profile URL, instructions showing the user's trails.cool profile URL to copy, Verify button +- [ ] 6.2 Show verification state: pending instructions → verifying spinner → success (connected, public) or error with retry +- [ ] 6.3 Show mode badge ("Public tours only" vs "All tours") on the connected state UI +- [ ] 6.4 Add i18n keys for all new public-mode strings (en + de) -## 7. Privacy & Config +## 7. Privacy -- [ ] 7.1 Update /privacy page to document Komoot integration (credentials stored encrypted, what data is imported) -- [ ] 7.2 Add `INTEGRATION_SECRET` env var to docker-compose.yml and CI -- [ ] 7.3 Add `INTEGRATION_SECRET` to deploy secrets documentation - -## 8. Verify - -- [ ] 8.1 Test full flow locally: connect Komoot → import tours → verify activities + routes created -- [ ] 8.2 Verify deduplication: re-import and confirm no duplicates -- [ ] 8.3 Verify disconnect removes credentials +- [ ] 7.1 Update `/privacy` page to document both modes: public (stores Komoot username only) and authenticated (stores encrypted password)