Wire INTEGRATION_SECRET into docker-compose and CI; archive komoot-import
- Add INTEGRATION_SECRET to journal service in docker-compose.yml with :? guard so a missing value fails loudly at compose-up time - Add INTEGRATION_SECRET to E2E test step in ci.yml via GitHub secret (unit tests already set their own value in the test file) - Archive openspec/changes/komoot-import → archive/2026-05-23-komoot-import - Sync delta specs: new openspec/specs/komoot-import/spec.md, updated openspec/specs/route-management/spec.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer
parent
5158298424
commit
ae23d31a5f
10 changed files with 397 additions and 0 deletions
|
|
@ -0,0 +1,2 @@
|
|||
schema: spec-driven
|
||||
created: 2026-03-25
|
||||
100
openspec/changes/archive/2026-05-23-komoot-import/design.md
Normal file
100
openspec/changes/archive/2026-05-23-komoot-import/design.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
## Context
|
||||
|
||||
trails.cool Journal supports manual route creation and GPX import. Users coming
|
||||
from Komoot have hundreds of tours they'd lose by switching. The Komoot public
|
||||
API (`api.komoot.de/v007`) exposes public tours and user profiles without
|
||||
authentication. Profile fields (`content_text`, `content_link`) are readable
|
||||
unauthenticated after a short cache delay (~minutes).
|
||||
|
||||
> **Note (added 2026-05-08, post `deepen-connected-services`)**:
|
||||
> Earlier drafts of this design proposed a separate `journal.integrations`
|
||||
> table for Komoot credentials. That has been **superseded** by the
|
||||
> connected-services architecture introduced in
|
||||
> `openspec/changes/deepen-connected-services/`. When this change is
|
||||
> revisited, Komoot must implement:
|
||||
>
|
||||
> - A row in `journal.connected_services` with `credential_kind = 'web-login'`
|
||||
> (authenticated mode) or `credential_kind = 'public'` (public mode).
|
||||
> - A `KomootImporter` in
|
||||
> `apps/journal/app/lib/connected-services/providers/komoot/importer.ts`
|
||||
> that branches on credential kind.
|
||||
> - A manifest at `providers/komoot/manifest.ts` registered via
|
||||
> `providers/index.ts`.
|
||||
>
|
||||
> Don't add a `journal.integrations` table. The user-facing "Connected
|
||||
> Services" list at `/settings/connections` should show Komoot alongside Wahoo.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Public import: verify Komoot profile ownership via bio field, import public tours, store no passwords
|
||||
- Authenticated import: connect via email + password, import all tours including private
|
||||
- Import all tours (paginated) as activities + routes
|
||||
- Track import progress with batch status
|
||||
- Deduplicate on re-import (same tour never imported twice)
|
||||
- Fetch GPX geometry per tour (not just metadata)
|
||||
- Cross-link: store Komoot username on the connection so the Journal profile can show "also on Komoot"
|
||||
|
||||
**Non-Goals:**
|
||||
- OAuth flow
|
||||
- Real-time sync or webhook-based updates
|
||||
- Other providers (future iteration)
|
||||
|
||||
## Import Modes
|
||||
|
||||
### Public mode
|
||||
|
||||
No credentials stored. Ownership is verified by checking that the user's
|
||||
trails.cool profile URL appears in their Komoot `content_text` (bio/"Über dich")
|
||||
field, which is readable via the unauthenticated public API.
|
||||
|
||||
Verification flow:
|
||||
1. User enters their Komoot profile URL (e.g. `komoot.com/user/27595800585`)
|
||||
2. trails.cool shows: *"Add your trails.cool profile link (`https://trails.cool/users/ullrich`) to your Komoot bio ('Über dich'), then click Verify"*
|
||||
3. trails.cool fetches `api.komoot.de/v007/users/{id}/` and checks `content_text` contains their trails.cool profile URL
|
||||
4. On success: store Komoot username in `connected_services` with `credential_kind = 'public'`, no password
|
||||
5. Fetch `api.komoot.de/v007/users/{id}/tours/?status=public` (paginated), import
|
||||
|
||||
### Authenticated mode
|
||||
|
||||
User provides email + password. Credentials validated and stored AES-256-GCM
|
||||
encrypted. All tours (public and private) imported.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1: Two connection modes in `connected_services`
|
||||
|
||||
A `mode` column (`'public' | 'authenticated'`) on the connection row controls
|
||||
which import path runs. Public mode rows have no `encryptedCredentials`.
|
||||
|
||||
### D2: Bio field for ownership verification
|
||||
|
||||
`api.komoot.de/v007/users/{id}/` returns `content_text` (bio) unauthenticated.
|
||||
Checking for the user's own trails.cool profile URL proves they control the
|
||||
Komoot account without any credential exchange. The profile link stays in the
|
||||
bio permanently, providing cross-platform discovery.
|
||||
|
||||
### D3: Import batches for progress tracking
|
||||
|
||||
Each import creates a batch row tracking: status, total found, imported count,
|
||||
duplicate count, error message. UI polls for live progress.
|
||||
|
||||
### D4: Deduplication via dedupe key
|
||||
|
||||
Activities get a `dedupeKey` of `komoot:{tourId}`. Unique constraint on
|
||||
`(ownerId, dedupeKey)` prevents duplicates on re-import.
|
||||
|
||||
### D5: AES-256-GCM for credential encryption (authenticated mode only)
|
||||
|
||||
`INTEGRATION_SECRET` env var → scrypt-derived key. Decrypt on use, never log.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **Komoot API is undocumented** → Could change without notice. All calls
|
||||
wrapped in error handling; connection marked as needing reauth on 401.
|
||||
- **Public mode: only public tours importable** → Expected and documented.
|
||||
Users who want private tours use authenticated mode.
|
||||
- **Bio field cache delay** → Verification shows a clear "allow a few minutes
|
||||
for changes to propagate" message and a retry button.
|
||||
- **Authenticated mode: storing third-party passwords** → AES-256-GCM,
|
||||
separate `INTEGRATION_SECRET`, documented in privacy manifest.
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
## Why
|
||||
|
||||
Users switching to trails.cool have years of tours on Komoot. Without import,
|
||||
they start with an empty Journal — no history, no motivation to switch. A
|
||||
Komoot import lets users bring their existing tours and start using
|
||||
trails.cool immediately.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **Public import** (no credentials): user proves ownership of their Komoot
|
||||
profile by placing their trails.cool profile URL in their Komoot bio field.
|
||||
trails.cool verifies via the public Komoot API, then imports all public tours.
|
||||
- **Authenticated import** (email + password): imports all tours including
|
||||
private ones. Credentials stored encrypted.
|
||||
- Both modes share the same batch import engine, progress tracking, and
|
||||
deduplication logic.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `komoot-import`: Two-mode Komoot import — public (bio verification, no
|
||||
credential storage) and authenticated (email + password, all tours including
|
||||
private). Both track batch progress and deduplicate on re-import.
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `route-management`: Routes can now be created via import (not just manual
|
||||
creation), with an external source tracking field.
|
||||
|
||||
## Impact
|
||||
|
||||
- **Database**: New tables for integration connections and import batches in
|
||||
`journal` schema. Connections can be credential-free (public mode).
|
||||
- **Files**: New routes (`/integrations`, `/api/integrations/*`), new server
|
||||
utilities for Komoot API, new DB schema tables
|
||||
- **Dependencies**: No new packages — uses fetch for Komoot API, existing
|
||||
Drizzle for DB
|
||||
- **Privacy**: Only authenticated mode stores credentials (encrypted). Public
|
||||
mode stores only the Komoot username. Both documented in privacy manifest.
|
||||
- **External**: Komoot public API (`api.komoot.de`) — unauthenticated for
|
||||
public tours and profile lookup; basic auth for authenticated mode
|
||||
|
|
@ -0,0 +1,91 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Connect Komoot account — public mode
|
||||
Users SHALL be able to connect their Komoot account without providing credentials
|
||||
by verifying ownership via their Komoot bio field.
|
||||
|
||||
#### Scenario: Initiate public connection
|
||||
- **WHEN** a user enters their Komoot profile URL on the integrations page
|
||||
- **THEN** the system displays their trails.cool profile URL and instructs them to add it to their Komoot bio ("Über dich" field)
|
||||
|
||||
#### Scenario: Verify public ownership
|
||||
- **WHEN** the user clicks Verify after updating their Komoot bio
|
||||
- **THEN** the system fetches the public Komoot profile, checks that `content_text` contains the user's trails.cool profile URL, and on success marks the connection as active with mode "public"
|
||||
|
||||
#### Scenario: Bio not yet updated
|
||||
- **WHEN** verification is attempted but the trails.cool URL is not found in the Komoot bio
|
||||
- **THEN** the system shows an error explaining that changes may take a few minutes to propagate, with a retry button
|
||||
|
||||
#### Scenario: Disconnect public account
|
||||
- **WHEN** a user disconnects their public Komoot integration
|
||||
- **THEN** the stored Komoot username is removed and no further imports can occur
|
||||
|
||||
### Requirement: Connect Komoot account — authenticated mode
|
||||
Users SHALL be able to connect their Komoot account by providing email and
|
||||
password to access all tours including private ones.
|
||||
|
||||
#### Scenario: Successful authenticated connection
|
||||
- **WHEN** a user enters valid Komoot email and password
|
||||
- **THEN** the system validates credentials via the Komoot API, stores them encrypted, and shows the connection as active with mode "authenticated"
|
||||
|
||||
#### Scenario: Invalid credentials
|
||||
- **WHEN** a user enters incorrect Komoot credentials
|
||||
- **THEN** the system shows an error and does not store credentials
|
||||
|
||||
#### Scenario: Disconnect authenticated account
|
||||
- **WHEN** a user disconnects their authenticated Komoot integration
|
||||
- **THEN** stored credentials are deleted and no further imports can occur
|
||||
|
||||
### Requirement: Import Komoot tours
|
||||
Users SHALL be able to import their Komoot tours as Journal activities with
|
||||
linked routes. Public mode imports public tours only; authenticated mode imports
|
||||
all tours.
|
||||
|
||||
#### Scenario: Import public tours
|
||||
- **WHEN** a user with a public-mode connection triggers import
|
||||
- **THEN** the system fetches all public tours from the Komoot API without credentials, creates an activity and route for each, and shows progress
|
||||
|
||||
#### Scenario: Import all tours (authenticated)
|
||||
- **WHEN** a user with an authenticated-mode connection triggers import
|
||||
- **THEN** the system fetches all tours (public and private, paginated) using stored credentials, creates an activity and route for each, and shows progress
|
||||
|
||||
#### Scenario: Tour with GPX geometry
|
||||
- **WHEN** a Komoot tour is imported
|
||||
- **THEN** the system fetches the tour's GPX and creates a route with the full track data
|
||||
|
||||
#### Scenario: Import progress
|
||||
- **WHEN** an import is running
|
||||
- **THEN** the UI shows: total tours found, imported so far, duplicates skipped, and current status
|
||||
|
||||
### Requirement: Deduplication
|
||||
The system SHALL NOT create duplicate activities when re-importing tours that
|
||||
were previously imported.
|
||||
|
||||
#### Scenario: Re-import skips existing tours
|
||||
- **WHEN** a user runs import and a tour was already imported previously
|
||||
- **THEN** the tour is skipped and counted as a duplicate in the import summary
|
||||
|
||||
### Requirement: Import batch tracking
|
||||
Each import run SHALL be tracked as a batch with status and statistics.
|
||||
|
||||
#### Scenario: Batch lifecycle
|
||||
- **WHEN** an import starts
|
||||
- **THEN** a batch record is created with status "running", updated to "completed" or "failed" when done
|
||||
|
||||
#### Scenario: Batch statistics
|
||||
- **WHEN** an import completes
|
||||
- **THEN** the batch shows: totalFound, importedCount, duplicateCount, and duration
|
||||
|
||||
### Requirement: Credential encryption (authenticated mode)
|
||||
Komoot credentials SHALL be encrypted at rest using AES-256-GCM.
|
||||
|
||||
#### Scenario: Credentials stored securely
|
||||
- **WHEN** a user connects via authenticated mode
|
||||
- **THEN** the password is encrypted before storage and only decrypted when making API calls
|
||||
|
||||
### Requirement: Privacy disclosure
|
||||
The Komoot integration SHALL be documented in the privacy manifest.
|
||||
|
||||
#### Scenario: Privacy manifest updated
|
||||
- **WHEN** Komoot import is available
|
||||
- **THEN** the /privacy page documents: that public mode stores only the Komoot username, that authenticated mode stores an encrypted password, and what tour data is imported
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Route creation
|
||||
The system SHALL support creating routes via import from external services, in addition to manual creation and GPX upload.
|
||||
|
||||
#### Scenario: Route created from import
|
||||
- **WHEN** a Komoot tour is imported
|
||||
- **THEN** a route is created with name, distance, elevation, GPX geometry, and a source field indicating "komoot"
|
||||
|
||||
#### Scenario: Route links to activity
|
||||
- **WHEN** a tour is imported as an activity
|
||||
- **THEN** the activity is linked to the created route
|
||||
42
openspec/changes/archive/2026-05-23-komoot-import/tasks.md
Normal file
42
openspec/changes/archive/2026-05-23-komoot-import/tasks.md
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
## 1. Komoot API Client
|
||||
|
||||
- [x] 1.1 Add `fetchPublicProfile(komootUserId)` to `komoot.server.ts` — unauthenticated GET of `api.komoot.de/v007/users/{id}/`, returns `{ displayName, contentText, contentLink }`
|
||||
- [x] 1.2 Add `fetchPublicTours(komootUserId, page)` to `komoot.server.ts` — unauthenticated GET with `?status=public&limit=50`
|
||||
- [x] 1.3 Add `fetchPublicTourGpx(tourId)` — unauthenticated GPX fetch for public tours
|
||||
- [x] 1.4 Update `KomootCredentials` type to be a discriminated union: `{ mode: 'public'; komootUserId: string } | { mode: 'authenticated'; email: string; encryptedPassword: string; komootUserId: string }`
|
||||
- [x] 1.5 Update `fetchTours` / `fetchTourGpx` to branch on credential mode (use auth header only in authenticated mode)
|
||||
- [x] 1.6 Add unit tests for `fetchPublicProfile` response parsing and bio verification logic
|
||||
|
||||
## 2. Verification Logic
|
||||
|
||||
- [x] 2.1 Add `verifyKomootOwnership(komootUserId, trailsProfileUrl)` in `komoot.server.ts` — fetches public profile, checks `content_text` contains the trails.cool URL (case-insensitive, trimmed)
|
||||
- [x] 2.2 Write unit tests for verification: match, no match, null bio, trailing slash variations
|
||||
|
||||
## 3. Database Schema
|
||||
|
||||
- [x] 3.1 Add `mode` column (`'public' | 'authenticated'`) to `journal.sync_connections` (or connected_services table per the connected-services architecture)
|
||||
- [x] 3.2 Make `encryptedCredentials` nullable (public mode has none)
|
||||
- [x] 3.3 Run `pnpm db:push` and verify schema locally
|
||||
|
||||
## 4. API Routes
|
||||
|
||||
- [x] 4.1 Create `POST /api/sync/komoot/verify` — accepts `{ komootProfileUrl }`, calls `verifyKomootOwnership`, on success stores public connection (no credentials)
|
||||
- [x] 4.2 Create `POST /api/sync/komoot/connect` — authenticated mode; accepts `{ email, password }`
|
||||
- [x] 4.3 `sync.import.komoot.tsx` — branches on connection mode to use public or authenticated fetch path
|
||||
- [x] 4.4 Register new routes in `apps/journal/app/routes.ts`
|
||||
|
||||
## 5. Import Logic
|
||||
|
||||
- [x] 5.1 `importer.ts` accepts the discriminated union credential type and routes to public or authenticated fetch functions accordingly
|
||||
- [x] 5.2 Credentials stored as discriminated union in connected_services; mode determined from stored credentials
|
||||
|
||||
## 6. UI
|
||||
|
||||
- [x] 6.1 Add public import section to `/settings/connections/komoot` above the authenticated form: input for Komoot profile URL, instructions showing the user's trails.cool profile URL to copy, Verify button
|
||||
- [x] 6.2 Show verification state: pending instructions → verifying spinner → success (connected, public) or error with retry
|
||||
- [x] 6.3 Show mode badge ("Public tours only" vs "All tours") on the connected state UI
|
||||
- [x] 6.4 Add i18n keys for all new public-mode strings (en + de)
|
||||
|
||||
## 7. Privacy
|
||||
|
||||
- [x] 7.1 Update `/privacy` page to document both modes: public (stores Komoot username only) and authenticated (stores encrypted password)
|
||||
Loading…
Add table
Add a link
Reference in a new issue