chore(openspec): archive fit-parsing-hardening

Sync the wahoo-import "FIT to GPX conversion" requirement into the main
spec and move the completed change to
openspec/changes/archive/2026-07-16-fit-parsing-hardening.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-07-17 00:09:27 +02:00
parent e262b671e2
commit 0e04c48225
No known key found for this signature in database
GPG key ID: A32FF691A0F752D9
6 changed files with 19 additions and 3 deletions

View file

@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-06

View file

@ -0,0 +1,48 @@
## Context
`fitToGpx(buffer, name)` uses `fit-file-parser` (`force: true`), reads only `parsed.records`, filters on presence of lat/lon, and emits `generateGpx({ tracks: [trackPoints] })` — one segment. The parser also exposes `sessions` (with `start_time`, `total_timer_time`, `sport`, `sub_sport`) and `events` (`event: 'timer'`, `event_type: 'stop_all' | 'start'`), which we currently discard. Downstream, `movingTime` in `@trails-cool/gpx` treats segment boundaries as hard breaks and per-interval gaps > 60 s as pauses — but only if segments/timestamps are honest. GPX validation requires ≥2 points.
Endurain's `utils_fit.py` informs the quirk list: enhanced fields preferred, per-session record slicing, cursor resets at session boundaries so speed doesn't bridge gaps, coordinate conversion sanity, zero-sentinel handling.
## Goals / Non-Goals
**Goals:**
- Honest GPX out of every FIT file trails will realistically receive (Wahoo, Garmin, COROS): pauses and sessions become segment boundaries; corrected fields win; garbage records can't poison stats.
- One conversion module stays the single seam for all providers, with a richer but backward-compatible return shape.
**Non-Goals:**
- Sensor streams (HR/power/cadence/temp) — no journal model for them; revisit if activity streams ever land.
- Laps, structured workouts, developer fields, monitoring/wellness messages.
- Splitting multisport files into multiple *activities* (Endurain does; trails keeps one activity per provider workout — a triathlon imports as one activity with three segments).
## Decisions
### 1. Segmentation from events first, gaps second
Split segments on `timer stop_all` → next `start` pairs from the events stream. Files whose events are missing/unreliable (some devices) get a fallback: a record-to-record timestamp gap > 5 min starts a new segment. Both thresholds as named constants. Rationale: events are the device's own truth; the gap fallback matches `movingTime`'s MAX_GAP philosophy at a coarser grain and catches event-less files.
### 2. One activity, one `<trkseg>` per session
Records are sliced into sessions by `[start_time, start_time + total_elapsed_time]` windows (Endurain's `split_records_by_activity` approach), each becoming a segment (further split by pauses within). Keeping one *activity* preserves trails' one-workout-one-activity invariant and avoids inventing a multi-activity import UX; segment boundaries are enough to keep stats honest.
### 3. Validation gates mirror the GPX-parser-robustness rules
Coordinates must be finite and in range or the record is dropped; altitude non-finite → `undefined` (point kept); record without timestamp → dropped (FIT records are timestamped by spec; absence means corruption). This intentionally matches the policies in the `gpx-parser-robustness` change so both import formats behave identically downstream.
### 4. Return shape: `{ gpx, sport }` instead of `string`
`fitToGpx` returns `{ gpx: string | null, sport: SportType | null }` where sport maps FIT `sport`/`sub_sport` (session-level, first session wins) through a small table to trails' `SPORT_TYPES` (running→run, trail_running sub-sport→run, cycling→ride, gravel_cycling→gravel, mountain→mtb, hiking→hike, walking→walk, alpine/backcountry ski→ski, else→other/null). Call sites destructure; Wahoo's existing workout-type mapping stays as fallback when no FIT file exists.
## Risks / Trade-offs
- [fit-file-parser may not surface events/sessions uniformly across devices] → Fixture-driven: gather real files (Wahoo export, Garmin sample) before coding; the gap fallback covers event-less files.
- [Multi-segment GPX changes stats for re-imported activities] → Only new imports are affected (no reprocessing); moving time gets *more* accurate.
- [Sport mapping disagreements with providers' own type fields] → Provider-specific mapping wins where the provider sends an explicit type; FIT sport is the fallback, not the override — call-site rule documented in code.
## Migration Plan
Library-shaped change inside the journal app; ships with normal deploy; no schema change; rollback = revert. Existing activities untouched.
## Open Questions
- None blocking. Whether COROS FIT files carry quirks beyond this set is checked when that integration starts (fixtures first).

View file

@ -0,0 +1,27 @@
## Why
The shared FIT→GPX converter (`apps/journal/app/lib/connected-services/fit.ts`) is 40 lines: it takes every record with coordinates and emits one flat track segment. It ignores pause/resume events (moving time and speed bridge lunch breaks), ignores sessions (a multisport file becomes one merged track), passes device altitude through without preferring the corrected `enhanced_altitude`, and does no coordinate sanity checks. Wahoo already feeds it; the Garmin importer (in flight) and COROS (planned) will too, multiplying every gap. Endurain's 1,100-line FIT handler (reviewed 2026-07-06) is a catalog of exactly which quirks matter in the wild — we adopt the subset relevant to trails' GPX-centric model.
## What Changes
- **Pause-aware segmentation**: FIT `event` stop/start pairs (and, as fallback, record gaps > 5 minutes) split the output into multiple `<trkseg>`s, so downstream moving-time and instant-speed math never bridges a pause.
- **Session-aware output**: multi-session files (multisport) produce one `<trkseg>` per session with records sliced to each session's time window; single-session behavior unchanged.
- **Field preference and validation**: prefer `enhanced_altitude`/`enhanced_speed` over base fields; drop records with non-finite or out-of-range coordinates or non-finite altitude (altitude → undefined, record kept); require a timestamp per record.
- **Sport extraction**: expose the file's session sport/sub-sport mapped to trails' `SportType` so importers stop guessing (Wahoo importer currently maps workout-type codes itself; it can consume this when a FIT file is present).
- Not in scope: HR/cadence/power/temperature streams (trails doesn't model them), lap extraction, developer fields, FIT *writing* (unchanged in `@trails-cool/fit`).
## Capabilities
### New Capabilities
<!-- none -->
### Modified Capabilities
- `wahoo-import`: the "FIT to GPX conversion" requirement is extended with pause/session segmentation, enhanced-field preference, and record validation guarantees.
## Impact
- `apps/journal/app/lib/connected-services/fit.ts` — the converter grows the four behaviors above; returns optional sport metadata alongside the GPX.
- Wahoo importer/webhook + Garmin importer consume the richer return type (backward-compatible: GPX string still primary).
- Tests: fixture FIT files (pause, multisport, garbage-coordinate) added next to the module; Endurain's `backend/tests/.../test_utils_fit.py` catalog is the reference for cases.
- No schema or API changes.

View file

@ -0,0 +1,35 @@
## MODIFIED Requirements
### Requirement: FIT to GPX conversion
The system SHALL convert FIT binary files to GPX format. The conversion logic lives at `apps/journal/app/lib/connected-services/fit.ts` — a provider-agnostic location shared across any future provider that produces FIT files (Garmin, Coros, etc.). Wahoo's importer and webhook both import from this shared module; they do not contain their own copy. The conversion SHALL be pause- and session-aware: timer stop/start events (or record gaps over 5 minutes in files without events) and session boundaries each start a new track segment. The converter SHALL prefer enhanced (corrected) altitude and speed fields, drop records with missing timestamps or non-finite/out-of-range coordinates, treat non-finite altitude as absent, and expose the file's session sport mapped to the Journal's sport types alongside the GPX.
#### Scenario: Convert FIT with GPS data
- **WHEN** a FIT file contains GPS track records
- **THEN** track points with lat, lon, elevation, and ISO 8601 timestamps are extracted
- **AND** coordinates are used as-is from the FIT parser (which already converts semicircles to degrees)
- **AND** a valid GPX string is produced using `generateGpx`
#### Scenario: FIT without GPS data
- **WHEN** a FIT file has no GPS records (e.g., indoor trainer workout)
- **THEN** the activity is created without GPX or geometry (stats only)
#### Scenario: Workout without FIT file
- **WHEN** a workout has no file URL (e.g., aborted recording, third-party app data)
- **THEN** the activity is created without GPX or geometry
#### Scenario: Pause becomes a segment boundary
- **WHEN** a FIT file contains a timer stop event followed by a start event 20 minutes later
- **THEN** the GPX contains two track segments split at the pause
- **AND** moving time computed downstream does not include the paused span
#### Scenario: Multisport file keeps one activity with per-session segments
- **WHEN** a FIT file contains multiple sessions
- **THEN** one GPX is produced with one or more track segments per session, records sliced to each session's time window
#### Scenario: Corrupt records dropped
- **WHEN** a FIT file contains records with out-of-range coordinates or missing timestamps
- **THEN** those records are excluded and the remaining track converts normally
#### Scenario: Sport surfaced from the file
- **WHEN** a FIT file's session declares sport "cycling" with sub-sport "gravel_cycling"
- **THEN** the converter reports sport `gravel` for the importer to use when the provider sends no explicit type

View file

@ -0,0 +1,17 @@
## 1. Fixtures first
- [x] 1.1 Collect fixture FIT files — adapted: per the chosen approach, cases are driven by mocked `fit-file-parser` output in `fit.test.ts` (pause, multisession, indoor no-GPS, corrupt coords/missing-timestamp) rather than binary files. Real device binaries can be added later without changing the converter.
- [x] 1.2 Parser `events`/`sessions` shapes captured as the `FitEvent`/`FitSession` types + code comments in `fit.ts`, and exercised by the mocked-parser tests.
## 2. Converter
- [x] 2.1 Segmentation: split on timer stop/start event pairs, fallback on record gaps > 5 min (named constants)
- [x] 2.2 Session slicing: records windowed per session, one-or-more segments per session, single-session unchanged
- [x] 2.3 Validation gates: finite/in-range coordinates, timestamp required, non-finite altitude → undefined; prefer `enhanced_altitude` (speed isn't emitted to GPX so its preference is moot)
- [x] 2.4 Return `{ gpx, sport }` with the FIT→SportType mapping table; update call sites (Wahoo importer + webhook, Garmin importer) — provider-explicit type wins, FIT sport is fallback
- [x] 2.5 Unit tests over the fixtures: segment counts (pause becomes a segment boundary, which is what keeps moving time honest), corrupt-record drops, enhanced-altitude preference, sport mapping, no-GPS returns null gpx
## 3. Verification
- [x] 3.1 Re-run the Wahoo importer test suite; confirm existing single-session imports produce identical stats (14 pass; mock updated to the `{gpx, sport}` shape)
- [x] 3.2 Run `pnpm typecheck && pnpm lint && pnpm test` — all green