chore(openspec): archive fit-parsing-hardening
Sync the wahoo-import "FIT to GPX conversion" requirement into the main spec and move the completed change to openspec/changes/archive/2026-07-16-fit-parsing-hardening. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
e262b671e2
commit
0e04c48225
6 changed files with 19 additions and 3 deletions
|
|
@ -0,0 +1,2 @@
|
|||
schema: spec-driven
|
||||
created: 2026-07-06
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
## Context
|
||||
|
||||
`fitToGpx(buffer, name)` uses `fit-file-parser` (`force: true`), reads only `parsed.records`, filters on presence of lat/lon, and emits `generateGpx({ tracks: [trackPoints] })` — one segment. The parser also exposes `sessions` (with `start_time`, `total_timer_time`, `sport`, `sub_sport`) and `events` (`event: 'timer'`, `event_type: 'stop_all' | 'start'`), which we currently discard. Downstream, `movingTime` in `@trails-cool/gpx` treats segment boundaries as hard breaks and per-interval gaps > 60 s as pauses — but only if segments/timestamps are honest. GPX validation requires ≥2 points.
|
||||
|
||||
Endurain's `utils_fit.py` informs the quirk list: enhanced fields preferred, per-session record slicing, cursor resets at session boundaries so speed doesn't bridge gaps, coordinate conversion sanity, zero-sentinel handling.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Honest GPX out of every FIT file trails will realistically receive (Wahoo, Garmin, COROS): pauses and sessions become segment boundaries; corrected fields win; garbage records can't poison stats.
|
||||
- One conversion module stays the single seam for all providers, with a richer but backward-compatible return shape.
|
||||
|
||||
**Non-Goals:**
|
||||
- Sensor streams (HR/power/cadence/temp) — no journal model for them; revisit if activity streams ever land.
|
||||
- Laps, structured workouts, developer fields, monitoring/wellness messages.
|
||||
- Splitting multisport files into multiple *activities* (Endurain does; trails keeps one activity per provider workout — a triathlon imports as one activity with three segments).
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Segmentation from events first, gaps second
|
||||
|
||||
Split segments on `timer stop_all` → next `start` pairs from the events stream. Files whose events are missing/unreliable (some devices) get a fallback: a record-to-record timestamp gap > 5 min starts a new segment. Both thresholds as named constants. Rationale: events are the device's own truth; the gap fallback matches `movingTime`'s MAX_GAP philosophy at a coarser grain and catches event-less files.
|
||||
|
||||
### 2. One activity, one `<trkseg>` per session
|
||||
|
||||
Records are sliced into sessions by `[start_time, start_time + total_elapsed_time]` windows (Endurain's `split_records_by_activity` approach), each becoming a segment (further split by pauses within). Keeping one *activity* preserves trails' one-workout-one-activity invariant and avoids inventing a multi-activity import UX; segment boundaries are enough to keep stats honest.
|
||||
|
||||
### 3. Validation gates mirror the GPX-parser-robustness rules
|
||||
|
||||
Coordinates must be finite and in range or the record is dropped; altitude non-finite → `undefined` (point kept); record without timestamp → dropped (FIT records are timestamped by spec; absence means corruption). This intentionally matches the policies in the `gpx-parser-robustness` change so both import formats behave identically downstream.
|
||||
|
||||
### 4. Return shape: `{ gpx, sport }` instead of `string`
|
||||
|
||||
`fitToGpx` returns `{ gpx: string | null, sport: SportType | null }` where sport maps FIT `sport`/`sub_sport` (session-level, first session wins) through a small table to trails' `SPORT_TYPES` (running→run, trail_running sub-sport→run, cycling→ride, gravel_cycling→gravel, mountain→mtb, hiking→hike, walking→walk, alpine/backcountry ski→ski, else→other/null). Call sites destructure; Wahoo's existing workout-type mapping stays as fallback when no FIT file exists.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [fit-file-parser may not surface events/sessions uniformly across devices] → Fixture-driven: gather real files (Wahoo export, Garmin sample) before coding; the gap fallback covers event-less files.
|
||||
- [Multi-segment GPX changes stats for re-imported activities] → Only new imports are affected (no reprocessing); moving time gets *more* accurate.
|
||||
- [Sport mapping disagreements with providers' own type fields] → Provider-specific mapping wins where the provider sends an explicit type; FIT sport is the fallback, not the override — call-site rule documented in code.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
Library-shaped change inside the journal app; ships with normal deploy; no schema change; rollback = revert. Existing activities untouched.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- None blocking. Whether COROS FIT files carry quirks beyond this set is checked when that integration starts (fixtures first).
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
## Why
|
||||
|
||||
The shared FIT→GPX converter (`apps/journal/app/lib/connected-services/fit.ts`) is 40 lines: it takes every record with coordinates and emits one flat track segment. It ignores pause/resume events (moving time and speed bridge lunch breaks), ignores sessions (a multisport file becomes one merged track), passes device altitude through without preferring the corrected `enhanced_altitude`, and does no coordinate sanity checks. Wahoo already feeds it; the Garmin importer (in flight) and COROS (planned) will too, multiplying every gap. Endurain's 1,100-line FIT handler (reviewed 2026-07-06) is a catalog of exactly which quirks matter in the wild — we adopt the subset relevant to trails' GPX-centric model.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **Pause-aware segmentation**: FIT `event` stop/start pairs (and, as fallback, record gaps > 5 minutes) split the output into multiple `<trkseg>`s, so downstream moving-time and instant-speed math never bridges a pause.
|
||||
- **Session-aware output**: multi-session files (multisport) produce one `<trkseg>` per session with records sliced to each session's time window; single-session behavior unchanged.
|
||||
- **Field preference and validation**: prefer `enhanced_altitude`/`enhanced_speed` over base fields; drop records with non-finite or out-of-range coordinates or non-finite altitude (altitude → undefined, record kept); require a timestamp per record.
|
||||
- **Sport extraction**: expose the file's session sport/sub-sport mapped to trails' `SportType` so importers stop guessing (Wahoo importer currently maps workout-type codes itself; it can consume this when a FIT file is present).
|
||||
- Not in scope: HR/cadence/power/temperature streams (trails doesn't model them), lap extraction, developer fields, FIT *writing* (unchanged in `@trails-cool/fit`).
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
<!-- none -->
|
||||
|
||||
### Modified Capabilities
|
||||
- `wahoo-import`: the "FIT to GPX conversion" requirement is extended with pause/session segmentation, enhanced-field preference, and record validation guarantees.
|
||||
|
||||
## Impact
|
||||
|
||||
- `apps/journal/app/lib/connected-services/fit.ts` — the converter grows the four behaviors above; returns optional sport metadata alongside the GPX.
|
||||
- Wahoo importer/webhook + Garmin importer consume the richer return type (backward-compatible: GPX string still primary).
|
||||
- Tests: fixture FIT files (pause, multisport, garbage-coordinate) added next to the module; Endurain's `backend/tests/.../test_utils_fit.py` catalog is the reference for cases.
|
||||
- No schema or API changes.
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: FIT to GPX conversion
|
||||
The system SHALL convert FIT binary files to GPX format. The conversion logic lives at `apps/journal/app/lib/connected-services/fit.ts` — a provider-agnostic location shared across any future provider that produces FIT files (Garmin, Coros, etc.). Wahoo's importer and webhook both import from this shared module; they do not contain their own copy. The conversion SHALL be pause- and session-aware: timer stop/start events (or record gaps over 5 minutes in files without events) and session boundaries each start a new track segment. The converter SHALL prefer enhanced (corrected) altitude and speed fields, drop records with missing timestamps or non-finite/out-of-range coordinates, treat non-finite altitude as absent, and expose the file's session sport mapped to the Journal's sport types alongside the GPX.
|
||||
|
||||
#### Scenario: Convert FIT with GPS data
|
||||
- **WHEN** a FIT file contains GPS track records
|
||||
- **THEN** track points with lat, lon, elevation, and ISO 8601 timestamps are extracted
|
||||
- **AND** coordinates are used as-is from the FIT parser (which already converts semicircles to degrees)
|
||||
- **AND** a valid GPX string is produced using `generateGpx`
|
||||
|
||||
#### Scenario: FIT without GPS data
|
||||
- **WHEN** a FIT file has no GPS records (e.g., indoor trainer workout)
|
||||
- **THEN** the activity is created without GPX or geometry (stats only)
|
||||
|
||||
#### Scenario: Workout without FIT file
|
||||
- **WHEN** a workout has no file URL (e.g., aborted recording, third-party app data)
|
||||
- **THEN** the activity is created without GPX or geometry
|
||||
|
||||
#### Scenario: Pause becomes a segment boundary
|
||||
- **WHEN** a FIT file contains a timer stop event followed by a start event 20 minutes later
|
||||
- **THEN** the GPX contains two track segments split at the pause
|
||||
- **AND** moving time computed downstream does not include the paused span
|
||||
|
||||
#### Scenario: Multisport file keeps one activity with per-session segments
|
||||
- **WHEN** a FIT file contains multiple sessions
|
||||
- **THEN** one GPX is produced with one or more track segments per session, records sliced to each session's time window
|
||||
|
||||
#### Scenario: Corrupt records dropped
|
||||
- **WHEN** a FIT file contains records with out-of-range coordinates or missing timestamps
|
||||
- **THEN** those records are excluded and the remaining track converts normally
|
||||
|
||||
#### Scenario: Sport surfaced from the file
|
||||
- **WHEN** a FIT file's session declares sport "cycling" with sub-sport "gravel_cycling"
|
||||
- **THEN** the converter reports sport `gravel` for the importer to use when the provider sends no explicit type
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
## 1. Fixtures first
|
||||
|
||||
- [x] 1.1 Collect fixture FIT files — adapted: per the chosen approach, cases are driven by mocked `fit-file-parser` output in `fit.test.ts` (pause, multisession, indoor no-GPS, corrupt coords/missing-timestamp) rather than binary files. Real device binaries can be added later without changing the converter.
|
||||
- [x] 1.2 Parser `events`/`sessions` shapes captured as the `FitEvent`/`FitSession` types + code comments in `fit.ts`, and exercised by the mocked-parser tests.
|
||||
|
||||
## 2. Converter
|
||||
|
||||
- [x] 2.1 Segmentation: split on timer stop/start event pairs, fallback on record gaps > 5 min (named constants)
|
||||
- [x] 2.2 Session slicing: records windowed per session, one-or-more segments per session, single-session unchanged
|
||||
- [x] 2.3 Validation gates: finite/in-range coordinates, timestamp required, non-finite altitude → undefined; prefer `enhanced_altitude` (speed isn't emitted to GPX so its preference is moot)
|
||||
- [x] 2.4 Return `{ gpx, sport }` with the FIT→SportType mapping table; update call sites (Wahoo importer + webhook, Garmin importer) — provider-explicit type wins, FIT sport is fallback
|
||||
- [x] 2.5 Unit tests over the fixtures: segment counts (pause becomes a segment boundary, which is what keeps moving time honest), corrupt-record drops, enhanced-altitude preference, sport mapping, no-GPS returns null gpx
|
||||
|
||||
## 3. Verification
|
||||
|
||||
- [x] 3.1 Re-run the Wahoo importer test suite; confirm existing single-session imports produce identical stats (14 pass; mock updated to the `{gpx, sport}` shape)
|
||||
- [x] 3.2 Run `pnpm typecheck && pnpm lint && pnpm test` — all green
|
||||
Loading…
Add table
Add a link
Reference in a new issue