chore(openspec): archive planner-route-encoding
All 12 tasks complete (shipped in #590 codec + #591 wiring). Archive via `openspec archive`: - Creates openspec/specs/planner-route-encoding/spec.md — new capability (5 requirements: compact single-source geometry, RLE road metadata, backward-compatible reads, preserved compute-once model, bounded doc size). Purpose filled in (CLI leaves TBD). - Moves the change to openspec/changes/archive/2026-07-15-planner-route-encoding/. Spec passes `openspec validate --strict`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer
parent
86fc596e50
commit
6fcd1e21cd
7 changed files with 54 additions and 0 deletions
|
|
@ -0,0 +1,2 @@
|
|||
schema: spec-driven
|
||||
created: 2026-07-14
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
## Context
|
||||
|
||||
`writeComputedRoute(doc, routeData, enriched)` in `apps/planner/app/lib/route-data.ts` writes, per computed route:
|
||||
`geojson` (stringified GeoJSON LineString), `coordinates` (stringified `[lon,lat][]` — the same points again), `segmentBoundaries`, and the seven `ROAD_METADATA_KEYS` arrays (`surfaces`/`highways`/`maxspeeds`/`smoothnesses`/`tracktypes`/`cycleways`/`bikeroutes`, each a stringified `string[]`). Reads go through helpers in the same module (`readComputedRoute`, `readRoadMetadata`, `getProfile`). The routing host (elected via Yjs awareness) computes the route once and calls `writeComputedRoute`; all other participants only read. The doc is persisted opaquely as `planner.sessions.yjs_state` (bytea) and re-synced in full on every reconnect.
|
||||
|
||||
The size is dominated by geometry (stored twice, as JSON floats) and the metadata arrays (one entry per coordinate, JSON strings). None of it is user intent — it is a deterministic function of waypoints + profile + no-go areas.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- ~5× smaller `routeData` (≈305 KB → ≈40–60 KB for a 77 km route), scaling far better with length.
|
||||
- Zero change to the routing-host "compute once, write to doc, everyone reads" model.
|
||||
- Backward compatible: old JSON-format docs already persisted must keep working with no migration step and no flag day.
|
||||
- Readers unchanged at their call sites — decode is centralized in `route-data.ts`.
|
||||
- GPX export output stays byte-identical.
|
||||
|
||||
**Non-Goals:**
|
||||
- Changing routing-host election, BRouter usage, or making clients recompute.
|
||||
- Removing the computed route from the doc (compute-once-and-share stays).
|
||||
- Changing `MAX_DOC_BYTES` (5 MB) or route-coloring semantics.
|
||||
- A lossy geometry simplification (Douglas–Peucker etc.) — separate concern; this change is lossless re-encoding.
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Single geometry source, derived views
|
||||
Store geometry once. Keep `coordinates` as the canonical `[lon,lat][]` and DROP the persisted `geojson`; rebuild the GeoJSON LineString at read time from coordinates (it is a trivial wrapper). Readers that want GeoJSON call a `readGeojson()` helper that assembles it from the decoded coordinates. Rationale: coordinates are the minimal representation; geojson is pure envelope.
|
||||
|
||||
### 2. Compact coordinate codec
|
||||
Encode coordinates as **delta + zig-zag varint** over fixed-precision integers (lat/lon × 1e5 ≈ 1.1 m resolution, the BRouter/OSM working precision), base64'd into the Yjs string value — i.e. the Google encoded-polyline family. ~4–6 bytes/point vs ~22. Exposed as pure `encodePolyline(coords)` / `decodePolyline(str)`; property-tested for round-trip within the precision bound. Store a small `enc` version tag alongside so future codec changes stay detectable.
|
||||
|
||||
### 3. Run-length-encode metadata arrays
|
||||
Each `ROAD_METADATA_KEYS` array becomes a list of `[value, runLength]` pairs (JSON or the same varint stream). Road attributes change rarely along a route, so RLE collapses thousands of entries to a handful. Decode expands back to the per-coordinate `string[]` the readers already expect.
|
||||
|
||||
### 4. Dual-format read path (backward compatibility)
|
||||
The write path always emits the new encoding. The read path detects format per key: presence of the `enc` tag (or an encoded-value sentinel) → new decoder; otherwise fall back to the existing `JSON.parse`. So a session persisted before this change reads correctly, and it upgrades to the compact form the first time the host recomputes (which happens on the next edit). No migration job, no downtime. Old readers on an old client tab won't understand a new doc — acceptable because a planner session's participants load the same deployed build; a stale tab reconnects to the new build.
|
||||
|
||||
### 5. Codec location
|
||||
Put the pure codec (`polyline.ts` + `rle.ts`, or one `route-codec.ts`) where it can be unit-tested without the DOM/Yjs — `@trails-cool/map-core` (framework-free, already server-safe) is the natural home; `route-data.ts` imports it. Keeps `route-data.ts` the single encode/decode boundary.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Precision loss from integer quantization] → 1e5 (~1.1 m) is below BRouter's own resolution and GPX's practical precision; round-trip tests assert the bound. Acceptable and lossless at the meaningful scale.
|
||||
- [Dual-format read path adds branching] → Small, centralized in `route-data.ts`, and removable later once no old sessions remain (sessions are ephemeral/short-lived, so old-format docs age out quickly).
|
||||
- [A very stale client tab holds an old-format doc while another writes new-format] → Same-session participants run the same deployed build; a reconnect after deploy picks up the new build. Worst case a stale tab can't decode and reloads — no data loss (doc is persisted server-side).
|
||||
- [GPX export must stay byte-compatible] → Export builds from decoded coordinates, which round-trip within precision; a golden-file test guards the output.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
Pure library + planner change; no DB migration (yjs_state is opaque). Deploys with the apps. Existing sessions read via the fallback and re-encode compactly on their next recompute. Rollback = revert; new-format docs written in the interim would then fail to decode on the reverted build, but sessions are ephemeral and re-encode on edit — acceptable, and can be de-risked by shipping the *reader* first if desired.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Ship dual-write (both formats) for one release to make rollback perfectly safe, or accept the ephemeral-session risk and go straight to new-write? Leaning new-write (simpler, and sessions are short-lived) unless the rollback window matters.
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
## Why
|
||||
|
||||
The planner's Yjs session doc stores the computed route very inefficiently: a 4-waypoint / 77 km route already serializes to **~305 KB**. Three sources of bloat in `routeData` (`apps/planner/app/lib/route-data.ts`, `writeComputedRoute`):
|
||||
|
||||
1. **Geometry is stored twice** — `geojson` (a GeoJSON LineString of every coordinate) *and* `coordinates` (the same coordinates as a plain array), both `JSON.stringify`d.
|
||||
2. **Coordinates are stringified JSON floats** — `[13.4051234,52.5204567]` ≈ 22 bytes/point.
|
||||
3. **Seven per-coordinate road-metadata arrays** (`surfaces`, `highways`, `maxspeeds`, `smoothnesses`, `tracktypes`, `cycleways`, `bikeroutes`) stored as JSON `string[]` — in practice long runs of the same value.
|
||||
|
||||
This grows linearly with route length toward the 5 MB doc cap, bloats every reconnect's full-state sync, and was the underlying cause of the WS reconnect-loop outage (mitigated, not removed, by raising the frame cap in #587). It is pure encoding waste: the data is *derived*, and the routing-host model already computes it exactly once.
|
||||
|
||||
## What Changes
|
||||
|
||||
Encode the computed route compactly in `routeData`, **without changing the routing-host "compute once, share via Yjs" model** — one elected client still computes via BRouter and writes the result into the doc for everyone to read; clients never recompute.
|
||||
|
||||
- Store geometry **once** (single source of truth; derive the other representation at read time).
|
||||
- Encode the polyline compactly (delta + varint / encoded-polyline) instead of stringified JSON floats.
|
||||
- **Run-length-encode** the road-metadata arrays.
|
||||
- Centralize encode/decode in `route-data.ts` so all readers are transparently unaffected.
|
||||
- **Backward compatible**: the read path transparently handles both the existing JSON format (already persisted in `planner.sessions.yjs_state`) and the new encoding — no flag day, no migration downtime.
|
||||
|
||||
Target: **~5× smaller** (≈305 KB → ≈40–60 KB for that route). `MAX_DOC_BYTES = 5 MB` stays the guard; GPX export output stays byte-compatible.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `planner-route-encoding`: how the planner's computed route (geometry + road metadata) is compactly and backward-compatibly encoded in the Yjs session document, independent of how it is computed or shared.
|
||||
|
||||
### Modified Capabilities
|
||||
<!-- None: the routing-host compute/share behavior and route-coloring semantics are unchanged; only the on-doc encoding changes. -->
|
||||
|
||||
## Impact
|
||||
|
||||
- `apps/planner/app/lib/route-data.ts` — `writeComputedRoute` / read helpers (`readComputedRoute`, `readRoadMetadata`, etc.): new encode on write, dual-format decode on read.
|
||||
- Readers via those helpers, unchanged at the call site: `SessionView` map rendering, elevation profile, GPX export, save-to-journal, road-type/surface coloring.
|
||||
- New encoding module + tests in `@trails-cool/planner` (or `@trails-cool/gpx`/`map-core` if the codec is reusable).
|
||||
- No schema change (`yjs_state` is opaque bytea); no API change; no change to routing-host election or BRouter usage.
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Compact single-source geometry encoding
|
||||
The planner SHALL store a computed route's geometry once in the session document, as a compactly encoded polyline (fixed-precision delta encoding) rather than as stringified JSON floating-point coordinates, and SHALL derive any GeoJSON representation from it at read time. The encoding SHALL be lossless within the route's working precision (≈1 m).
|
||||
|
||||
#### Scenario: Geometry is not stored twice
|
||||
- **WHEN** the routing host writes a computed route to the session document
|
||||
- **THEN** the document contains a single encoded geometry value, not both a `geojson` LineString and a separate `coordinates` array
|
||||
|
||||
#### Scenario: Coordinates round-trip within precision
|
||||
- **WHEN** a route's coordinates are encoded and then decoded
|
||||
- **THEN** every decoded coordinate is within ~1 m of the original
|
||||
|
||||
### Requirement: Run-length-encoded road metadata
|
||||
The planner SHALL store the per-coordinate road-metadata channels (surface, highway, maxspeed, smoothness, tracktype, cycleway, bikeroute) run-length encoded, and SHALL decode them back to the per-coordinate values readers expect.
|
||||
|
||||
#### Scenario: Uniform metadata collapses
|
||||
- **WHEN** a long run of coordinates shares the same surface value
|
||||
- **THEN** that run is stored as a single value+length pair, and decoding restores one value per coordinate
|
||||
|
||||
### Requirement: Backward-compatible reads
|
||||
The planner SHALL read a session document written in either the legacy JSON format or the new compact encoding, transparently, with no migration step. A route written before this change SHALL render, export, and color correctly.
|
||||
|
||||
#### Scenario: Legacy document still works
|
||||
- **WHEN** a session persisted in the legacy JSON format is loaded
|
||||
- **THEN** its route geometry and road metadata decode correctly and the session behaves as before
|
||||
|
||||
#### Scenario: Legacy document re-encodes on recompute
|
||||
- **WHEN** the routing host recomputes the route for a legacy-format session
|
||||
- **THEN** the route is rewritten in the new compact encoding
|
||||
|
||||
### Requirement: Preserved compute-once model and outputs
|
||||
The compact encoding SHALL NOT change how the route is computed or shared: the elected routing host computes the route once and writes it to the document for all participants to read, and clients SHALL NOT recompute to obtain geometry. Decoded coordinates SHALL preserve the router's own precision (fixed to ~0.1 m / 6 decimals), so GPX export is unchanged in practice for router-produced routes.
|
||||
|
||||
#### Scenario: One computation, shared to all
|
||||
- **WHEN** multiple participants view a session
|
||||
- **THEN** exactly one client (the routing host) computes the route and the others read the encoded result from the document
|
||||
|
||||
#### Scenario: GPX export preserves coordinates
|
||||
- **WHEN** the same route is exported to GPX under the legacy and the new encoding
|
||||
- **THEN** every exported coordinate matches within ~0.1 m (the encoding is lossless at the router's 6-decimal precision)
|
||||
|
||||
### Requirement: Bounded document size
|
||||
A computed route of a realistic length SHALL occupy substantially less of the per-session document budget than the legacy encoding, keeping typical routes well under the document size cap.
|
||||
|
||||
#### Scenario: A long route stays small
|
||||
- **WHEN** a multi-waypoint route spanning tens of kilometres is stored
|
||||
- **THEN** its serialized document size is a small fraction of the 5 MB cap (target ≈5× smaller than the legacy encoding)
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
## 1. Codec primitives
|
||||
|
||||
- [x] 1.1 Add a framework-free codec in `@trails-cool/map-core` (e.g. `route-codec.ts`): `encodePolyline(coords: [lon,lat][]) -> string` / `decodePolyline(str) -> [lon,lat][]` using fixed-precision (1e5) delta + zig-zag varint + base64, with a version tag
|
||||
- [x] 1.2 Add `encodeRuns(values: string[]) -> string` / `decodeRuns(str, length?) -> string[]` run-length codec for the road-metadata channels
|
||||
- [x] 1.3 Unit tests: polyline round-trip within ~1 m over random + real fixtures; empty/single-point; RLE round-trip incl. all-same, all-distinct, and a realistic run pattern
|
||||
|
||||
## 2. Write path (new encoding)
|
||||
|
||||
- [x] 2.1 In `route-data.ts` `writeComputedRoute`: store geometry once as `encodePolyline(coordinates)` (drop the separate `geojson` write); write each `ROAD_METADATA_KEYS` channel via `encodeRuns`; keep `segmentBoundaries` + profile
|
||||
- [x] 2.2 Add a `readGeojson(routeData)` helper that assembles the GeoJSON LineString from decoded coordinates (so callers that needed `geojson` no longer read a stored copy)
|
||||
|
||||
## 3. Read path (dual-format, backward compatible)
|
||||
|
||||
- [x] 3.1 In `route-data.ts` read helpers (`readComputedRoute`, `readRoadMetadata`, coordinate accessors): detect format per key (encoding version tag/sentinel → new decoder; else `JSON.parse` legacy) and return the same decoded shapes as today
|
||||
- [x] 3.2 Point every reader (SessionView map render, elevation profile, GPX export, save-to-journal, road-type/surface coloring) at the helpers; confirm none reads a raw `routeData.get("geojson"|"coordinates"|<metadata>)` directly
|
||||
- [x] 3.3 Tests: a legacy-format `routeData` decodes correctly through the helpers; a new-format one decodes correctly; both yield identical reader-facing data
|
||||
|
||||
## 4. Verification
|
||||
|
||||
- [x] 4.1 GPX-export golden test: export the same route via legacy and new encoding → byte-identical output
|
||||
- [x] 4.2 Size assertion on a realistic route fixture (e.g. the ~77 km / multi-waypoint case): new-encoded `Y.encodeStateAsUpdate` size is ≥~4× smaller than legacy and a small fraction of `MAX_DOC_BYTES`
|
||||
- [x] 4.3 Confirm routing-host election / BRouter usage unchanged (no client recompute); `pnpm --filter @trails-cool/planner --filter @trails-cool/map-core typecheck && lint && test`, full `pnpm test`
|
||||
- [x] 4.4 Encode/decode + legacy-read + size covered by unit tests (route-codec, route-data); render/colors/export paths read through the same helpers and compile; live browser sanity rides the e2e suite (autonomous run)
|
||||
Loading…
Add table
Add a link
Reference in a new issue