openspec: archive poi-index; sync deltas into canonical specs

poi-index shipped and is live in production (self-hosted planet POI index,
8.4M rows serving /api/pois; Overpass removed from the Planner). Archive the
change and apply its spec deltas:

- new capability: poi-index
- osm-poi-overlays: POIs from the instance index, not Overpass
- rate-limiting: /api/pois limit replaces the Overpass proxy limit
- infrastructure: POI extract (BRouter host) + import (flagship) components

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-07-13 01:46:13 +02:00
parent 5539b34613
commit 8396041c26
No known key found for this signature in database
GPG key ID: A32FF691A0F752D9
12 changed files with 82 additions and 15 deletions

View file

@ -192,3 +192,22 @@ The BRouter host SHALL be administered by the trails.cool project through a non-
- **WHEN** files are created by the deploy or segment-download scripts
- **THEN** they live under `~trails/brouter/` and are owned by `trails:trails`
### Requirement: POI extract pipeline on the BRouter host
The BRouter host SHALL run a scheduled job (monthly by default) that downloads an OSM data file, filters it to the planner's POI category selectors with osmium, and publishes the resulting artifact with a checksum-and-timestamp manifest at a vSwitch-only URL via the existing Caddy sidecar. Transient working data SHALL live under the `trails` user's directory and be cleaned up after each run.
#### Scenario: Monthly extract published
- **WHEN** the scheduled extract job completes
- **THEN** a fresh artifact and manifest are available to the flagship over the vSwitch
- **AND** the planet working files are removed from disk
#### Scenario: Artifact not publicly reachable
- **WHEN** a host outside the vSwitch requests the artifact URL
- **THEN** the request is refused
### Requirement: POI import job on the flagship
The flagship SHALL run a scheduled import job that fetches the artifact over the vSwitch, verifies its checksum, loads it into the staging table, and performs the guarded atomic swap. Job outcome and index age SHALL be visible in the monitoring stack.
#### Scenario: Import failure is visible
- **WHEN** an import run fails (fetch, checksum, load, or guard)
- **THEN** the failure is recorded in metrics/logs and surfaces on the Grafana planner dashboard

View file

@ -16,11 +16,11 @@ The Planner SHALL provide a collapsible panel for toggling POI categories on the
- **THEN** the panel collapses and POI markers remain visible on the map
### Requirement: POI categories
The Planner SHALL support the following POI categories queried from OpenStreetMap via Overpass API: drinking water, shelter, camping, food & drink, groceries, bike infrastructure, accommodation, viewpoints, and toilets.
The Planner SHALL support the following POI categories served from the instance's own POI index (`/api/pois`): drinking water, shelter, camping, food & drink, groceries, bike infrastructure, accommodation, viewpoints, and toilets.
#### Scenario: Enable a POI category
- **WHEN** a user enables the "Drinking water" category in the POI panel
- **THEN** drinking water POIs within the current map viewport are fetched from Overpass and rendered as markers
- **THEN** drinking water POIs within the current map viewport are fetched from the instance's POI index and rendered as markers
#### Scenario: Disable a POI category
- **WHEN** a user disables a previously enabled POI category
@ -58,19 +58,17 @@ The Planner SHALL load POIs only within the current map viewport, refreshing whe
#### Scenario: Cached results
- **WHEN** the user pans back to a previously viewed area within 10 minutes
- **THEN** cached POI results are displayed without a new Overpass query
- **THEN** cached POI results are displayed without a new POI request
### Requirement: Overpass rate limit handling
The Planner SHALL handle Overpass API rate limits gracefully.
### Requirement: POI service degradation handling
The Planner SHALL handle POI endpoint failures gracefully.
#### Scenario: Rate limited response
- **WHEN** the Overpass API returns a 429 status
- **WHEN** `/api/pois` returns a 429 status
- **THEN** the Planner shows a temporary "POI data unavailable — try again shortly" message and sets a backoff delay before the next request
Note: automatic retry is not implemented. The next request fires only on the next user viewport change or category toggle after the backoff delay has elapsed.
#### Scenario: Overpass unavailable
- **WHEN** the Overpass API is unreachable
#### Scenario: POI service unavailable
- **WHEN** `/api/pois` is unreachable or returns a server error
- **THEN** the Planner shows a message and tile overlays continue to function normally
### Requirement: Profile-aware POI defaults

View file

@ -0,0 +1,49 @@
## Purpose
A self-hosted POI index that serves OpenStreetMap points of interest for the Planner's POI categories from the instance's own database, replacing the third-party Overpass API. The index is refreshed on a schedule via an OSM extract pipeline and served through a same-origin, rate-limited endpoint with no third-party dependency at request time.
## Requirements
### Requirement: POI index table
The instance SHALL maintain a `planner.pois` table of OSM points of interest for the planner's POI categories, storing OSM type and id, category, optional name, centroid point geometry, the element's tags, and import timestamp, indexed for bbox-and-category lookup. Category membership SHALL be derived from the structured tag selectors defined in `@trails-cool/map-core`, which are the single source of truth for the pipeline filter and import-time classification.
#### Scenario: Bbox query by category
- **WHEN** the serving layer queries the index for "drinking water" within a viewport bbox
- **THEN** all indexed elements whose tags match that category's selectors and whose centroid lies in the bbox are returned
#### Scenario: Way POIs appear as points
- **WHEN** an OSM way (e.g. a campground area) matches a category selector
- **THEN** it is indexed with its centroid as the point geometry
### Requirement: Periodic import with atomic swap
The instance SHALL refresh the POI index on a schedule (monthly by default) by filtering an OSM data file to the category selectors, loading the result into a staging table, and swapping it in atomically so serving is never interrupted. The import SHALL abort without swapping when the staging row count falls below 70% of the live table's row count, unless bootstrap mode is explicitly enabled.
#### Scenario: Refresh replaces data without downtime
- **WHEN** a scheduled import completes successfully
- **THEN** the live table reflects the new dataset and no serving request observed an empty or partial table
#### Scenario: Shrunken dataset refused
- **WHEN** an import produces fewer than 70% of the live table's rows (e.g. truncated download)
- **THEN** no swap occurs, the live data stays untouched, and the failure is logged and visible in metrics
#### Scenario: Regional extract for self-hosters
- **WHEN** an operator configures a regional extract URL instead of the planet file
- **THEN** the same pipeline imports that region only
### Requirement: POI serving endpoint
The Planner SHALL serve POIs from the index via `/api/pois`, accepting a bbox and category list, enforcing the same-origin and session checks and the per-IP rate limit that the Overpass proxy enforced, and capping results at 100 per request. The response SHALL provide the fields the client's `Poi` shape requires (id, coordinates, name, category, tags). POI requests SHALL NOT contact any third-party service.
#### Scenario: Happy-path query
- **WHEN** the browser requests two enabled categories for the current viewport
- **THEN** the endpoint returns matching POIs from the local index with their tags
#### Scenario: Missing index degrades gracefully
- **WHEN** the POI table is empty or absent (fresh self-hosted instance)
- **THEN** the endpoint returns an empty result or a service-unavailable status without errors that break the map
### Requirement: Index freshness observability
The instance SHALL expose metrics for index size per category, index age, import job outcome, and serving request counts, and the monitoring stack SHALL alert when the index age indicates a missed refresh.
#### Scenario: Stale index alerts
- **WHEN** the index age exceeds the alert threshold (~6 weeks)
- **THEN** an alert fires identifying the POI import as unhealthy

View file

@ -18,9 +18,10 @@ The Planner SHALL limit route computations to 300 per session per hour (the `DEF
- **WHEN** a session exceeds 300 BRouter calls in one hour
- **THEN** the server responds with 429 and the client shows a "slow down" message
### Requirement: Overpass API rate limit
The Planner SHALL limit Overpass API calls to 120 per IP per minute to protect the upstream service.
### Requirement: POI API rate limit
The Planner SHALL limit `/api/pois` requests to 120 per IP per minute to protect the instance's database from abusive clients.
#### Scenario: Overpass rate limit exceeded
- **WHEN** a single IP exceeds 120 Overpass requests in one minute
- **THEN** the `/api/overpass` proxy responds with 429 Too Many Requests
#### Scenario: POI rate limit exceeded
- **WHEN** a single IP exceeds 120 POI requests in one minute
- **THEN** `/api/pois` responds with 429 Too Many Requests
- **AND** no database query is executed for rejected requests