Task group 4 of federation-hardening.
4.1 — FEDERATION.md at the repo root: actor discovery (WebFinger, actor,
NodeInfo), object/activity types with real JSON examples (Note, Create,
Delete, the narrow follow-graph inbox), addressing, HTTP-Signature
expectations, the two-layer dedup contract, durable delivery/retry
policy, and blocklist moderation semantics — precise enough for another
implementation to interoperate. Linked from README and docs/architecture.
4.2 — three prom-client metrics + a journal dashboard row:
- `federation_delivery_total{outcome}` — incremented in deliver-activity
(delivered/skipped/failed).
- `federation_inbox_dropped_total{reason}` — incremented at every inbox
drop (duplicate | blocked); this is the counter deferred from task 3.2.
- `federation_queue_depth` — gauge sampled at scrape time in
/api/metrics from PgBossMessageQueue.getDepth(); the restart-loss
regression detector.
Grafana journal.json gains a Federation row (delivery rate, queue depth,
inbox drops); the logs panels shift down to make room.
Verified: dashboard JSON valid; journal typecheck + lint clean; unit
suite 357 passing (route-template guard unaffected).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task group 3 of federation-hardening. There was no blocklist of any kind;
the only lever against a hostile instance was an IP/host block in Caddy.
- New `federation_blocked_instances` table (domain PK, reason,
created_at). Additive → created by drizzle-kit push.
- `federation-blocklist.server.ts`: exact-host matching —
`isBlockedDomain`, `isBlockedIri` (unparseable IRI ⇒ treated as
blocked), and `filterBlockedDomains` for batch recipient filtering.
- Enforced at all three boundaries (spec: federation-operations
"Instance blocklist"):
- inbox — each of the 4 listeners silently drops a blocked actor's
activity (202, no error oracle) before dedup/side effects;
- delivery enqueue — `enqueueActivityDeliveries` filters blocked
recipients in one batch query;
- outbox poll / actor fetch — `pollRemoteActor` refuses a blocked host
up front (`skipped: "blocked instance"`), before any network.
- Operator procedure (SQL insert/list/delete) documented in the
deployment runbook's federation section.
- Tests: unit (hostOfIri) + integration against real Postgres covering
the helper and the delivery + outbox boundaries; inbox uses the same
tested isBlockedIri primitive.
Note: the inbox-drop *counter* (federation_inbox_dropped_total{reason})
lands with the other metrics in task 4.2; this commit is the enforcement.
Verified: db + journal typecheck + lint clean; drizzle-kit push creates
the table; blocklist integration tests green against real Postgres;
journal unit suite 357 passing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task group 2 of federation-hardening. The narrow inbox
(Follow/Undo/Accept/Reject) had no replay protection — only Create(Note)
did, via the activities.remote_origin_iri unique constraint. A remote
redelivering a signed follow-graph activity would re-run its side
effects.
- New `federation_processed_activities` table (activity IRI PK,
received_at + index). Additive, so drizzle-kit push creates it; no
hand-written migration needed.
- `federation-replay.server.ts`: `markInboundActivityProcessed` does an
insert-or-drop (ON CONFLICT DO NOTHING RETURNING) and reports whether
the IRI is fresh; `sweepProcessedActivities` deletes rows > 30 days old
(signature date-freshness already rejects older replays).
- Each inbox listener drops a duplicate before side effects. The
follow-graph handlers are idempotent, so a handler failure whose retry
is later dropped as a duplicate can't corrupt state.
- `federation-dedup-sweep` job (daily 04:30 UTC) runs the TTL sweep.
Verified: db + journal typecheck + lint clean; drizzle-kit push creates
the table; replay integration test (fresh-vs-duplicate + 30-day sweep)
green against real Postgres; journal unit suite 355 passing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task group 1 of federation-hardening. Fedify was configured with
`InProcessMessageQueue`, so every queued outbound delivery, its pending
retry state, and inbox processing task was lost on a container restart
(routine here: deploys, OOM history) — directly contradicting the
social-federation spec's promise that fan-out survives a deploy.
- `federation-queue.server.ts`: `PgBossMessageQueue` implementing Fedify's
`MessageQueue` over the pg-boss instance the journal already runs,
mirroring the `PostgresKvStore` adapter. `nativeRetrial = false` keeps
Fedify the retry-policy owner; pg-boss supplies durability + delayed
jobs (delay → whole-second `startAfter`, `retryLimit: 0`).
- Swap it in for `InProcessMessageQueue` in `federation.server.ts`;
document the two intentional queueing layers (our fan-out jobs feed
Fedify; Fedify's sends now durable underneath).
- `server.ts`: create the durable queue at startup when federation is on.
- Broaden the structural `BossLike` in `boss.server.ts` with the
work/offWork/createQueue/getQueue methods the adapter needs.
- Tests: enqueue maps retry/delay correctly, consume roundtrip, restart
durability (fresh listener drains a prior instance's backlog), abort
stops the worker, depth reports ready vs delayed.
Verified: journal typecheck + lint clean, 7/7 new unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The route-surface-breakdown change is fully implemented (18/18 tasks,
all artifacts done). Archive it via `openspec archive` (CLI 1.6.0):
- Moves the change to openspec/changes/archive/2026-07-13-route-surface-breakdown/
- Promotes its 5 delta requirements into a new capability spec at
openspec/specs/route-surface-breakdown/spec.md (surface/waytype
breakdown from BRouter waytags, async Overpass backfill, live SSE
update, proportion bars, localized labels)
Purpose paragraph filled in (the CLI leaves a TBD placeholder). Both the
spec and archived change pass `openspec validate --strict`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The workflow was renamed "Dependabot dedupe" -> "Dependabot auto-fix" in
the previous commit; rename the file to match and fix the self-reference
in its error message. Safe — this workflow is not a required status check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the dependabot auto-fix workflow (formerly "Dependabot dedupe")
to also run `openspec update --force`, so a @fission-ai/openspec bump
regenerates the generated agent skills (.agents/skills/openspec-*) and
opsx slash commands (.claude/commands/opsx/*) and pushes them back to the
PR branch — the manual step that PR #567 had to do by hand for 1.2.0 ->
1.6.0.
Kept as a single workflow (one checkout/commit/push) so the dedupe and
openspec fixups don't race to push the same branch. The commit message
names only the parts that actually changed; no-op when neither applies.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Regenerated the OpenSpec agent skills (.agents/skills/openspec-*) and
opsx slash commands (.claude/commands/opsx/*) from openspec CLI 1.6.0
(was 1.2.0). Adds `allowed-tools` frontmatter, multi-store selection
support (`--store <id>`), and new status fields (planningHome,
changeRoot, artifactPaths, actionContext).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All 12 open Dependabot alerts are transitive dependencies, so resolve
them with pnpm.overrides (the pattern already used in this repo for
picomatch, brace-expansion, path-to-regexp, lodash).
- shell-quote <1.8.4 -> 1.8.4 (critical, #22; react-devtools)
- undici 7.x <7.28.0 -> 7.28.0 (high/med/low ×6, #30-36; jsdom/vitest)
- form-data <4.0.6 -> 4.0.6 (high, #28; jest-expo)
- js-yaml <3.15.0 -> 3.15.0 (medium, #37; jest istanbul)
- esbuild 0.27.x -> 0.28.1 (low, #23; vite/tsx)
- @opentelemetry/core <2.8.0 -> 2.9.0 (medium, #25; fedify runtime)
- uuid 7.0.3 -> 11.1.1 (medium, #19; xcode/expo prebuild)
Notes:
- undici override is scoped to 7.x so fedify's undici@6.27.0 (not in any
vulnerable range) is left untouched.
- @opentelemetry/core pinned to 2.9.0 (not the minimum 2.8.0) to dedupe
with the copy Sentry already pulls in, avoiding an otel version skew.
- uuid pinned narrowly to the vulnerable 7.0.3; xcode calls
require('uuid').v4(), which still works under v11 (verified).
Verified: pnpm install / build / typecheck / test all pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumps 35 dependencies in the production group; lockfile regenerated and deduped against latest main.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
poi-index shipped and is live in production (self-hosted planet POI index,
8.4M rows serving /api/pois; Overpass removed from the Planner). Archive the
change and apply its spec deltas:
- new capability: poi-index
- osm-poi-overlays: POIs from the instance index, not Overpass
- rate-limiting: /api/pois limit replaces the Overpass proxy limit
- infrastructure: POI extract (BRouter host) + import (flagship) components
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The import hit a PK violation on (osm_type, osm_id, category): an OSM element
that matches one category through two selectors on different keys (e.g. shelter
= amenity=shelter OR tourism=wilderness_hut, both present) produced two rows for
that category. Add DISTINCT ON (osm_type, osm_id, category) so classification is
one row per (element, category), matching matchingCategoryIds semantics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
poi-import.sh ran the classifier selectors via psql \i, but psql executes
inside the postgres container (docker compose exec), so it couldn't find the
host path /opt/trails-cool/scripts/poi-selectors.sql. Concatenate BEGIN + the
selectors file contents + the build statements on the host and pipe them into
psql stdin instead, keeping the ON COMMIT DROP temp table in one transaction.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove ullrich.is / 1.8 TB / hardcoded 'trails' username assumptions from the
poi-extract README and service unit; refer to 'the pipeline user' and $USER /
%h instead so the docs serve self-hosters too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The BRouter host's trails user is non-root without sudo, so osmium-tool can't
be apt-installed. Ship osmium.Dockerfile (built on first use) and run osmium in
a container with the work dir bind-mounted (native I/O, negligible overhead on
Linux). python3/curl/gzip/sha256sum remain on the host. Correct the README
disk figure to the /home free space (~595 GB).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cd-brouter.yml bundles an explicit file list; the new poi-extract dir was
missing, so the extract scripts would never land on the host. Add it (+ a
defensive chmod).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- planner dashboard: replace Overpass panels with POI index freshness +
serving panels (age, rows/category, import status, API request rate/errors)
- alerts: replace overpass-upstream-unhealthy with poi-index-stale (>6 weeks)
- architecture.md: POI data flow now the self-hosted index
- privacy manifest (DE+EN): Overpass is Journal-surface-backfill-only;
Planner POIs served same-origin from /api/pois
- self-host-overpass README + roadmap: superseded for POIs by poi-index
- poi-extract README: self-hoster story (optional pipeline, regional extract,
graceful empty index)
- map-core tsconfig: exclude *.sync.test.ts from tsc to keep it zero-dep
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- poi-extract.sh: PBF download -> osmium tags-filter -> centroid NDJSON +
manifest, published via the Caddy sidecar at vSwitch-only /poi/*
- osmium-filters.txt + poi-selectors.sql generated from map-core selectors,
guarded by sync tests so poiCategories stays the single source of truth
- poi-import.sh: checksum verify -> classify into category rows ->
70% guard (--bootstrap override) -> atomic rename swap; node_exporter
textfile metric for import outcome
- systemd service+timer units for both halves (monthly, offset)
- cd-infra.yml: ship infrastructure/scripts to the flagship
- e2e: mock /api/pois instead of /api/overpass
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the planner's Overpass proxy with a self-hosted POI index:
- map-core POI categories become structured tag selectors (single source of
truth) + osmium filter / classification helpers
- planner.pois PostGIS table (centroid points, GiST + category indexes)
- /api/pois serving route (session + same-origin + rate limit, 100 cap)
- lib/pois.ts client (renamed from overpass.ts; GET, quantized bbox)
- metrics: poi_api_requests_total + DB-collected index rows/age gauges;
drop overpass_* metrics
- remove /api/overpass proxy + dead OVERPASS_URLS on the planner service
(Journal surface backfill still uses it via code default)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Docs + OpenSpec proposals only (no app/infra code); merged directly to
main via admin override — no CI-relevant surface, no deploy triggers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add docs/inspirations.md as the durable record of the 2026-07-05/06
prior-art research — per-project learnings with source paths, canonical
credit lines, and the changes each spawned — and extend the
acknowledgment lists in philosophy.md/architecture.md (Organic Maps,
Endurain, wanderer).
New OpenSpec changes (proposal/design/specs/tasks each):
- Organic Maps: elevation-profile-hardening, gpx-parser-robustness,
hiking-time-estimate, poi-index, hiking-foot-profile
- Endurain: account-export, activity-duplicate-review,
fit-parsing-hardening, activity-locations, self-hosting-guide,
activity-privacy-controls
- wanderer: federation-hardening, link-share-tokens
- credits-page (user-visible acknowledgments)
Updated in-flight changes with wanderer prior-art sections:
route-federation, route-discovery.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nothing in CI booted the journal's production image. typecheck / lint /
test / build all run against the source tree (where every file is
present), and the e2e job boots the journal via `react-router-serve` —
not the production `node server.ts` entrypoint. The runtime stage of
apps/journal/Dockerfile copies source files in by name, so a refactor
that adds a file server.ts imports without a matching COPY builds green
everywhere and only crash-loops once deployed (ERR_MODULE_NOT_FOUND).
That has taken prod down more than once: app/lib (8631c8f), app/jobs
(e16bd6d), and serve-static.ts (#554/#555, today's outage).
Add a job that builds the `runtime` stage and actually boots it against
a throwaway Postgres, polling /api/health for a 200. A missing static
OR dynamic import never reaches a healthy boot, so this fails the PR
instead of the deploy. Verified locally: the fixed image goes healthy;
the image with the serve-static.ts COPY removed exits with the exact
ERR_MODULE_NOT_FOUND and the job fails.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #554 extracted serve-static.ts as a sibling of server.ts, but the
journal Dockerfile copies runtime source files explicitly by name and
the COPY list was never updated. The runtime runs `node server.ts`,
whose `import "./serve-static.ts"` then fails with ERR_MODULE_NOT_FOUND,
crash-looping both the production and staging journals (they share the
image). Caddy stayed up with no upstream, so the site hung.
Add the missing COPY line. serve-static.ts imports only node: builtins,
so this single file restores startup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
