All 13 tasks complete (implementation shipped in #570–573; staging
verification in #574). Archive via `openspec archive` (CLI 1.6.0):
- Creates openspec/specs/federation-operations/spec.md — the new
capability: durable federation queue, inbound replay defense, instance
blocklist, published protocol doc, delivery observability (Purpose
filled in; CLI leaves a TBD placeholder).
- Applies the MODIFIED requirement to openspec/specs/social-federation:
"Push delivery on local activity create" now guarantees persistent
queueing + a "Fan-out survives a deploy" scenario.
- Moves the change to openspec/changes/archive/2026-07-13-federation-hardening/.
Both specs pass `openspec validate --strict`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ran the post-deploy staging check on staging.trails.cool:
- Durability: enqueued a real delivery, restarted the journal mid-flight;
the job survived the restart (still `created` in Postgres afterward)
and completed after — "Successfully sent activity … to
social.ullrich.is/users/ullrich/inbox", federation_delivery_total
{outcome="delivered"}=1, queue drained to 0. The old in-process queue
would have dropped it.
- Blocklist outbound: a poll-remote-actor for a blocked domain returned
{skipped: "blocked instance"} with no network fetch.
- Operator block/unblock procedure works against the live staging DB.
Inbound-drop leg is covered by the group-2/3 real-Postgres integration
tests + the two-instance e2e harness (firing it live needs a
peer-initiated signed request). All 13 tasks now complete.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5.2 typecheck/lint/test pass locally; test:e2e is enforced by the
required "E2E Tests" CI check. 5.1 is a post-deploy staging check
(fan-out survives a restart; a blocked domain is inert both ways) —
left open with the runbook commands in the PR body.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task group 4 of federation-hardening.
4.1 — FEDERATION.md at the repo root: actor discovery (WebFinger, actor,
NodeInfo), object/activity types with real JSON examples (Note, Create,
Delete, the narrow follow-graph inbox), addressing, HTTP-Signature
expectations, the two-layer dedup contract, durable delivery/retry
policy, and blocklist moderation semantics — precise enough for another
implementation to interoperate. Linked from README and docs/architecture.
4.2 — three prom-client metrics + a journal dashboard row:
- `federation_delivery_total{outcome}` — incremented in deliver-activity
(delivered/skipped/failed).
- `federation_inbox_dropped_total{reason}` — incremented at every inbox
drop (duplicate | blocked); this is the counter deferred from task 3.2.
- `federation_queue_depth` — gauge sampled at scrape time in
/api/metrics from PgBossMessageQueue.getDepth(); the restart-loss
regression detector.
Grafana journal.json gains a Federation row (delivery rate, queue depth,
inbox drops); the logs panels shift down to make room.
Verified: dashboard JSON valid; journal typecheck + lint clean; unit
suite 357 passing (route-template guard unaffected).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task group 3 of federation-hardening. There was no blocklist of any kind;
the only lever against a hostile instance was an IP/host block in Caddy.
- New `federation_blocked_instances` table (domain PK, reason,
created_at). Additive → created by drizzle-kit push.
- `federation-blocklist.server.ts`: exact-host matching —
`isBlockedDomain`, `isBlockedIri` (unparseable IRI ⇒ treated as
blocked), and `filterBlockedDomains` for batch recipient filtering.
- Enforced at all three boundaries (spec: federation-operations
"Instance blocklist"):
- inbox — each of the 4 listeners silently drops a blocked actor's
activity (202, no error oracle) before dedup/side effects;
- delivery enqueue — `enqueueActivityDeliveries` filters blocked
recipients in one batch query;
- outbox poll / actor fetch — `pollRemoteActor` refuses a blocked host
up front (`skipped: "blocked instance"`), before any network.
- Operator procedure (SQL insert/list/delete) documented in the
deployment runbook's federation section.
- Tests: unit (hostOfIri) + integration against real Postgres covering
the helper and the delivery + outbox boundaries; inbox uses the same
tested isBlockedIri primitive.
Note: the inbox-drop *counter* (federation_inbox_dropped_total{reason})
lands with the other metrics in task 4.2; this commit is the enforcement.
Verified: db + journal typecheck + lint clean; drizzle-kit push creates
the table; blocklist integration tests green against real Postgres;
journal unit suite 357 passing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task group 2 of federation-hardening. The narrow inbox
(Follow/Undo/Accept/Reject) had no replay protection — only Create(Note)
did, via the activities.remote_origin_iri unique constraint. A remote
redelivering a signed follow-graph activity would re-run its side
effects.
- New `federation_processed_activities` table (activity IRI PK,
received_at + index). Additive, so drizzle-kit push creates it; no
hand-written migration needed.
- `federation-replay.server.ts`: `markInboundActivityProcessed` does an
insert-or-drop (ON CONFLICT DO NOTHING RETURNING) and reports whether
the IRI is fresh; `sweepProcessedActivities` deletes rows > 30 days old
(signature date-freshness already rejects older replays).
- Each inbox listener drops a duplicate before side effects. The
follow-graph handlers are idempotent, so a handler failure whose retry
is later dropped as a duplicate can't corrupt state.
- `federation-dedup-sweep` job (daily 04:30 UTC) runs the TTL sweep.
Verified: db + journal typecheck + lint clean; drizzle-kit push creates
the table; replay integration test (fresh-vs-duplicate + 30-day sweep)
green against real Postgres; journal unit suite 355 passing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task group 1 of federation-hardening. Fedify was configured with
`InProcessMessageQueue`, so every queued outbound delivery, its pending
retry state, and inbox processing task was lost on a container restart
(routine here: deploys, OOM history) — directly contradicting the
social-federation spec's promise that fan-out survives a deploy.
- `federation-queue.server.ts`: `PgBossMessageQueue` implementing Fedify's
`MessageQueue` over the pg-boss instance the journal already runs,
mirroring the `PostgresKvStore` adapter. `nativeRetrial = false` keeps
Fedify the retry-policy owner; pg-boss supplies durability + delayed
jobs (delay → whole-second `startAfter`, `retryLimit: 0`).
- Swap it in for `InProcessMessageQueue` in `federation.server.ts`;
document the two intentional queueing layers (our fan-out jobs feed
Fedify; Fedify's sends now durable underneath).
- `server.ts`: create the durable queue at startup when federation is on.
- Broaden the structural `BossLike` in `boss.server.ts` with the
work/offWork/createQueue/getQueue methods the adapter needs.
- Tests: enqueue maps retry/delay correctly, consume roundtrip, restart
durability (fresh listener drains a prior instance's backlog), abort
stops the worker, depth reports ready vs delayed.
Verified: journal typecheck + lint clean, 7/7 new unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The route-surface-breakdown change is fully implemented (18/18 tasks,
all artifacts done). Archive it via `openspec archive` (CLI 1.6.0):
- Moves the change to openspec/changes/archive/2026-07-13-route-surface-breakdown/
- Promotes its 5 delta requirements into a new capability spec at
openspec/specs/route-surface-breakdown/spec.md (surface/waytype
breakdown from BRouter waytags, async Overpass backfill, live SSE
update, proportion bars, localized labels)
Purpose paragraph filled in (the CLI leaves a TBD placeholder). Both the
spec and archived change pass `openspec validate --strict`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
poi-index shipped and is live in production (self-hosted planet POI index,
8.4M rows serving /api/pois; Overpass removed from the Planner). Archive the
change and apply its spec deltas:
- new capability: poi-index
- osm-poi-overlays: POIs from the instance index, not Overpass
- rate-limiting: /api/pois limit replaces the Overpass proxy limit
- infrastructure: POI extract (BRouter host) + import (flagship) components
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- planner dashboard: replace Overpass panels with POI index freshness +
serving panels (age, rows/category, import status, API request rate/errors)
- alerts: replace overpass-upstream-unhealthy with poi-index-stale (>6 weeks)
- architecture.md: POI data flow now the self-hosted index
- privacy manifest (DE+EN): Overpass is Journal-surface-backfill-only;
Planner POIs served same-origin from /api/pois
- self-host-overpass README + roadmap: superseded for POIs by poi-index
- poi-extract README: self-hoster story (optional pipeline, regional extract,
graceful empty index)
- map-core tsconfig: exclude *.sync.test.ts from tsc to keep it zero-dep
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- poi-extract.sh: PBF download -> osmium tags-filter -> centroid NDJSON +
manifest, published via the Caddy sidecar at vSwitch-only /poi/*
- osmium-filters.txt + poi-selectors.sql generated from map-core selectors,
guarded by sync tests so poiCategories stays the single source of truth
- poi-import.sh: checksum verify -> classify into category rows ->
70% guard (--bootstrap override) -> atomic rename swap; node_exporter
textfile metric for import outcome
- systemd service+timer units for both halves (monthly, offset)
- cd-infra.yml: ship infrastructure/scripts to the flagship
- e2e: mock /api/pois instead of /api/overpass
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the planner's Overpass proxy with a self-hosted POI index:
- map-core POI categories become structured tag selectors (single source of
truth) + osmium filter / classification helpers
- planner.pois PostGIS table (centroid points, GiST + category indexes)
- /api/pois serving route (session + same-origin + rate limit, 100 cap)
- lib/pois.ts client (renamed from overpass.ts; GET, quantized bbox)
- metrics: poi_api_requests_total + DB-collected index rows/age gauges;
drop overpass_* metrics
- remove /api/overpass proxy + dead OVERPASS_URLS on the planner service
(Journal surface backfill still uses it via code default)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add docs/inspirations.md as the durable record of the 2026-07-05/06
prior-art research — per-project learnings with source paths, canonical
credit lines, and the changes each spawned — and extend the
acknowledgment lists in philosophy.md/architecture.md (Organic Maps,
Endurain, wanderer).
New OpenSpec changes (proposal/design/specs/tasks each):
- Organic Maps: elevation-profile-hardening, gpx-parser-robustness,
hiking-time-estimate, poi-index, hiking-foot-profile
- Endurain: account-export, activity-duplicate-review,
fit-parsing-hardening, activity-locations, self-hosting-guide,
activity-privacy-controls
- wanderer: federation-hardening, link-share-tokens
- credits-page (user-visible acknowledgments)
Updated in-flight changes with wanderer prior-art sections:
route-federation, route-discovery.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers routes/activities that enter the journal without BRouter waytags
(imports, uploads, pre-existing rows):
- overpass-ways.server.ts: server-side Overpass client (way[highway] + geom in
a bbox, configurable OVERPASS_URLS, timeout + oversized-bbox guard).
- surface-match.server.ts: pure nearest-way map-matcher → per-segment
surface/highway (unmatched → unknown). Unit-tested.
- surface-backfill pg-boss job: load geom → skip if breakdown exists → Overpass
→ match → computeSurfaceBreakdown → store → emit `surface_breakdown` SSE to
the owner. Idempotent, retry-safe, best-effort; registered in server.ts.
- Enqueued from createActivity (imports/uploads) + owner-on-open in the route &
activity detail loaders (non-Planner routes, old rows), deduped via singletonKey.
- useSurfaceBackfillUpdates: detail pages subscribe to /api/events and
revalidate() when their row's backfill lands (live bars, no reload).
- Privacy manifest updated (DE + EN) for the Overpass bbox lookup.
Tests: matchSurfaces unit; surface-backfill job (mocked Overpass/db/events:
store + emit, skip-if-present, skip-if-no-ways). Verified the real
Overpass→match→breakdown pipeline against a live Berlin bbox (509 ways →
plausible asphalt/paving_stones/footway mix). typecheck + lint + unit
(journal 333) green.
Note: the worker runs via server.ts (prod/staging), not `react-router dev`, so
the live job + SSE exercise on a deployed instance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Synchronous path + rendering for the surface/waytype breakdown:
- map-core `computeSurfaceBreakdown(coords, surfaces, highways)` → distance-
weighted metres per surface + waytype category (unit-tested);
- `SurfaceBreakdownSchema` in @trails-cool/api;
- nullable `surfaceBreakdown` jsonb on routes + activities;
- Planner `SaveToJournalButton` computes it from the BRouter waytags already in
routeData and sends it; `api.save-to-journal` forwards it; the journal route
callback validates + persists (journal is the authoritative validator);
- `SurfaceBreakdown` component (stacked bars per dimension, map-core palettes,
legend category · % · km largest-first, unknown → "other", hidden when empty)
on route + activity detail; journal gains a @trails-cool/map-core dep;
- i18n journal.surface.* (en + de).
Phase 2 (async Overpass backfill + SSE for imports/uploads, and the e2e that
seeds a breakdown) follows in a separate PR.
Tests: computeSurfaceBreakdown unit (map-core 36); SurfaceBreakdown component
(jsdom, journal 326). typecheck + lint green; verified in the browser.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Builds on the merged A-only proposal. Broadens the capability:
- breakdown now on routes AND activities;
- Path 1 (sync) unchanged: BRouter waytags → breakdown at Planner save;
- Path 2 (async): a `surface-backfill` pg-boss job map-matches geometry to OSM
via Overpass for imports/uploads/older rows, stores the breakdown, and pushes
a `surface_breakdown` SSE event so an open detail page fills in live;
- privacy note: backfill sends geometry to Overpass → route via the proxy /
self-hostable Overpass; documented in the manifest.
Validates with `openspec validate --strict`. Specs only; no code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surface + waytype proportion bars on route detail, from the BRouter waytags the
Planner already computes (and currently discards on save). Design records the
data-source decision: persist a compact distance-weighted breakdown for
Planner-created routes (accurate, no external call); Overpass map-matching for
imports/uploads is parked as a future option.
Validates with `openspec validate --strict`. Specs only; no code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both changes are implemented and merged (#539, #541). Promote their deltas into
openspec/specs/, move the changes to changes/archive/2026-06-14-*, and add the
two capabilities to the CAPABILITIES index.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the profile-weekly-distance change (specs/profile-weekly-distance):
- getWeeklyDistance(ownerId, { publicOnly, weeks=12 }) in activities.server.ts:
one query that gap-fills in SQL — generate_series of week-starts LEFT JOINed
to activities — so it returns exactly 12 contiguous { weekStart, distance }
rows with matching Postgres week boundaries, viewer-scoped, no cache, no
schema change.
- WeeklyDistanceChart: SVG bars normalized to the busiest week (empty weeks keep
their slot as zero-height bars), per-bar title distance, localized label;
renders nothing when there's no distance in the window. Mounted under the
ProfileStats header.
- i18n journal.profileStats.weeklyDistance (en + de).
Tests: WeeklyDistanceChart component (jsdom: bar count incl. zero weeks, empty
→ hidden, normalization); e2e creates an activity with distance and asserts the
chart renders. typecheck + lint + unit (journal 321) green; verified in the
browser.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A 12-week distance bar chart under the profile stats header, viewer-scoped,
built on profile-stats (#539). Design records the same no-cache rationale: one
indexed grouped aggregate over a 12-week slice (≤12 rows), gap-filled in JS.
Validates with `openspec validate --strict`. Specs only; no code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the profile-stats change (specs/profile-stats):
- getActivityStats(ownerId, { publicOnly }) in activities.server.ts: one indexed
aggregate over stored columns (count, sum distance/ascent/elapsed duration) +
a rolling last-4-weeks count. No cache table, no schema change, no GPX parsing
(design §D1).
- Profile loader computes it scoped to the viewer (public-only for visitors,
full totals for the owner); ProfileStats header renders count · distance ·
ascent · time + "N in the last 4 weeks" via the shared StatRow + stats.ts
formatters; hidden when there are no visible activities.
- i18n journal.profileStats.* in en + de.
Tests: ProfileStats component (jsdom: totals, empty, last-4-weeks toggle);
e2e asserts the owner roll-up counts their activities. typecheck + lint + unit
(journal 318) green; verified in the browser.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lifetime totals (count · distance · ascent · elapsed time) + a last-4-weeks
count on the profile, viewer-scoped (public-only for visitors, full for the
owner).
Design records the cost decision: compute on the fly with a single aggregate
over stored columns on the owner_id-leading indexes (cheap even for power
users) — no cache table, no schema change — with a documented escape hatch to
a user_activity_stats cache if profiling ever shows it's hot.
Validates with `openspec validate --strict`. Specs only; no code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Archive the three implemented + merged changes (activity-sport-type,
activity-stats, journal-elevation-profile): their deltas are promoted into
openspec/specs/, the changes move to changes/archive/2026-06-12-*, and the
three new capabilities are added to the CAPABILITIES index.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the journal-elevation-profile change (specs/journal-elevation-profile):
- gpx: `elevationSeries(tracks)` → { d, e, lat, lng }[] with cumulative distance,
downsampled (keeps first/last), empty when <2 points carry elevation.
- ElevationProfile: read-only SVG area chart (vertical gradient fill), highest/
lowest summary + a hover readout. Reports hovered index (onActive) and clicked
index (onSeek); draws a marker at the active index. Renders nothing for an
empty series.
- RouteMapThumbnail: ActiveMarker (CircleMarker at the chart's active point),
HoverTracker (route hover → nearest sample → onHoverIndex), Recenter (panTo on
chart click). Props forwarded through ClientMap.
- Wired into the activity + route detail pages via a shared activeIndex/centerOn
state; loaders expose the series (activity reuses the moving-time parse).
- i18n journal.elevation.{highest,lowest} in en + de.
Ascent/descent stay in the stat row (#532); the chart summary shows highest/
lowest to avoid duplication.
Tests: elevationSeries unit; ElevationProfile component (jsdom); e2e creates an
activity from an elevation GPX and asserts the chart renders. typecheck + lint +
unit (gpx 67, journal 315) green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- e2e/activity-sport-type.test.ts: register → create activity with a sport →
assert the sport badge renders on the detail page. Passes locally against an
E2E=true server.
- playwright.config.ts: add the `activity-sport-type` project so the spec
actually runs (specs only execute if a project testMatch matches them).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the activity-sport-type change (specs/activity-sport-type):
- db: nullable `sport_type` column on journal.activities + SportType /
SPORT_TYPES (text().$type<> convention).
- api: optional sportType on the activity read + create schemas (mirrored
SPORT_TYPES; @trails-cool/api stays zod-only).
- write: ActivityInput + createActivity persist it; mapSportType() normalizes
provider strings (Komoot bulk import passes tour.sport; Garmin unset);
threaded through the unified importActivity.
- read/display: sportType added to the detail/feed/profile loaders and the v1
REST endpoints; shared SportBadge (glyph + i18n label) on detail, feed, and
profile; sport-aware feed verb; create-form <select>.
- i18n: journal.activities.sport.* (labels + verbs) in en + de.
- federation: `sport` PropertyValue on the Note when set.
Tests: mapSportType unit table; federation asserts the sport attachment is
present when set and omitted when unset. typecheck + lint + unit all green.
E2E (create→badge) still to add.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three OpenSpec proposals to bring the journal's route/activity detail and feed
closer to best-in-class outdoor apps, in the order we'll implement them:
- activity-sport-type: nullable sport enum on activities (hike/walk/run/
ride/gravel/mtb/ski/other), set on create or normalized on import,
shown as a badge + feed verb, federated as a Note PropertyValue.
- activity-stats: one reusable StatRow + canonical metric set, sport-aware
avg pace/speed, moving-vs-elapsed time. Derive-only, no schema change.
- journal-elevation-profile: elevation profile chart + chart<->map
"active distance" hover/click sync on the read-only detail pages.
All three validate with `openspec validate --strict`. Specs only; no code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Garmin Connect as the third connected-services provider (spec:
garmin-import). The interesting parts:
- Push-first ingestion: Garmin has no list endpoint. The webhook
normalizes ping (callbackURL) and push (inline) notification batches
into events; the slow work (authorized FIT download, FIT→GPX via the
shared converter, activity creation) runs in a garmin-import-activity
pg-boss job so the webhook answers fast. Callback URLs are validated
against Garmin's API host before any fetch (SSRF guard).
- History via backfill requests: /sync/import/garmin is a date-range
requester with honest async progress (no pick list — the concept
doesn't exist in a push model). Ranges chunk to Garmin's 90-day cap;
overlaps are free via sync_imports dedupe. Requests persist in
import_batches via two new nullable columns (range_start/range_end).
- OAuth2 + PKCE on the existing oauth credential kind. Design
correction from apply: the verifier rides a short-lived httpOnly
cookie scoped to the callback path — the state param is visible in
redirect URLs and must never carry it. Manifests opt in via pkce:true.
- Deregistration notifications flip the connection to 'revoked'
(row kept for audit, imports retained, re-connect prompt shown).
- Framework evolutions, all additive: parseWebhook returns
WebhookEvent[] (Garmin batches; Wahoo adapted), manifest gains
configured()/importUrl/pkce, importActivity accepts summary stats
for FIT-less imports, manager gains markRevoked.
- Env-gated: no GARMIN_CLIENT_ID → provider hidden on
/settings/connections. Privacy manifest entry (DE+EN). i18n en+de.
Rollout (§6) stays gated on the Garmin Developer Program application
(submitted 2026-06-07). Fixtures are doc-shaped; the staging soak
swaps in recorded payloads if shapes differ.
Gate: typecheck ✓ lint ✓ unit+integration ✓ e2e 70/72 + both known
flakes green isolated ✓ openspec validate ✓
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Garmin Connect as the third connected-services provider, modeled on
wahoo-import but adapted to Garmin's push-first API: OAuth2+PKCE on the
existing oauth credential kind, ping/push webhook ingestion with async
pg-boss processing and an SSRF allowlist on callback URLs, date-range
backfill instead of a pick list (Garmin has no activity-list endpoint),
mandatory deregistration handling. Route push (Courses API) explicitly
deferred to a follow-up, mirroring wahoo-route-push.
Build is fixtures-first; rollout tasks are gated on Garmin Connect
Developer Program approval, which runs in parallel.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flagship ⇄ staging, both directions, 2026-06-07: follows settled to
Accepted in seconds, first polls ingested each side's public
activities, both feeds render remote attribution. The cross-origin
Accept fix (#490) verified working in production.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wiring follows the IS_FLAGSHIP pattern: the compose env defaults every
FEDERATION_* var to empty (off — the safe self-host default; all
federation surfaces 404), and cd-apps.yml appends
FEDERATION_ENABLED=true + FEDERATION_LOG_LEVEL=info to the flagship's
app.env. FEDERATION_KEY_ENCRYPTION_KEY was already in SOPS.
Rollout 12.2/12.3 soaked on staging against a real Mastodon
(2026-06-06/07); the trails↔trails Accept bug found by the §11 harness
is fixed and deployed. Rollback: drop the two echo lines, merge, rerun
cd-apps — instant off, follow rows persist.
First boot with the flag enqueues backfill-user-keypairs (idempotent)
and registers the federation jobs. Applied via manual cd-apps dispatch
after merge since workflow-file changes don't trigger the path filter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
12.1/12.2/12.3/12.6 marked with provenance (additive schema behind the
off flag on prod; the 2026-06-06/07 staging soak against a real
Mastodon covered inbound + push delivery; rollback documented in the
runbook). 12.4 annotated: protocol verified by the two-instance
harness, live staging⇄preview soak queued now that previews federate.
12.5 (prod flag flip) explicitly left as an operator decision.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 11.4 harness — e2e/federation/: postgres (two DBs) + a Caddy with
an internal CA terminating real HTTPS between two complete journal
containers (journal-a.test / journal-b.test), driven by an opt-in
integration test (FEDERATION_TWO_INSTANCE=1, via run.sh). The driver
seeds users straight into each DB, mints a signed session cookie
(known SESSION_SECRET), follows across instances through the real
/follows/outgoing route, and asserts the full pipeline: Follow →
auto-Accept → settle → first outbox poll → bob's public activity
rendered in alice's /feed with @bob@journal-b.test attribution. Plus a
wire-level 11.3 check: unsigned Create(Note) → 4xx, no DB writes.
And the harness immediately earned its keep — it caught a real
trails↔trails bug Mastodon interop never could: our Follow ids are
fragment URIs on OUR domain, so the Follow embedded in another trails
instance's Accept is cross-origin and Fedify rightly distrusts it;
re-fetching the id returns the actor document, getObject() yields a
Person, and the Accept/Reject/Undo listeners bailed silently — follows
stayed Pending forever. The listeners now fall back to the wire
objectId (captured BEFORE getObject(), which memoizes the fetched
document and changes what objectId reports) validated against our
Follow-id shape + the personal inbox's ctx.recipient. Forgery-safe:
settleOutgoingFollow only matches a Pending row toward the
HTTP-Signature-authenticated sender.
Supporting changes:
- federation.server.ts: env-gated allowPrivateAddress
(FEDERATION_ALLOW_PRIVATE_ADDRESS=true) so the harness's RFC 1918
Docker network is fetchable — testing only, loudly documented.
- Root .dockerignore: local builds shipped a 1GB+ context and
overlaid host (darwin) node_modules over the image's own install
via COPY . . — CI never noticed because it builds from clean
checkouts. Context is now ~5MB.
- tasks.md: 11.1–11.7 marked with provenance notes; design.md gains
the cross-origin embedded-object lesson.
Gate: typecheck ✓ lint ✓ unit+integration (FEDERATION_INTEGRATION=1) ✓
e2e 71/72 + known komoot flake green isolated ✓ harness run clean from
scratch (2/2, teardown included) ✓
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the 7.4 remainder. Fedify's FetchError carries the failed
Response, so a 429 from a remote's outbox now reads Retry-After
(delta-seconds or HTTP-date per RFC 9110), arms a per-host backoff
(15 min default, capped at the 1 h poll interval), and pollRemoteActor
skips backing-off hosts before doing any DB or network work. State is
in-process like the pacing map — a restart costs at most one extra
request that re-arms the backoff. last_polled_at is deliberately not
stamped on a rate-limited poll so the hourly sweep retries.
Unit tests cover the header parser (incl. the V8 footgun where
Date.parse reads bare negative integers as years), default/cap
behavior, and window expiry; an integration test drives the full
FetchError path and asserts the second poll never touches the network.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8.1/8.2: listSocialFeed is now a UNION ALL of local and remote
branches, sorted on COALESCE(remote_published_at, created_at):
- local rows: visibility='public' from accepted local follows — and
the previously missing accepted_at filter is added (the spec's
'Pending follows contribute nothing' scenario)
- remote rows: gated structurally by joining the viewer's OWN accepted
follow against the originating actor — which is exactly the
followers-only audience rule (a row reaches only viewers whose
follow brought it in); attribution from the remote_actors cache;
cards link outward to the canonical origin page (no local detail
page for remote rows)
9.1: annotated — local Pending button shipped with locked accounts;
remote Pending lives on /follows/outgoing.
9.2: already enforced + tested since §3 (actor/webfinger 404).
9.3 hardening: deliver-activity now re-checks the OWNER's profile
visibility at send time, closing the enqueue→delivery flip window
(enqueue-side and inbound-side gates already existed).
Integration tests: the §8 audience-leak guard (A sees followers-only,
B does not), public remote attribution + outward link, pending
contributes nothing, mixed local/remote COALESCE ordering.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tasks 7.1–7.3, 7.5 (+7.4 pacing; Retry-After-duration backoff noted as
remaining). Resolves the activities.owner_id open question.
Schema (design decision from the open question):
- activities.owner_id nullable + check constraint enforcing exactly
one of (owner_id, remote_actor_iri) — the follows pattern again.
- remote_published_at carries the origin's publish time for the §8
feed sort; index on remote_actor_iri for the feed join.
- Compiler-audited fallout: notification fan-out, the Note object
dispatcher, and the activity detail loader explicitly skip/404
remote rows (their canonical page is the origin; feed links there).
Every other surface joins users on owner_id and excludes remote rows
structurally.
Ingestion:
- federation-ingest.server.ts: parseOutboxItem targets exactly our own
outgoing Create(Note) shape (PropertyValue stats, first-paragraph
name, audience from to/cc); unknown items skipped, never fatal.
- ingestRemoteActivities: replay-safe via unique remote_origin_iri,
conflict-streak early exit (outboxes are newest-first).
- pollRemoteActor: signed fetch (Authorized Fetch via a local
follower's key), outbox resolution via remote_actors cache with
actor-doc fallback (which refreshes the cache), 1 req/5 s per-host
pacing, last_polled_at stamping. Network + pacing injectable.
Jobs: poll-remote-actor (per-actor; the §4 Accept listener already
enqueues it as the first poll) + poll-remote-outboxes (5-min cron
sweep over accepted remote follows not polled within the hour).
Tests: parse-shape units; integration suite for ingestion provenance,
audience→visibility mirroring, replay safety, the DB author invariant,
poll flow (actor-doc → collection → page), signer requirement, and
due-polling selection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
10.1 Privacy manifest (legal/privacy):
- German legal half: Föderation entry under Empfänger/Drittanbieter —
what a public profile exposes, what accepted followers' servers
receive, the loss-of-control over delivered copies, Art. 6(1)(a)
basis, private profiles don't federate.
- English manifest half: dedicated Federation (ActivityPub) section —
actor object contents, push delivery + retraction semantics (incl.
the tombstone caveat), encrypted-at-rest signing keys, remote actor
cache, ingested remote content with the followers-only viewer gate,
inbox logging/rate limits. PRIVACY_LAST_UPDATED bumped.
10.2 docs/deployment.md federation runbook: enabling per environment,
key rotation posture, abuse monitoring, and the troubleshooting
checklist distilled from the 2026-06-06/07 soak (IP families first,
no outbox backfill, tombstones, the 10s delivery timeout, actor
re-fetch).
10.3 Home marketing blurb (en+de) aligned to what actually ships:
Mastodon can follow you + trails-to-trails outbound — no more 'follow
friends anywhere' overstatement.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tasks 6.1–6.6. A local user can follow a user on another trails
instance: WebFinger-resolve the handle, verify the target runs
trails.cool via NodeInfo — checked against the ACTOR IRI's host, never
the handle's domain (split-domain constraint) — record a Pending
follow row, deliver a signed Follow. The §4 inbox listeners already
settle (Accept) or drop (Reject) the row.
- federation-outbound.server.ts: followRemoteActor / cancelRemoteFollow
(Undo delivery) / listOutgoingRemoteFollows. Network steps (lookup,
NodeInfo, delivery) injectable for offline integration tests.
Software allowlist: trails-cool.
- /follows/outgoing: follow-by-handle form + pending/accepted list +
cancel; linked from the feed empty state; i18n en+de. Remote actors
have no local profile page, so the remote Pending state lives here
(the profile Pending button from locked accounts covers local).
- /.well-known/trails-cool now publishes software: trails-cool (6.2).
- Clear 4xx codes: invalid_handle, local_handle, remote_resolve_failed,
not_trails (6.3).
- Tests: handle-parser + allowlist units; integration suite for the
Pending lifecycle (create/idempotent/refuse-non-trails/refuse-
unresolvable/own-host/cancel+Undo) with injected network deps; e2e
anonymous-redirect guard for the new page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prompted by Hollo's split-domain setup. Two parts:
- docs/ideas/split-domain-handles.md: offering split handle/server
domains to self-hosters is config-level work — Fedify's origin
option natively accepts { handleHost, webOrigin } — and handles are
permanent identity, so apex handles matter. Single-domain stays the
default (Decision #18); post-launch polish.
- Interop constraint pinned in social-federation task 6.1 and
route-federation's gating decision: the trails-to-trails NodeInfo
check must run against the actor IRI's host after WebFinger
resolution, never the handle's domain — split-domain remote
instances would otherwise be wrongly refused.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gap analysis of docs/architecture.md against openspec/ and docs/ideas/
found the envisioned-but-uncaptured remainder. This commit captures it:
New OpenSpec change (validated):
- route-federation — the collaboration half of the federation vision:
routes as dereferenceable trails:Route objects, Create/Update
fan-out, Invite/Accept collaboration mirroring (arch decision #2),
cross-instance Planner edits via HTTP-Signature-requested scoped
tokens (decisions #3/#12), mirror sync healing (#16). Depends on
social-federation §6 + route-sharing; carries the 2026-06-07 soak
lessons as design constraints.
New docs/ideas/ explorations:
- instance-administration — registration toggle, suspend/ban,
federation blocklists, reports (moderation now gates the federated
comments idea)
- social-interactions — local likes + comments (don't exist even
locally; the foundation federated kudos/comments attach to)
- activity-participants — group tagging with confirm/decline +
federated mentions (the participants jsonb column is an untyped stub)
- multi-day-collections — architecture open question #1, directions
evaluated
architecture.md cleanup:
- Mastodon-compat section annotated with shipped/captured state + the
live-soak interop lessons (attachment arrays, tombstones, 10s
timeout, no backfill)
- api.trails.cool removed (contradicted resolved decision #18)
- Phase 1 ticked (shipped); Phase 2/3 items annotated with where each
is tracked; specs/ → openspec/, activity.* → journal.*, cx21 → cx23
- brouter-web open question marked resolved-in-practice; new section
pointing to where the unshipped vision is tracked
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A received Delete is recorded by Mastodon as a permanent tombstone —
later Creates for the same URI are silently refused forever. The old
code sent Delete on every non-public save 'just in case' (the comment
even called over-sending harmless); on the 2026-06-07 soak this
tombstoned an unlisted activity's URI before its first real publish,
making its later flip to public invisible on the remote with no error
anywhere.
- visibilityTransitionAction(previous, next): Create on any transition
to public (re-publish doubles as back-delivery; remotes dedupe by
id), Delete only when leaving public, nothing for
non-public→non-public.
- updateActivityVisibility reads the previous visibility and acts on
the transition.
- design.md documents the tombstone permanence + the user-facing
consequence (un-publish then re-publish won't resurrect the post on
remotes that processed the retraction — same as Mastodon's own
delete-and-redraft).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
social-federation tasks 5.1–5.6. Completes the inbound-federation
story: a Mastodon follower now receives a trails user's new public
activities in their home timeline.
Outbox (5.1/5.2):
- /users/:username/outbox — paginated OrderedCollection of public
activities as Create(Note), newest first; unlisted/private never
federate. Private-user 404 enforced at the route layer because
Fedify builds collection-level responses from counter/cursors
without consulting the page dispatcher.
- Note shape: HTML content (escaped name/description/stats + link to
the activity page) with structured PropertyValue attachments
(distance-m, elevation-gain-m, duration-s) — Mastodon renders the
text, trails consumers read the structured fields. Resolves the
design open question toward Create(Note).
- Authorized Fetch: signed and unsigned outbox fetches deliberately
see the same (public-only) content until locked accounts exist.
Push delivery (5.3–5.6):
- createActivity / updateActivityVisibility(→public) enqueue one
deliver-activity job per accepted remote follower; flips away from
public and hard deletes enqueue Delete(Tombstone) retractions
(enqueued before the row disappears).
- deliver-activity job: re-reads the row at delivery time (skips if
gone or no longer public), resolves the recipient inbox via the
remote_actors cache with actor-document fetch fallback (priming the
cache), HTTP-signs via the owner's key, and POSTs. retryLimit 8 +
exponential backoff at enqueue time; outbound paced at 1 req/s per
remote host.
- Actor objects now advertise the outbox IRI.
- @js-temporal/polyfill added (same range Fedify uses) for published
timestamps; Fedify's types want the global esnext.temporal namespace,
bridged with a documented cast.
Tests: 9 unit tests for the AS mapping (escaping, stats, attachments,
published fallback, stable ids, tombstones), 4 outbox integration
tests (collection count, page shape/visibility filtering, private-404,
delivery audience query).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
social-federation tasks 3.1–3.4, 4.1–4.8. With this, a Mastodon user
can follow a public trails user: WebFinger → actor fetch → signed
Follow → recorded + Accept(Follow) pushed back.
Identity surface (section 3):
- Actor objects now carry the user's public key (publicKey +
assertionMethods via Fedify key pairs dispatcher; keys generated
lazily as a fallback to the backfill) and an inbox IRI; url uses
localActorIri (3.1).
- Software discovery shipped as standard NodeInfo
(/.well-known/nodeinfo + /nodeinfo/2.1, software.name trails-cool)
instead of the originally-sketched custom AS actor field — Fedify's
typed vocab can't emit arbitrary actor props and NodeInfo is what
the fediverse reads. Artifacts updated accordingly (3.4).
Inbox (section 4):
- /users/:username/inbox resource route; HTTP Signatures verified by
Fedify before any listener runs (4.1). Rate-limited 60 req/5 min per
source instance (host from Signature keyId) BEFORE verification so
hostile instances can't burn CPU on key fetches (4.8).
- Listeners: Follow → auto-accept for public profiles + Accept pushed
back (4.2); Undo(Follow) → row removed (4.3); Accept(Follow) →
Pending settled + first outbox poll enqueued (4.4); Reject(Follow) →
Pending dropped (4.5; UI notice deferred to 6.6). Unhandled types
are acknowledged + dropped by Fedify (4.6).
- Replay protection via Fedify's KvStore, now Postgres-backed
(journal.federation_kv + daily sweep job) so dedupe survives
restarts (4.7).
Schema (discovered requirement):
- follows.follower_id relaxed to nullable + follows.follower_actor_iri
for inbound remote followers — the proposal's 'follows is already
federation-ready' only held for outbound. Check constraint enforces
exactly one follower identity; partial unique index dedupes remote
follows. Notification fan-out + approve flow now filter local
followers explicitly. Design/proposal updated.
Tests: 7 inbox integration tests (accept/refuse/idempotence/undo/
settle/reject/check-constraint), 6 KvStore integration tests, 2 unit
tests for source-host extraction. All against real Postgres, gated on
FEDERATION_INTEGRATION=1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
