Commit graph

5 commits

Author SHA1 Message Date
Ullrich Schäfer
881991ca18
feat(journal): federation protocol doc + delivery observability
Task group 4 of federation-hardening.

4.1 — FEDERATION.md at the repo root: actor discovery (WebFinger, actor,
NodeInfo), object/activity types with real JSON examples (Note, Create,
Delete, the narrow follow-graph inbox), addressing, HTTP-Signature
expectations, the two-layer dedup contract, durable delivery/retry
policy, and blocklist moderation semantics — precise enough for another
implementation to interoperate. Linked from README and docs/architecture.

4.2 — three prom-client metrics + a journal dashboard row:
- `federation_delivery_total{outcome}` — incremented in deliver-activity
  (delivered/skipped/failed).
- `federation_inbox_dropped_total{reason}` — incremented at every inbox
  drop (duplicate | blocked); this is the counter deferred from task 3.2.
- `federation_queue_depth` — gauge sampled at scrape time in
  /api/metrics from PgBossMessageQueue.getDepth(); the restart-loss
  regression detector.
Grafana journal.json gains a Federation row (delivery rate, queue depth,
inbox drops); the logs panels shift down to make room.

Verified: dashboard JSON valid; journal typecheck + lint clean; unit
suite 357 passing (route-template guard unaffected).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:11:43 +02:00
Ullrich Schäfer
57696286e4
feat(journal): federation instance blocklist
Task group 3 of federation-hardening. There was no blocklist of any kind;
the only lever against a hostile instance was an IP/host block in Caddy.

- New `federation_blocked_instances` table (domain PK, reason,
  created_at). Additive → created by drizzle-kit push.
- `federation-blocklist.server.ts`: exact-host matching —
  `isBlockedDomain`, `isBlockedIri` (unparseable IRI ⇒ treated as
  blocked), and `filterBlockedDomains` for batch recipient filtering.
- Enforced at all three boundaries (spec: federation-operations
  "Instance blocklist"):
  - inbox — each of the 4 listeners silently drops a blocked actor's
    activity (202, no error oracle) before dedup/side effects;
  - delivery enqueue — `enqueueActivityDeliveries` filters blocked
    recipients in one batch query;
  - outbox poll / actor fetch — `pollRemoteActor` refuses a blocked host
    up front (`skipped: "blocked instance"`), before any network.
- Operator procedure (SQL insert/list/delete) documented in the
  deployment runbook's federation section.
- Tests: unit (hostOfIri) + integration against real Postgres covering
  the helper and the delivery + outbox boundaries; inbox uses the same
  tested isBlockedIri primitive.

Note: the inbox-drop *counter* (federation_inbox_dropped_total{reason})
lands with the other metrics in task 4.2; this commit is the enforcement.

Verified: db + journal typecheck + lint clean; drizzle-kit push creates
the table; blocklist integration tests green against real Postgres;
journal unit suite 357 passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:59:11 +02:00
Ullrich Schäfer
105659df7c
feat(journal): inbound federation replay defense
Task group 2 of federation-hardening. The narrow inbox
(Follow/Undo/Accept/Reject) had no replay protection — only Create(Note)
did, via the activities.remote_origin_iri unique constraint. A remote
redelivering a signed follow-graph activity would re-run its side
effects.

- New `federation_processed_activities` table (activity IRI PK,
  received_at + index). Additive, so drizzle-kit push creates it; no
  hand-written migration needed.
- `federation-replay.server.ts`: `markInboundActivityProcessed` does an
  insert-or-drop (ON CONFLICT DO NOTHING RETURNING) and reports whether
  the IRI is fresh; `sweepProcessedActivities` deletes rows > 30 days old
  (signature date-freshness already rejects older replays).
- Each inbox listener drops a duplicate before side effects. The
  follow-graph handlers are idempotent, so a handler failure whose retry
  is later dropped as a duplicate can't corrupt state.
- `federation-dedup-sweep` job (daily 04:30 UTC) runs the TTL sweep.

Verified: db + journal typecheck + lint clean; drizzle-kit push creates
the table; replay integration test (fresh-vs-duplicate + 30-day sweep)
green against real Postgres; journal unit suite 355 passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:07:30 +02:00
Ullrich Schäfer
b8fc8fadff
feat(journal): durable Fedify message queue over pg-boss
Task group 1 of federation-hardening. Fedify was configured with
`InProcessMessageQueue`, so every queued outbound delivery, its pending
retry state, and inbox processing task was lost on a container restart
(routine here: deploys, OOM history) — directly contradicting the
social-federation spec's promise that fan-out survives a deploy.

- `federation-queue.server.ts`: `PgBossMessageQueue` implementing Fedify's
  `MessageQueue` over the pg-boss instance the journal already runs,
  mirroring the `PostgresKvStore` adapter. `nativeRetrial = false` keeps
  Fedify the retry-policy owner; pg-boss supplies durability + delayed
  jobs (delay → whole-second `startAfter`, `retryLimit: 0`).
- Swap it in for `InProcessMessageQueue` in `federation.server.ts`;
  document the two intentional queueing layers (our fan-out jobs feed
  Fedify; Fedify's sends now durable underneath).
- `server.ts`: create the durable queue at startup when federation is on.
- Broaden the structural `BossLike` in `boss.server.ts` with the
  work/offWork/createQueue/getQueue methods the adapter needs.
- Tests: enqueue maps retry/delay correctly, consume roundtrip, restart
  durability (fresh listener drains a prior instance's backlog), abort
  stops the worker, depth reports ready vs delayed.

Verified: journal typecheck + lint clean, 7/7 new unit tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 21:46:46 +02:00
Ullrich Schäfer
5dd4968626 docs+openspec: prior-art research (Organic Maps, Endurain, wanderer) and 15 proposals
Add docs/inspirations.md as the durable record of the 2026-07-05/06
prior-art research — per-project learnings with source paths, canonical
credit lines, and the changes each spawned — and extend the
acknowledgment lists in philosophy.md/architecture.md (Organic Maps,
Endurain, wanderer).

New OpenSpec changes (proposal/design/specs/tasks each):
- Organic Maps: elevation-profile-hardening, gpx-parser-robustness,
  hiking-time-estimate, poi-index, hiking-foot-profile
- Endurain: account-export, activity-duplicate-review,
  fit-parsing-hardening, activity-locations, self-hosting-guide,
  activity-privacy-controls
- wanderer: federation-hardening, link-share-tokens
- credits-page (user-visible acknowledgments)

Updated in-flight changes with wanderer prior-art sections:
route-federation, route-discovery.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 07:23:00 +02:00