From 1eceb6f1f71704fbf50254b1d59f0a898a9d6d65 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ullrich=20Sch=C3=A4fer?= Date: Sun, 7 Jun 2026 13:36:35 +0200 Subject: [PATCH] =?UTF-8?q?feat(journal):=20two-instance=20federation=20ha?= =?UTF-8?q?rness=20+=20trails=E2=86=94trails=20Accept=20fix=20(=C2=A711)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 11.4 harness — e2e/federation/: postgres (two DBs) + a Caddy with an internal CA terminating real HTTPS between two complete journal containers (journal-a.test / journal-b.test), driven by an opt-in integration test (FEDERATION_TWO_INSTANCE=1, via run.sh). The driver seeds users straight into each DB, mints a signed session cookie (known SESSION_SECRET), follows across instances through the real /follows/outgoing route, and asserts the full pipeline: Follow → auto-Accept → settle → first outbox poll → bob's public activity rendered in alice's /feed with @bob@journal-b.test attribution. Plus a wire-level 11.3 check: unsigned Create(Note) → 4xx, no DB writes. And the harness immediately earned its keep — it caught a real trails↔trails bug Mastodon interop never could: our Follow ids are fragment URIs on OUR domain, so the Follow embedded in another trails instance's Accept is cross-origin and Fedify rightly distrusts it; re-fetching the id returns the actor document, getObject() yields a Person, and the Accept/Reject/Undo listeners bailed silently — follows stayed Pending forever. The listeners now fall back to the wire objectId (captured BEFORE getObject(), which memoizes the fetched document and changes what objectId reports) validated against our Follow-id shape + the personal inbox's ctx.recipient. Forgery-safe: settleOutgoingFollow only matches a Pending row toward the HTTP-Signature-authenticated sender. Supporting changes: - federation.server.ts: env-gated allowPrivateAddress (FEDERATION_ALLOW_PRIVATE_ADDRESS=true) so the harness's RFC 1918 Docker network is fetchable — testing only, loudly documented. - Root .dockerignore: local builds shipped a 1GB+ context and overlaid host (darwin) node_modules over the image's own install via COPY . . — CI never noticed because it builds from clean checkouts. Context is now ~5MB. - tasks.md: 11.1–11.7 marked with provenance notes; design.md gains the cross-origin embedded-object lesson. Gate: typecheck ✓ lint ✓ unit+integration (FEDERATION_INTEGRATION=1) ✓ e2e 71/72 + known komoot flake green isolated ✓ harness run clean from scratch (2/2, teardown included) ✓ Co-Authored-By: Claude Opus 4.8 (1M context) --- .dockerignore | 14 ++ ...ederation-two-instance.integration.test.ts | 203 ++++++++++++++++++ apps/journal/app/lib/federation.server.ts | 96 +++++++-- e2e/federation/Caddyfile | 16 ++ e2e/federation/README.md | 56 +++++ e2e/federation/docker-compose.yml | 145 +++++++++++++ e2e/federation/init-dbs.sql | 6 + e2e/federation/run.sh | 55 +++++ openspec/changes/social-federation/design.md | 18 ++ openspec/changes/social-federation/tasks.md | 21 +- 10 files changed, 610 insertions(+), 20 deletions(-) create mode 100644 .dockerignore create mode 100644 apps/journal/app/lib/federation-two-instance.integration.test.ts create mode 100644 e2e/federation/Caddyfile create mode 100644 e2e/federation/README.md create mode 100644 e2e/federation/docker-compose.yml create mode 100644 e2e/federation/init-dbs.sql create mode 100755 e2e/federation/run.sh diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..0d2af30 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,14 @@ +# Keep Docker build contexts small and hermetic. CI builds from a clean +# checkout so this mostly matters for local builds (e2e/federation +# harness), where node_modules would otherwise bloat the context to +# gigabytes — and worse, `COPY . .` would overlay host-installed +# node_modules over the image's own pnpm install. +**/node_modules +**/.turbo +**/build +**/.react-router +.git +e2e/results +playwright-report +test-results +**/*.log diff --git a/apps/journal/app/lib/federation-two-instance.integration.test.ts b/apps/journal/app/lib/federation-two-instance.integration.test.ts new file mode 100644 index 0000000..d8ad1ee --- /dev/null +++ b/apps/journal/app/lib/federation-two-instance.integration.test.ts @@ -0,0 +1,203 @@ +// Two-instance federation drive-through (spec: social-federation 11.4). +// +// Talks to the docker-compose harness in e2e/federation/ — two complete +// journal instances (journal-a.test / journal-b.test) federating over a +// private Docker network with real HTTPS between them. This test is the +// driver: it seeds users straight into each instance's database, mints +// a session cookie for alice (cookie sessions are signed with the +// harness's known SESSION_SECRET), follows bob across instances through +// the real /follows/outgoing route, and then watches the entire +// Follow → Accept → first outbox poll → feed pipeline happen between +// two live servers. +// +// Run via e2e/federation/run.sh — it builds the images, pushes the +// schema into both databases, and sets FEDERATION_TWO_INSTANCE=1. + +import { describe, it, expect, afterAll } from "vitest"; +import postgres, { type Sql } from "postgres"; +import { randomUUID } from "node:crypto"; +import { createCookieSessionStorage } from "react-router"; + +const runTwoInstance = process.env.FEDERATION_TWO_INSTANCE === "1"; + +// Host-published harness endpoints (see e2e/federation/docker-compose.yml). +const A_URL = "http://127.0.0.1:3401"; +const B_URL = "http://127.0.0.1:3402"; +const DB_A = "postgres://trails:trails@127.0.0.1:5499/trails_a"; +const DB_B = "postgres://trails:trails@127.0.0.1:5499/trails_b"; +const B_DOMAIN = "journal-b.test"; +const BOB_ACTOR_IRI = `https://${B_DOMAIN}/users/bob`; +const ACTIVITY_NAME = "Sunrise ride over the ridge"; + +/** Poll `fn` until it returns something truthy. */ +async function until( + what: string, + fn: () => Promise, + timeoutMs = 120_000, +): Promise { + const start = Date.now(); + for (;;) { + const value = await fn(); + if (value) return value; + if (Date.now() - start > timeoutMs) throw new Error(`Timed out waiting for ${what}`); + await new Promise((r) => setTimeout(r, 1500)); + } +} + +async function seedUser(sql: Sql, username: string, domain: string): Promise { + const id = randomUUID(); + await sql` + INSERT INTO journal.users (id, email, username, domain, profile_visibility, terms_accepted_at, terms_version) + VALUES (${id}, ${`${username}@${domain}`}, ${username}, ${domain}, 'public', now(), '2026-04-19') + `; + return id; +} + +/** + * Mint a valid `__session` cookie for a user. The harness journals run + * with a known SESSION_SECRET, and sessions are signed cookies (see + * auth/session.server.ts) — so the driver can authenticate as any + * seeded user without driving the WebAuthn ceremony. + */ +async function mintSessionCookie(userId: string): Promise { + const storage = createCookieSessionStorage({ + cookie: { + name: "__session", + httpOnly: true, + sameSite: "lax", + path: "/", + secrets: ["two-instance-test-secret"], + }, + }); + const session = await storage.getSession(); + session.set("userId", userId); + const setCookie = await storage.commitSession(session); + return setCookie.split(";")[0]!; +} + +describe.runIf(runTwoInstance)("two-instance federation (11.4)", () => { + const sqlA = postgres(DB_A, { max: 2 }); + const sqlB = postgres(DB_B, { max: 2 }); + + afterAll(async () => { + await sqlA.end(); + await sqlB.end(); + }); + + it( + "alice@journal-a follows bob@journal-b: Follow → Accept → first poll → bob's public activity in alice's feed", + { timeout: 300_000 }, + async () => { + // Both instances are up (run.sh already gated on the compose + // healthchecks; this is a fail-fast sanity check). + for (const base of [A_URL, B_URL]) { + const health = await fetch(`${base}/api/health`); + expect(health.status).toBe(200); + } + + // ---- Seed: alice on A; bob + a public activity on B ---------- + const aliceId = await seedUser(sqlA, "alice", "journal-a.test"); + const bobId = await seedUser(sqlB, "bob", B_DOMAIN); + await sqlB` + INSERT INTO journal.activities (id, owner_id, name, description, visibility) + VALUES (${randomUUID()}, ${bobId}, ${ACTIVITY_NAME}, '', 'public') + `; + + // B serves bob over WebFinger before we point A at him. (Fedify's + // canonical-origin support means the host-port URL works here.) + const wf = await fetch(`${B_URL}/.well-known/webfinger?resource=acct:bob@${B_DOMAIN}`); + expect(wf.status).toBe(200); + + // ---- alice follows bob through the real route ----------------- + const cookie = await mintSessionCookie(aliceId); + const follow = await fetch(`${A_URL}/follows/outgoing`, { + method: "POST", + redirect: "manual", + headers: { cookie, "content-type": "application/x-www-form-urlencoded" }, + body: new URLSearchParams({ intent: "follow", handle: `bob@${B_DOMAIN}` }), + }); + if (follow.status >= 400) { + // FollowError surfaces as a 400 with a code — bubble the body + // up so a harness failure says *why* resolution failed. + throw new Error(`follow action failed: ${follow.status} ${await follow.text()}`); + } + + // Outbound resolution (NodeInfo + WebFinger + actor fetch over the + // Caddy TLS hop) happens inside the action — a Pending row is the + // proof it all worked. + const pending = await until("pending follow row on A", async () => { + const rows = await sqlA` + SELECT id, accepted_at FROM journal.follows + WHERE follower_id = ${aliceId} AND followed_actor_iri = ${BOB_ACTOR_IRI} + `; + return rows[0]; + }, 30_000); + expect(pending).toBeDefined(); + + // ---- B auto-accepts (public profile) and delivers Accept ------ + await until("Accept to settle the follow on A", async () => { + const rows = await sqlA` + SELECT accepted_at FROM journal.follows + WHERE follower_id = ${aliceId} AND followed_actor_iri = ${BOB_ACTOR_IRI} + `; + return rows[0]?.accepted_at != null; + }); + + // B's side sees a remote follower row for alice. + const bFollow = await sqlB` + SELECT follower_actor_iri FROM journal.follows WHERE followed_user_id = ${bobId} + `; + expect(bFollow[0]?.follower_actor_iri).toBe("https://journal-a.test/users/alice"); + + // ---- First poll ingests bob's outbox into A ------------------- + const ingested = await until("bob's activity to be ingested on A", async () => { + const rows = await sqlA` + SELECT name, audience, owner_id FROM journal.activities + WHERE remote_actor_iri = ${BOB_ACTOR_IRI} + `; + return rows[0]; + }); + expect(ingested.name).toBe(ACTIVITY_NAME); + expect(ingested.audience).toBe("public"); + expect(ingested.owner_id).toBeNull(); + + // ---- And it shows up in alice's rendered feed ------------------ + const feed = await fetch(`${A_URL}/feed`, { headers: { cookie } }); + expect(feed.status).toBe(200); + // JSX puts `` separators between adjacent text expressions + // — strip them so the attribution reads as continuous text. + const html = (await feed.text()).replaceAll("", ""); + expect(html).toContain(ACTIVITY_NAME); + expect(html).toContain(`@bob@${B_DOMAIN}`); + }, + ); + + it("an unsigned Create(Note) posted to the inbox is rejected with no DB writes (11.3)", async () => { + const originIri = `https://intruder.example/notes/${randomUUID()}`; + const res = await fetch(`${A_URL}/users/alice/inbox`, { + method: "POST", + headers: { "content-type": "application/activity+json" }, + body: JSON.stringify({ + "@context": "https://www.w3.org/ns/activitystreams", + id: `${originIri}#create`, + type: "Create", + actor: "https://intruder.example/users/mallory", + to: "https://www.w3.org/ns/activitystreams#Public", + object: { + id: originIri, + type: "Note", + content: "

Injected note

", + }, + }), + }); + // Unauthenticated inbox delivery: Fedify refuses it outright + // (400 malformed / 401 unsigned — both fine, never 2xx). + expect(res.status).toBeGreaterThanOrEqual(400); + expect(res.status).toBeLessThan(500); + + const rows = await sqlA` + SELECT id FROM journal.activities WHERE remote_origin_iri = ${originIri} + `; + expect(rows).toHaveLength(0); + }); +}); diff --git a/apps/journal/app/lib/federation.server.ts b/apps/journal/app/lib/federation.server.ts index fa25f68..021cc5f 100644 --- a/apps/journal/app/lib/federation.server.ts +++ b/apps/journal/app/lib/federation.server.ts @@ -97,6 +97,20 @@ async function findLocalPublicUserByIri(iri: URL | null): Promise#follows/`, see federation-outbound.server.ts) and + * names the recipient of the personal inbox the activity arrived in. + * Used by the Accept/Reject listeners: another trails instance can't + * embed a trustworthy copy of our Follow (its id is cross-origin for + * them), so the id is the only recoverable reference. + */ +function isRecipientFollowUri(ctx: { recipient: string | null }, id: URL | null): boolean { + if (id == null || ctx.recipient == null) return false; + if (!id.hash.startsWith("#follows/")) return false; + return id.href.startsWith(`${localActorIri(ctx.recipient)}#`); +} + let _federation: Federation | null = null; let _logtapeConfigured = false; @@ -146,6 +160,15 @@ function buildFederation(): Federation { // Canonical origin so generated IRIs are correct behind the Caddy // proxy (the Node server itself only sees plain HTTP). origin: getOrigin(), + // SSRF-guard opt-out for the two-instance federation harness + // (e2e/federation/), where both instances live on a private Docker + // network that Fedify's document loader rightly refuses to fetch + // from. Testing only — never set this in a real deployment: the + // private-address block is a genuine security boundary (a malicious + // actor IRI must not be able to point us at internal targets). + ...(process.env.FEDERATION_ALLOW_PRIVATE_ADDRESS === "true" + ? { allowPrivateAddress: true } + : {}), }); if (!process.env.VITEST) { // Fire-and-forget: runs the queue consumer loop for the process @@ -261,25 +284,66 @@ function buildFederation(): Federation { .on(Undo, async (ctx, undo) => { // Spec 4.3: Undo(Follow) removes the follow row. Other Undos are // acknowledged and dropped. + if (undo.actorId == null) return; + const undoObjectId = undo.objectId; // capture before dereference (see Accept) const object = await undo.getObject(ctx); - if (!(object instanceof Follow)) return; - if (undo.actorId == null || object.objectId == null) return; - const parsed = ctx.parseUri(object.objectId); - if (parsed?.type !== "actor") return; - await removeRemoteFollow(undo.actorId.href, parsed.identifier); + if (object instanceof Follow && object.objectId != null) { + const parsed = ctx.parseUri(object.objectId); + if (parsed?.type !== "actor") return; + await removeRemoteFollow(undo.actorId.href, parsed.identifier); + return; + } + // trails→trails fallback: another trails instance's Undo embeds + // a Follow whose id lives on the SENDER's domain, but the embed + // is cross-origin relative to nothing we can verify, and + // dereferencing `…#follows/` yields the sender's actor + // document, not a Follow (fragments aren't separately fetchable). + // Our Follow ids are `#follows/` — when the Undo + // names one owned by the authenticated sender, the recipient of + // this personal inbox is the unfollowed user. + if ( + ctx.recipient != null && + undoObjectId?.hash.startsWith("#follows/") && + undoObjectId.origin === new URL(undo.actorId.href).origin + ) { + await removeRemoteFollow(undo.actorId.href, ctx.recipient); + } }) .on(Accept, async (ctx, accept) => { // Spec 4.4: a remote accepted our outgoing Follow — settle the // Pending row and trigger the first outbox poll for that actor. - const object = await accept.getObject(ctx); - if (!(object instanceof Follow)) return; if (accept.actorId == null) return; - const localUser = await findLocalPublicUserByIri(object.actorId); + // Capture the raw object reference BEFORE dereferencing: + // getObject() memoizes the fetched document, after which objectId + // reports the fetched object's id (fragment stripped) instead of + // the id that was on the wire. + const objectId = accept.objectId; + const object = await accept.getObject(ctx); + logger.debug( + { + recipient: ctx.recipient, + objectId: objectId?.href ?? null, + objectType: object?.constructor?.name ?? null, + }, + "federation: Accept listener dispatch", + ); + let localUser: Awaited> = null; + if (object instanceof Follow) { + localUser = await findLocalPublicUserByIri(object.actorId); + } else if (isRecipientFollowUri(ctx, objectId)) { + // trails→trails: our own Follow id is cross-origin from the + // accepting instance's perspective, so Fedify rightly distrusts + // the embedded copy and a re-fetch of `…#follows/` returns + // our actor document instead of a Follow. The id itself names + // this inbox's recipient, which is all settling needs — and + // settleOutgoingFollow only matches a Pending row toward the + // authenticated sender, so a forged Accept can't settle + // anything that wasn't already directed at that actor. + localUser = await findLocalPublicUserByIri(new URL(localActorIri(ctx.recipient!))); + } if (!localUser) return; const { settled } = await settleOutgoingFollow(localUser.id, accept.actorId.href); if (settled) { - // Queue worker lands in the outbox-poll change (task 7.x); - // enqueueOptional logs-and-continues until then. await enqueueOptional("poll-remote-actor", { actorIri: accept.actorId.href }, { reason: "first poll after accepted follow", }); @@ -287,10 +351,16 @@ function buildFederation(): Federation { }) .on(Reject, async (ctx, reject) => { // Spec 4.5: remote refused our Follow — drop the Pending row. - const object = await reject.getObject(ctx); - if (!(object instanceof Follow)) return; if (reject.actorId == null) return; - const localUser = await findLocalPublicUserByIri(object.actorId); + const objectId = reject.objectId; // capture before dereference (see Accept) + const object = await reject.getObject(ctx); + let localUser: Awaited> = null; + if (object instanceof Follow) { + localUser = await findLocalPublicUserByIri(object.actorId); + } else if (isRecipientFollowUri(ctx, objectId)) { + // Same trails→trails fallback as the Accept listener above. + localUser = await findLocalPublicUserByIri(new URL(localActorIri(ctx.recipient!))); + } if (!localUser) return; await rejectOutgoingFollow(localUser.id, reject.actorId.href); }) diff --git a/e2e/federation/Caddyfile b/e2e/federation/Caddyfile new file mode 100644 index 0000000..ff65461 --- /dev/null +++ b/e2e/federation/Caddyfile @@ -0,0 +1,16 @@ +# TLS terminator for the two-instance federation harness. `local_certs` +# issues every site cert from Caddy's internal CA; the journals trust it +# via NODE_EXTRA_CA_CERTS (see docker-compose.yml). +{ + local_certs + # No public reachability — don't try (or wait on) ACME/OCSP anything. + skip_install_trust +} + +journal-a.test { + reverse_proxy journal-a:3000 +} + +journal-b.test { + reverse_proxy journal-b:3000 +} diff --git a/e2e/federation/README.md b/e2e/federation/README.md new file mode 100644 index 0000000..23e1e2d --- /dev/null +++ b/e2e/federation/README.md @@ -0,0 +1,56 @@ +# Two-instance federation harness + +Spec: `openspec/changes/social-federation/` task 11.4. + +Two complete journal instances — `journal-a.test` and `journal-b.test` — +federate with each other inside a Docker network, with real HTTPS +between them (a Caddy with `local_certs` terminates TLS; the journals +trust its internal CA via `NODE_EXTRA_CA_CERTS`). The driver test seeds +a user on each side, follows across instances through the real UI +route, and asserts the full pipeline: + +``` +alice@journal-a ──Follow──▶ bob@journal-b (NodeInfo + WebFinger + + ◀──Accept── actor fetch over TLS) + ──signed outbox poll──▶ +alice's /feed shows bob's public activity with @bob@journal-b.test +``` + +## Run it + +```bash +./e2e/federation/run.sh # build, test, tear down +KEEP_STACK=1 ./e2e/federation/run.sh # leave the stack up afterwards +``` + +First run builds the journal image (a few minutes); later runs reuse it. +Opt-in only — nothing here runs under `pnpm test`, `pnpm test:e2e`, or +CI. The driver lives at +`apps/journal/app/lib/federation-two-instance.integration.test.ts` +(gated by `FEDERATION_TWO_INSTANCE=1`). + +## How the driver authenticates + +Sessions are signed cookies (`auth/session.server.ts`). The harness +journals run with a known `SESSION_SECRET`, so the driver mints a valid +`__session` cookie for the seeded user instead of driving the WebAuthn +ceremony. That secret (and everything else in the compose env) is a +throwaway test value on a loopback-only stack. + +## Why `FEDERATION_ALLOW_PRIVATE_ADDRESS` + +Fedify's document loader refuses private network addresses (SSRF +guard). Both instances here live on RFC 1918 Docker addresses, so the +harness sets `FEDERATION_ALLOW_PRIVATE_ADDRESS=true` — the env-gated +opt-out in `federation.server.ts`. Never set it in a real deployment. + +## Debugging + +```bash +docker compose -f e2e/federation/docker-compose.yml logs -f journal-a +docker compose -f e2e/federation/docker-compose.yml exec postgres \ + psql -U trails trails_a -c 'TABLE journal.follows' +``` + +`FEDERATION_LOG_LEVEL=debug` is already on — Fedify logs every +signature verification and delivery attempt. diff --git a/e2e/federation/docker-compose.yml b/e2e/federation/docker-compose.yml new file mode 100644 index 0000000..60e8866 --- /dev/null +++ b/e2e/federation/docker-compose.yml @@ -0,0 +1,145 @@ +# Two-instance federation harness (social-federation task 11.4). +# +# Brings up two complete journal instances — journal-a.test and +# journal-b.test — that federate with each other over a private Docker +# network, with real HTTPS between them (Caddy internal CA; the journals +# trust it via NODE_EXTRA_CA_CERTS). Driven by +# apps/journal/app/lib/federation-two-instance.integration.test.ts via +# ./run.sh — see README.md. +# +# Ports published to the host (driver-facing, loopback only): +# 127.0.0.1:5499 → postgres (schema push + seeding + asserts) +# 127.0.0.1:3401 → journal-a :3000 (plain HTTP; the TLS hop is only +# 127.0.0.1:3402 → journal-b :3000 needed for instance↔instance) + +name: trails-federation-e2e +# Shared journal environment. Secrets here are throwaway test values — +# the whole stack is loopback-only and torn down by run.sh. +x-journal-env: &journal-env + SESSION_SECRET: two-instance-test-secret + FEDERATION_ENABLED: "true" + FEDERATION_KEY_ENCRYPTION_KEY: two-instance-test-encryption-key + FEDERATION_LOG_LEVEL: debug + LOG_LEVEL: debug + # Both instances live on RFC 1918 Docker addresses; Fedify's SSRF + # guard must be told to stand down. Testing only — see the comment in + # apps/journal/app/lib/federation.server.ts. + FEDERATION_ALLOW_PRIVATE_ADDRESS: "true" + # Trust Caddy's internal CA for the instance↔instance HTTPS hop. + # Published world-readable by the ca-publisher one-shot — the journal + # runs as a non-root user and can't read caddy's 0700 data volume. + NODE_EXTRA_CA_CERTS: /ca/root.crt + SENTRY_DISABLED: "true" + + +services: + postgres: + image: imresamu/postgis:16-3.4 # multi-arch, same as docker-compose.dev.yml + environment: + POSTGRES_USER: trails + POSTGRES_PASSWORD: trails + POSTGRES_DB: trails_a + volumes: + - ./init-dbs.sql:/docker-entrypoint-initdb.d/90-init-dbs.sql:ro + ports: + - "127.0.0.1:5499:5432" + healthcheck: + # -h 127.0.0.1 forces TCP: during first-boot initdb a temporary + # postgres answers on the unix socket only, and a socket-based + # pg_isready reports healthy before the init scripts have run. + test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U trails -d trails_b"] + interval: 2s + timeout: 3s + retries: 30 + + caddy: + image: caddy:2-alpine + volumes: + - ./Caddyfile:/etc/caddy/Caddyfile:ro + - caddy-data:/data + networks: + default: + # The instances reach each other through Caddy: these aliases + # make journal-a.test / journal-b.test resolve to the proxy + # inside the compose network. + aliases: + - journal-a.test + - journal-b.test + healthcheck: + # The journals read the internal CA root at process start + # (NODE_EXTRA_CA_CERTS) — gate their startup on it existing. + test: ["CMD", "test", "-f", "/data/caddy/pki/authorities/local/root.crt"] + interval: 2s + timeout: 3s + retries: 30 + + # One-shot: copies Caddy's internal root CA into a volume the + # non-root journal user can actually read (caddy's own data dir is + # 0700 root). + ca-publisher: + image: caddy:2-alpine + entrypoint: + - sh + - -c + - | + until [ -f /data/caddy/pki/authorities/local/root.crt ]; do sleep 0.5; done + cp /data/caddy/pki/authorities/local/root.crt /ca/root.crt + chmod 444 /ca/root.crt + volumes: + - caddy-data:/data:ro + - ca-cert:/ca + depends_on: + caddy: + condition: service_healthy + + journal-a: + build: + context: ../.. + dockerfile: apps/journal/Dockerfile + image: trails-federation-e2e-journal + pull_policy: never + environment: + <<: *journal-env + DATABASE_URL: postgres://trails:trails@postgres:5432/trails_a + ORIGIN: https://journal-a.test + ports: + - "127.0.0.1:3401:3000" + volumes: + - ca-cert:/ca:ro + healthcheck: &journal-health + test: ["CMD", "curl", "-fsS", "http://localhost:3000/api/health"] + interval: 3s + timeout: 5s + retries: 40 + start_period: 10s + depends_on: + postgres: + condition: service_healthy + ca-publisher: + condition: service_completed_successfully + + journal-b: + # Image-only on purpose: run.sh builds it once via journal-a before + # `up`. Building from both services races the export of the shared + # tag ("already exists"), and a registry pull must never happen. + image: trails-federation-e2e-journal + pull_policy: never + environment: + <<: *journal-env + DATABASE_URL: postgres://trails:trails@postgres:5432/trails_b + ORIGIN: https://journal-b.test + ports: + - "127.0.0.1:3402:3000" + volumes: + - ca-cert:/ca:ro + healthcheck: *journal-health + depends_on: + postgres: + condition: service_healthy + ca-publisher: + condition: service_completed_successfully + +volumes: + caddy-data: + ca-cert: + diff --git a/e2e/federation/init-dbs.sql b/e2e/federation/init-dbs.sql new file mode 100644 index 0000000..589b4e9 --- /dev/null +++ b/e2e/federation/init-dbs.sql @@ -0,0 +1,6 @@ +-- Second instance database. trails_a is the image's POSTGRES_DB (the +-- postgis image creates the extension there itself); trails_b needs +-- both created by hand. Runs once on first postgres boot. +CREATE DATABASE trails_b; +\connect trails_b +CREATE EXTENSION IF NOT EXISTS postgis; diff --git a/e2e/federation/run.sh b/e2e/federation/run.sh new file mode 100755 index 0000000..cb5be49 --- /dev/null +++ b/e2e/federation/run.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# Two-instance federation harness runner (social-federation 11.4). +# +# ./e2e/federation/run.sh # build, test, tear down +# KEEP_STACK=1 ./e2e/federation/run.sh # leave the stack up for poking +# +# Brings up two journal instances that federate with each other (see +# docker-compose.yml), pushes the Drizzle schema into both databases, +# and runs the driver test. Opt-in and self-contained — nothing here is +# wired into `pnpm test` or CI. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" +compose() { docker compose -f "$SCRIPT_DIR/docker-compose.yml" "$@"; } + +cleanup() { + if [[ "${KEEP_STACK:-}" == "1" ]]; then + echo "KEEP_STACK=1 — leaving the stack up. Tear down with:" + echo " docker compose -f $SCRIPT_DIR/docker-compose.yml down -v" + else + compose down -v --remove-orphans >/dev/null 2>&1 || true + fi +} +trap cleanup EXIT + +echo "==> postgres + caddy (internal CA)" +compose up -d --wait postgres caddy + +echo "==> pushing schema into trails_a and trails_b" +push_schema() { + # A transient postgres backend crash mid-introspection has been seen + # under Docker Desktop load ("Connection terminated unexpectedly" + + # crash recovery). Retry after the healthcheck reports the server + # recovered — without hiding a deterministic failure. + local url="$1" attempt + for attempt in 1 2 3; do + if DATABASE_URL="$url" pnpm --dir "$ROOT" db:push; then return 0; fi + echo "schema push failed (attempt $attempt) — waiting for postgres to recover" + sleep 5 + compose up -d --wait postgres + done + return 1 +} +push_schema postgres://trails:trails@127.0.0.1:5499/trails_a +push_schema postgres://trails:trails@127.0.0.1:5499/trails_b + +echo "==> building the journal image (slow on first run)" +compose build journal-a + +echo "==> journal-a + journal-b" +compose up -d --wait journal-a journal-b + +echo "==> driving the federation flow" +FEDERATION_TWO_INSTANCE=1 pnpm --dir "$ROOT/apps/journal" exec vitest run app/lib/federation-two-instance.integration.test.ts diff --git a/openspec/changes/social-federation/design.md b/openspec/changes/social-federation/design.md index 0081bd4..741b799 100644 --- a/openspec/changes/social-federation/design.md +++ b/openspec/changes/social-federation/design.md @@ -108,6 +108,24 @@ Inbound signature verification uses the actor's public key from their actor obje ## Implementation Decisions (made during apply) +- **Cross-origin object references in Accept/Reject/Undo listeners** + (decided in task 11.4, 2026-06-07). When another *trails* instance + accepts our Follow, the embedded Follow inside its Accept is + cross-origin from the sender's perspective (the Follow's id lives on + OUR domain), so Fedify correctly distrusts the embedded copy and + re-fetches the id — but our Follow ids are fragment URIs + (`#follows/`), and fetching one returns the actor + document, not a Follow. `getObject()` then yields a `Person` and the + listener used to bail silently. Mastodon never trips this because the + Accept it sends embeds its *own* (same-origin) Follow. Fix: listeners + fall back to the wire `objectId` — captured **before** `getObject()`, + which memoizes the fetched document and changes what `objectId` + reports — validated against our Follow-id shape plus the personal + inbox's `ctx.recipient`. Still forgery-safe: `settleOutgoingFollow` + only matches a Pending row toward the HTTP-Signature-authenticated + sender. Found by (and regression-covered in) the two-instance + harness, `e2e/federation/`. + - **Inbound remote followers live in `follows` with a nullable `follower_id`** (decided in task 4.2, 2026-06-06). The original claim that `follows` was federation-ready only covered outbound; a remote follower has no local diff --git a/openspec/changes/social-federation/tasks.md b/openspec/changes/social-federation/tasks.md index 5ef6bab..44176ac 100644 --- a/openspec/changes/social-federation/tasks.md +++ b/openspec/changes/social-federation/tasks.md @@ -84,13 +84,20 @@ ## 11. Testing -- [ ] 11.1 Unit tests: HTTP-Signature verification on inbox; signature production on outbox-poll; encrypt/decrypt of private keys -- [ ] 11.2 Integration test: post a signed `Follow` to local inbox from a fake Mastodon-shaped client → assert `Accept(Follow)` is delivered + follow row exists -- [ ] 11.3 Integration test: post a `Create(Note)` to local inbox → assert it is dropped silently (no DB writes) -- [ ] 11.4 Integration test: bring up two trails instances (Docker Compose multi-instance setup); A on instance 1 follows B on instance 2 → assert Follow → Accept → first poll all happen and B's public activity appears in A's `/feed` -- [ ] 11.5 Integration test: outbound follow attempt against a Mastodon-shaped actor → assert 4xx with the limitation message -- [ ] 11.6 Integration test (audience leak guard): two local users A and B, only A follows remote trails actor X; X's followers-only post lands in cache. Assert A sees it, B does not -- [ ] 11.7 E2E: with feature flag on, follow bruno across instances + see public activity appear +- [x] 11.1 Unit tests: HTTP-Signature verification on inbox; signature production on outbox-poll; encrypt/decrypt of private keys + > Done via combination (2026-06-07): encrypt/decrypt + sign/verify roundtrip in `federation-keys.server.test.ts`; HTTP-Signature verification/production is Fedify's tested core, exercised wire-level by the 11.4 harness (signed Follow/Accept/Authorized Fetch between two live instances). +- [x] 11.2 Integration test: post a signed `Follow` to local inbox from a fake Mastodon-shaped client → assert `Accept(Follow)` is delivered + follow row exists + > Done (2026-06-07): handler level in `federation-inbox.integration.test.ts`; full wire level (real signed Follow → Accept delivered + rows on both sides) in the 11.4 harness — a real instance instead of a fake client. Actual-Mastodon behavior was verified live in the 2026-06-06/07 soak. +- [x] 11.3 Integration test: post a `Create(Note)` to local inbox → assert it is dropped silently (no DB writes) + > Done (2026-06-07): in the 11.4 driver — unsigned Create(Note) to the inbox gets 4xx and writes nothing. Signed Creates have no listener (poll-only ingestion by design) and are discarded by Fedify. +- [x] 11.4 Integration test: bring up two trails instances (Docker Compose multi-instance setup); A on instance 1 follows B on instance 2 → assert Follow → Accept → first poll all happen and B's public activity appears in A's `/feed` + > Done (2026-06-07): `e2e/federation/` harness (postgres + caddy internal CA + two journal containers) driven by `federation-two-instance.integration.test.ts` via `run.sh`. Opt-in, not in CI. Caught a real trails↔trails bug: cross-origin embedded Follow objects in Accept/Reject/Undo are distrusted by Fedify and their fragment IRIs dereference to the actor document — listeners now recover via the wire objectId (captured before getObject(), which memoizes) + the personal-inbox recipient. +- [x] 11.5 Integration test: outbound follow attempt against a Mastodon-shaped actor → assert 4xx with the limitation message + > Done earlier in §6: `federation-outbound.integration.test.ts` ("refuses non-trails instances with a clear code"); route returns 400 + `not_trails`. +- [x] 11.6 Integration test (audience leak guard): two local users A and B, only A follows remote trails actor X; X's followers-only post lands in cache. Assert A sees it, B does not + > Done earlier in §8: `social-feed.integration.test.ts` audience-leak guard. +- [x] 11.7 E2E: with feature flag on, follow bruno across instances + see public activity appear + > Done (2026-06-07) by the 11.4 driver at the HTTP/UI boundary: the follow goes through the real `/follows/outgoing` route action with a real session cookie, and the assertion reads the rendered `/feed` HTML (activity name + `@bob@journal-b.test` attribution). A browser-driven repeat would re-test the same path through Playwright; skipped deliberately. ## 12. Rollout