trails-cool/trails
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(journal): two-instance federation harness + trails↔trails Accept fix (§11)

Browse source 
The 11.4 harness — e2e/federation/: postgres (two DBs) + a Caddy with
an internal CA terminating real HTTPS between two complete journal
containers (journal-a.test / journal-b.test), driven by an opt-in
integration test (FEDERATION_TWO_INSTANCE=1, via run.sh). The driver
seeds users straight into each DB, mints a signed session cookie
(known SESSION_SECRET), follows across instances through the real
/follows/outgoing route, and asserts the full pipeline: Follow →
auto-Accept → settle → first outbox poll → bob's public activity
rendered in alice's /feed with @bob@journal-b.test attribution. Plus a
wire-level 11.3 check: unsigned Create(Note) → 4xx, no DB writes.

And the harness immediately earned its keep — it caught a real
trails↔trails bug Mastodon interop never could: our Follow ids are
fragment URIs on OUR domain, so the Follow embedded in another trails
instance's Accept is cross-origin and Fedify rightly distrusts it;
re-fetching the id returns the actor document, getObject() yields a
Person, and the Accept/Reject/Undo listeners bailed silently — follows
stayed Pending forever. The listeners now fall back to the wire
objectId (captured BEFORE getObject(), which memoizes the fetched
document and changes what objectId reports) validated against our
Follow-id shape + the personal inbox's ctx.recipient. Forgery-safe:
settleOutgoingFollow only matches a Pending row toward the
HTTP-Signature-authenticated sender.

Supporting changes:
- federation.server.ts: env-gated allowPrivateAddress
  (FEDERATION_ALLOW_PRIVATE_ADDRESS=true) so the harness's RFC 1918
  Docker network is fetchable — testing only, loudly documented.
- Root .dockerignore: local builds shipped a 1GB+ context and
  overlaid host (darwin) node_modules over the image's own install
  via COPY . . — CI never noticed because it builds from clean
  checkouts. Context is now ~5MB.
- tasks.md: 11.1–11.7 marked with provenance notes; design.md gains
  the cross-origin embedded-object lesson.

Gate: typecheck ✓ lint ✓ unit+integration (FEDERATION_INTEGRATION=1) ✓
e2e 71/72 + known komoot flake green isolated ✓ harness run clean from
scratch (2/2, teardown included) ✓

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-06-07 13:36:35 +02:00
parent 314196847c
commit 1eceb6f1f7
10 changed files with 610 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

203
apps/journal/app/lib/federation-two-instance.integration.test.ts Normal file
View file

@ -0,0 +1,203 @@
// Two-instance federation drive-through (spec: social-federation 11.4).
//
// Talks to the docker-compose harness in e2e/federation/ — two complete
// journal instances (journal-a.test / journal-b.test) federating over a
// private Docker network with real HTTPS between them. This test is the
// driver: it seeds users straight into each instance's database, mints
// a session cookie for alice (cookie sessions are signed with the
// harness's known SESSION_SECRET), follows bob across instances through
// the real /follows/outgoing route, and then watches the entire
// Follow → Accept → first outbox poll → feed pipeline happen between
// two live servers.
//
// Run via e2e/federation/run.sh — it builds the images, pushes the
// schema into both databases, and sets FEDERATION_TWO_INSTANCE=1.
import { describe, it, expect, afterAll } from "vitest";
import postgres, { type Sql } from "postgres";
import { randomUUID } from "node:crypto";
import { createCookieSessionStorage } from "react-router";
const runTwoInstance = process.env.FEDERATION_TWO_INSTANCE === "1";
// Host-published harness endpoints (see e2e/federation/docker-compose.yml).
const A_URL = "http://127.0.0.1:3401";
const B_URL = "http://127.0.0.1:3402";
const DB_A = "postgres://trails:trails@127.0.0.1:5499/trails_a";
const DB_B = "postgres://trails:trails@127.0.0.1:5499/trails_b";
const B_DOMAIN = "journal-b.test";
const BOB_ACTOR_IRI = `https://${B_DOMAIN}/users/bob`;
const ACTIVITY_NAME = "Sunrise ride over the ridge";
/** Poll `fn` until it returns something truthy. */
async function until<T>(
what: string,
fn: () => Promise<T | null | undefined | false>,
timeoutMs = 120_000,
): Promise<T> {
const start = Date.now();
for (;;) {
const value = await fn();
if (value) return value;
if (Date.now() - start > timeoutMs) throw new Error(`Timed out waiting for ${what}`);
await new Promise((r) => setTimeout(r, 1500));
}
}
async function seedUser(sql: Sql, username: string, domain: string): Promise<string> {
const id = randomUUID();
await sql`
INSERT INTO journal.users (id, email, username, domain, profile_visibility, terms_accepted_at, terms_version)
VALUES (${id}, ${`${username}@${domain}`}, ${username}, ${domain}, 'public', now(), '2026-04-19')
`;
return id;
}
/**
* Mint a valid `__session` cookie for a user. The harness journals run
* with a known SESSION_SECRET, and sessions are signed cookies (see
* auth/session.server.ts) — so the driver can authenticate as any
* seeded user without driving the WebAuthn ceremony.
*/
async function mintSessionCookie(userId: string): Promise<string> {
const storage = createCookieSessionStorage({
cookie: {
name: "__session",
httpOnly: true,
sameSite: "lax",
path: "/",
secrets: ["two-instance-test-secret"],
},
});
const session = await storage.getSession();
session.set("userId", userId);
const setCookie = await storage.commitSession(session);
return setCookie.split(";")[0]!;
}
describe.runIf(runTwoInstance)("two-instance federation (11.4)", () => {
const sqlA = postgres(DB_A, { max: 2 });
const sqlB = postgres(DB_B, { max: 2 });
afterAll(async () => {
await sqlA.end();
await sqlB.end();
});
it(
"alice@journal-a follows bob@journal-b: Follow → Accept → first poll → bob's public activity in alice's feed",
{ timeout: 300_000 },
async () => {
// Both instances are up (run.sh already gated on the compose
// healthchecks; this is a fail-fast sanity check).
for (const base of [A_URL, B_URL]) {
const health = await fetch(`${base}/api/health`);
expect(health.status).toBe(200);
}
// ---- Seed: alice on A; bob + a public activity on B ----------
const aliceId = await seedUser(sqlA, "alice", "journal-a.test");
const bobId = await seedUser(sqlB, "bob", B_DOMAIN);
await sqlB`
INSERT INTO journal.activities (id, owner_id, name, description, visibility)
VALUES (${randomUUID()}, ${bobId}, ${ACTIVITY_NAME}, '', 'public')
`;
// B serves bob over WebFinger before we point A at him. (Fedify's
// canonical-origin support means the host-port URL works here.)
const wf = await fetch(`${B_URL}/.well-known/webfinger?resource=acct:bob@${B_DOMAIN}`);
expect(wf.status).toBe(200);
// ---- alice follows bob through the real route -----------------
const cookie = await mintSessionCookie(aliceId);
const follow = await fetch(`${A_URL}/follows/outgoing`, {
method: "POST",
redirect: "manual",
headers: { cookie, "content-type": "application/x-www-form-urlencoded" },
body: new URLSearchParams({ intent: "follow", handle: `bob@${B_DOMAIN}` }),
});
if (follow.status >= 400) {
// FollowError surfaces as a 400 with a code — bubble the body
// up so a harness failure says *why* resolution failed.
throw new Error(`follow action failed: ${follow.status} ${await follow.text()}`);
}
// Outbound resolution (NodeInfo + WebFinger + actor fetch over the
// Caddy TLS hop) happens inside the action — a Pending row is the
// proof it all worked.
const pending = await until("pending follow row on A", async () => {
const rows = await sqlA`
SELECT id, accepted_at FROM journal.follows
WHERE follower_id = ${aliceId} AND followed_actor_iri = ${BOB_ACTOR_IRI}
`;
return rows[0];
}, 30_000);
expect(pending).toBeDefined();
// ---- B auto-accepts (public profile) and delivers Accept ------
await until("Accept to settle the follow on A", async () => {
const rows = await sqlA`
SELECT accepted_at FROM journal.follows
WHERE follower_id = ${aliceId} AND followed_actor_iri = ${BOB_ACTOR_IRI}
`;
return rows[0]?.accepted_at != null;
});
// B's side sees a remote follower row for alice.
const bFollow = await sqlB`
SELECT follower_actor_iri FROM journal.follows WHERE followed_user_id = ${bobId}
`;
expect(bFollow[0]?.follower_actor_iri).toBe("https://journal-a.test/users/alice");
// ---- First poll ingests bob's outbox into A -------------------
const ingested = await until("bob's activity to be ingested on A", async () => {
const rows = await sqlA`
SELECT name, audience, owner_id FROM journal.activities
WHERE remote_actor_iri = ${BOB_ACTOR_IRI}
`;
return rows[0];
});
expect(ingested.name).toBe(ACTIVITY_NAME);
expect(ingested.audience).toBe("public");
expect(ingested.owner_id).toBeNull();
// ---- And it shows up in alice's rendered feed ------------------
const feed = await fetch(`${A_URL}/feed`, { headers: { cookie } });
expect(feed.status).toBe(200);
// JSX puts `<!-- -->` separators between adjacent text expressions
// — strip them so the attribution reads as continuous text.
const html = (await feed.text()).replaceAll("<!-- -->", "");
expect(html).toContain(ACTIVITY_NAME);
expect(html).toContain(`@bob@${B_DOMAIN}`);
},
);
it("an unsigned Create(Note) posted to the inbox is rejected with no DB writes (11.3)", async () => {
const originIri = `https://intruder.example/notes/${randomUUID()}`;
const res = await fetch(`${A_URL}/users/alice/inbox`, {
method: "POST",
headers: { "content-type": "application/activity+json" },
body: JSON.stringify({
"@context": "https://www.w3.org/ns/activitystreams",
id: `${originIri}#create`,
type: "Create",
actor: "https://intruder.example/users/mallory",
to: "https://www.w3.org/ns/activitystreams#Public",
object: {
id: originIri,
type: "Note",
content: "<p>Injected note</p>",
},
}),
});
// Unauthenticated inbox delivery: Fedify refuses it outright
// (400 malformed / 401 unsigned — both fine, never 2xx).
expect(res.status).toBeGreaterThanOrEqual(400);
expect(res.status).toBeLessThan(500);
const rows = await sqlA`
SELECT id FROM journal.activities WHERE remote_origin_iri = ${originIri}
`;
expect(rows).toHaveLength(0);
});
});

96
apps/journal/app/lib/federation.server.ts
View file

@ -97,6 +97,20 @@ async function findLocalPublicUserByIri(iri: URL | null): Promise<UserRow | null
return user && user.profileVisibility === "public" ? user : null;
}
/**
* Whether `id` is one of OUR outgoing Follow activity ids
* (`<actorIri>#follows/<uuid>`, see federation-outbound.server.ts) and
* names the recipient of the personal inbox the activity arrived in.
* Used by the Accept/Reject listeners: another trails instance can't
* embed a trustworthy copy of our Follow (its id is cross-origin for
* them), so the id is the only recoverable reference.
*/
function isRecipientFollowUri(ctx: { recipient: string | null }, id: URL | null): boolean {
if (id == null || ctx.recipient == null) return false;
if (!id.hash.startsWith("#follows/")) return false;
return id.href.startsWith(`${localActorIri(ctx.recipient)}#`);
}
let _federation: Federation<void> | null = null;
let _logtapeConfigured = false;
@ -146,6 +160,15 @@ function buildFederation(): Federation<void> {
// Canonical origin so generated IRIs are correct behind the Caddy
// proxy (the Node server itself only sees plain HTTP).
origin: getOrigin(),
// SSRF-guard opt-out for the two-instance federation harness
// (e2e/federation/), where both instances live on a private Docker
// network that Fedify's document loader rightly refuses to fetch
// from. Testing only — never set this in a real deployment: the
// private-address block is a genuine security boundary (a malicious
// actor IRI must not be able to point us at internal targets).
...(process.env.FEDERATION_ALLOW_PRIVATE_ADDRESS === "true"
? { allowPrivateAddress: true }
: {}),
});
if (!process.env.VITEST) {
// Fire-and-forget: runs the queue consumer loop for the process
@ -261,25 +284,66 @@ function buildFederation(): Federation<void> {
.on(Undo, async (ctx, undo) => {
// Spec 4.3: Undo(Follow) removes the follow row. Other Undos are
// acknowledged and dropped.
if (undo.actorId == null) return;
const undoObjectId = undo.objectId; // capture before dereference (see Accept)
const object = await undo.getObject(ctx);
if (!(object instanceof Follow)) return;
if (undo.actorId == null || object.objectId == null) return;
const parsed = ctx.parseUri(object.objectId);
if (parsed?.type !== "actor") return;
await removeRemoteFollow(undo.actorId.href, parsed.identifier);
if (object instanceof Follow && object.objectId != null) {
const parsed = ctx.parseUri(object.objectId);
if (parsed?.type !== "actor") return;
await removeRemoteFollow(undo.actorId.href, parsed.identifier);
return;
}
// trails→trails fallback: another trails instance's Undo embeds
// a Follow whose id lives on the SENDER's domain, but the embed
// is cross-origin relative to nothing we can verify, and
// dereferencing `…#follows/<id>` yields the sender's actor
// document, not a Follow (fragments aren't separately fetchable).
// Our Follow ids are `<actorIri>#follows/<uuid>` — when the Undo
// names one owned by the authenticated sender, the recipient of
// this personal inbox is the unfollowed user.
if (
ctx.recipient != null &&
undoObjectId?.hash.startsWith("#follows/") &&
undoObjectId.origin === new URL(undo.actorId.href).origin
) {
await removeRemoteFollow(undo.actorId.href, ctx.recipient);
}
})
.on(Accept, async (ctx, accept) => {
// Spec 4.4: a remote accepted our outgoing Follow — settle the
// Pending row and trigger the first outbox poll for that actor.
const object = await accept.getObject(ctx);
if (!(object instanceof Follow)) return;
if (accept.actorId == null) return;
const localUser = await findLocalPublicUserByIri(object.actorId);
// Capture the raw object reference BEFORE dereferencing:
// getObject() memoizes the fetched document, after which objectId
// reports the fetched object's id (fragment stripped) instead of
// the id that was on the wire.
const objectId = accept.objectId;
const object = await accept.getObject(ctx);
logger.debug(
{
recipient: ctx.recipient,
objectId: objectId?.href ?? null,
objectType: object?.constructor?.name ?? null,
},
"federation: Accept listener dispatch",
);
let localUser: Awaited<ReturnType<typeof findLocalPublicUserByIri>> = null;
if (object instanceof Follow) {
localUser = await findLocalPublicUserByIri(object.actorId);
} else if (isRecipientFollowUri(ctx, objectId)) {
// trails→trails: our own Follow id is cross-origin from the
// accepting instance's perspective, so Fedify rightly distrusts
// the embedded copy and a re-fetch of `…#follows/<id>` returns
// our actor document instead of a Follow. The id itself names
// this inbox's recipient, which is all settling needs — and
// settleOutgoingFollow only matches a Pending row toward the
// authenticated sender, so a forged Accept can't settle
// anything that wasn't already directed at that actor.
localUser = await findLocalPublicUserByIri(new URL(localActorIri(ctx.recipient!)));
}
if (!localUser) return;
const { settled } = await settleOutgoingFollow(localUser.id, accept.actorId.href);
if (settled) {
// Queue worker lands in the outbox-poll change (task 7.x);
// enqueueOptional logs-and-continues until then.
await enqueueOptional("poll-remote-actor", { actorIri: accept.actorId.href }, {
reason: "first poll after accepted follow",
});
@ -287,10 +351,16 @@ function buildFederation(): Federation<void> {
})
.on(Reject, async (ctx, reject) => {
// Spec 4.5: remote refused our Follow — drop the Pending row.
const object = await reject.getObject(ctx);
if (!(object instanceof Follow)) return;
if (reject.actorId == null) return;
const localUser = await findLocalPublicUserByIri(object.actorId);
const objectId = reject.objectId; // capture before dereference (see Accept)
const object = await reject.getObject(ctx);
let localUser: Awaited<ReturnType<typeof findLocalPublicUserByIri>> = null;
if (object instanceof Follow) {
localUser = await findLocalPublicUserByIri(object.actorId);
} else if (isRecipientFollowUri(ctx, objectId)) {
// Same trails→trails fallback as the Accept listener above.
localUser = await findLocalPublicUserByIri(new URL(localActorIri(ctx.recipient!)));
}
if (!localUser) return;
await rejectOutgoingFollow(localUser.id, reject.actorId.href);
})