feat(journal): federation instance blocklist

Task group 3 of federation-hardening. There was no blocklist of any kind;
the only lever against a hostile instance was an IP/host block in Caddy.

- New `federation_blocked_instances` table (domain PK, reason,
  created_at). Additive → created by drizzle-kit push.
- `federation-blocklist.server.ts`: exact-host matching —
  `isBlockedDomain`, `isBlockedIri` (unparseable IRI ⇒ treated as
  blocked), and `filterBlockedDomains` for batch recipient filtering.
- Enforced at all three boundaries (spec: federation-operations
  "Instance blocklist"):
  - inbox — each of the 4 listeners silently drops a blocked actor's
    activity (202, no error oracle) before dedup/side effects;
  - delivery enqueue — `enqueueActivityDeliveries` filters blocked
    recipients in one batch query;
  - outbox poll / actor fetch — `pollRemoteActor` refuses a blocked host
    up front (`skipped: "blocked instance"`), before any network.
- Operator procedure (SQL insert/list/delete) documented in the
  deployment runbook's federation section.
- Tests: unit (hostOfIri) + integration against real Postgres covering
  the helper and the delivery + outbox boundaries; inbox uses the same
  tested isBlockedIri primitive.

Note: the inbox-drop *counter* (federation_inbox_dropped_total{reason})
lands with the other metrics in task 4.2; this commit is the enforcement.

Verified: db + journal typecheck + lint clean; drizzle-kit push creates
the table; blocklist integration tests green against real Postgres;
journal unit suite 357 passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-07-13 22:58:42 +02:00
parent 3bd1f87e63
commit 57696286e4
No known key found for this signature in database
GPG key ID: A32FF691A0F752D9
9 changed files with 234 additions and 7 deletions

View file

@ -0,0 +1,99 @@
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import { inArray } from "drizzle-orm";
import { randomUUID } from "node:crypto";
import { getDb } from "./db.ts";
import { users, follows, federationBlockedInstances } from "@trails-cool/db/schema/journal";
import {
isBlockedDomain,
isBlockedIri,
filterBlockedDomains,
} from "./federation-blocklist.server.ts";
import { enqueueActivityDeliveries } from "./federation-delivery.server.ts";
import { pollRemoteActor, type PollDeps } from "./federation-ingest.server.ts";
import { setBoss } from "./boss.server.ts";
// Opt-in: real Postgres; no instance is contacted (poll deps injected).
const runIntegration = process.env.FEDERATION_INTEGRATION === "1";
const BLOCKED = `blocked-${Date.now()}.example`;
const ALLOWED = `allowed-${Date.now()}.example`;
const userIds: string[] = [];
describe.runIf(runIntegration)("federation instance blocklist (integration)", () => {
beforeAll(async () => {
process.env.FEDERATION_ENABLED = "true";
process.env.ORIGIN ??= "https://test.local";
const db = getDb();
await db.insert(federationBlockedInstances).values({ domain: BLOCKED, reason: "test" });
});
afterAll(async () => {
const db = getDb();
await db.delete(federationBlockedInstances).where(inArray(federationBlockedInstances.domain, [BLOCKED, ALLOWED]));
if (userIds.length) await db.delete(users).where(inArray(users.id, userIds));
setBoss(null);
});
it("matches blocked domains exactly, by host", async () => {
expect(await isBlockedDomain(BLOCKED)).toBe(true);
expect(await isBlockedDomain(ALLOWED)).toBe(false);
expect(await isBlockedDomain(`sub.${BLOCKED}`)).toBe(false); // exact-host, no subdomain wildcard
});
it("isBlockedIri checks the IRI host; unparseable IRIs are treated as blocked", async () => {
expect(await isBlockedIri(`https://${BLOCKED}/users/x`)).toBe(true);
expect(await isBlockedIri(`https://${ALLOWED}/users/x`)).toBe(false);
expect(await isBlockedIri("garbage")).toBe(true);
});
it("filterBlockedDomains resolves a batch in one pass", async () => {
const set = await filterBlockedDomains([BLOCKED, ALLOWED, "another.example"]);
expect([...set]).toEqual([BLOCKED]);
});
it("delivery enqueue: recipients on a blocked domain are filtered out", async () => {
const db = getDb();
const ownerId = randomUUID();
userIds.push(ownerId);
await db.insert(users).values({
id: ownerId,
email: `owner-${ownerId}@example.test`,
username: `owner_${Date.now()}`,
domain: "test.local",
profileVisibility: "public",
});
const now = new Date();
const followedActorIri = `https://test.local/users/owner_${ownerId}`;
await db.insert(follows).values([
{ id: randomUUID(), followedUserId: ownerId, followedActorIri, followerActorIri: `https://${BLOCKED}/users/a`, acceptedAt: now },
{ id: randomUUID(), followedUserId: ownerId, followedActorIri, followerActorIri: `https://${ALLOWED}/users/b`, acceptedAt: now },
]);
const sent: Array<{ recipientActorIri: string }> = [];
setBoss({
async send(_q: string, data: unknown) {
sent.push(data as { recipientActorIri: string });
return "id";
},
} as unknown as Parameters<typeof setBoss>[0]);
await enqueueActivityDeliveries(ownerId, randomUUID(), "create");
// Only the allowed follower gets a delivery job.
expect(sent).toHaveLength(1);
expect(sent[0]?.recipientActorIri).toBe(`https://${ALLOWED}/users/b`);
});
it("outbox poll refuses a blocked instance without fetching", async () => {
const deps: PollDeps = {
async fetchJson() {
throw new Error("must not fetch a blocked instance");
},
async pace() {
throw new Error("must not pace a blocked instance");
},
};
const result = await pollRemoteActor(`https://${BLOCKED}/users/x`, deps);
expect(result).toEqual({ skipped: "blocked instance" });
});
});

View file

@ -0,0 +1,14 @@
import { describe, it, expect } from "vitest";
import { hostOfIri } from "./federation-blocklist.server.ts";
describe("hostOfIri", () => {
it("extracts the host from an actor/object IRI", () => {
expect(hostOfIri("https://mastodon.social/users/alice")).toBe("mastodon.social");
expect(hostOfIri("https://sub.example.com:8443/x")).toBe("sub.example.com:8443");
});
it("returns null for an unparseable IRI", () => {
expect(hostOfIri("not a url")).toBeNull();
expect(hostOfIri("")).toBeNull();
});
});

View file

@ -0,0 +1,54 @@
// Instance blocklist (spec: federation-operations "Instance blocklist").
// A blocked domain is inert in both directions: its inbound activities are
// silently dropped, we send it no deliveries, and we don't fetch its
// actors/outboxes. Matching is exact-host (subdomain wildcarding is
// deferred until a real need). Management is a documented operator SQL
// procedure — see FEDERATION.md — with this table as the seam a future
// admin UI can sit on.
import { inArray, eq } from "drizzle-orm";
import { federationBlockedInstances } from "@trails-cool/db/schema/journal";
import { getDb } from "./db.ts";
/** The host of an actor/object IRI, or null if it doesn't parse. */
export function hostOfIri(iri: string): string | null {
try {
return new URL(iri).host;
} catch {
return null;
}
}
/** Whether an exact host is on the blocklist. */
export async function isBlockedDomain(domain: string): Promise<boolean> {
const db = getDb();
const [row] = await db
.select({ domain: federationBlockedInstances.domain })
.from(federationBlockedInstances)
.where(eq(federationBlockedInstances.domain, domain))
.limit(1);
return row !== undefined;
}
/** Whether an actor/object IRI's host is blocked. Unparseable IRIs are
* treated as blocked we won't process something we can't attribute. */
export async function isBlockedIri(iri: string): Promise<boolean> {
const host = hostOfIri(iri);
if (host === null) return true;
return isBlockedDomain(host);
}
/**
* The subset of `domains` that are blocked, resolved in one query. Used
* by delivery fan-out to filter a batch of recipients without a query per
* recipient.
*/
export async function filterBlockedDomains(domains: string[]): Promise<Set<string>> {
if (domains.length === 0) return new Set();
const db = getDb();
const rows = await db
.select({ domain: federationBlockedInstances.domain })
.from(federationBlockedInstances)
.where(inArray(federationBlockedInstances.domain, domains));
return new Set(rows.map((r) => r.domain));
}

View file

@ -9,6 +9,7 @@ import { getDb } from "./db.ts";
import { federationEnabled } from "./federation.server.ts";
import { enqueueOptional } from "./boss.server.ts";
import { activityObjectIri } from "./federation-objects.server.ts";
import { filterBlockedDomains, hostOfIri } from "./federation-blocklist.server.ts";
export type DeliveryAction = "create" | "delete";
@ -76,7 +77,19 @@ export async function enqueueActivityDeliveries(
action: DeliveryAction,
): Promise<void> {
if (!federationEnabled()) return;
const recipients = await listAcceptedRemoteFollowers(ownerId);
const allRecipients = await listAcceptedRemoteFollowers(ownerId);
if (allRecipients.length === 0) return;
// Blocklist boundary: never enqueue a delivery to a blocked instance
// (spec: federation-operations "Instance blocklist"). Resolve the
// blocked set once, then drop recipients whose host is on it.
const blocked = await filterBlockedDomains(
allRecipients.map(hostOfIri).filter((h): h is string => h !== null),
);
const recipients = allRecipients.filter((iri) => {
const host = hostOfIri(iri);
return host !== null && !blocked.has(host);
});
if (recipients.length === 0) return;
const db = getDb();

View file

@ -16,6 +16,7 @@ import { getDb } from "./db.ts";
import { getOrigin } from "./config.server.ts";
import { getFederation } from "./federation.server.ts";
import { upsertRemoteActor } from "./federation-delivery.server.ts";
import { isBlockedDomain } from "./federation-blocklist.server.ts";
import { logger } from "./logger.server.ts";
/** Spec: fetch at most the 50 most recent items per remote actor. */
@ -268,6 +269,9 @@ export async function pollRemoteActor(
// 7.4: respect an active 429 backoff before doing any work.
const host = new URL(actorIri).host;
if (isHostBackingOff(host)) return { skipped: "host rate-limited (backing off)" };
// Blocklist boundary: never fetch a blocked instance's actor/outbox
// (spec: federation-operations "Instance blocklist").
if (await isBlockedDomain(host)) return { skipped: "blocked instance" };
const db = getDb();

View file

@ -37,6 +37,7 @@ import { localActorIri } from "./actor-iri.ts";
import { PostgresKvStore } from "./federation-kv.server.ts";
import { PgBossMessageQueue } from "./federation-queue.server.ts";
import { markInboundActivityProcessed } from "./federation-replay.server.ts";
import { isBlockedIri } from "./federation-blocklist.server.ts";
import { ensureUserKeypair, loadUserKeypair } from "./federation-keys.server.ts";
import { activityToCreate, activityToNote } from "./federation-objects.server.ts";
import {
@ -272,6 +273,7 @@ function buildFederation(): Federation<void> {
// when the local target is public; otherwise drop (the actor
// already 404s for private users).
if (follow.id == null || follow.actorId == null || follow.objectId == null) return;
if (await isBlockedIri(follow.actorId.href)) return; // blocked instance: silent 202 drop
if (!(await markInboundActivityProcessed(follow.id.href)).fresh) return; // replay: drop
const parsed = ctx.parseUri(follow.objectId);
if (parsed?.type !== "actor") return;
@ -293,6 +295,7 @@ function buildFederation(): Federation<void> {
// Spec 4.3: Undo(Follow) removes the follow row. Other Undos are
// acknowledged and dropped.
if (undo.actorId == null) return;
if (await isBlockedIri(undo.actorId.href)) return; // blocked instance: silent 202 drop
if (undo.id != null && !(await markInboundActivityProcessed(undo.id.href)).fresh) return; // replay: drop
const undoObjectId = undo.objectId; // capture before dereference (see Accept)
const object = await undo.getObject(ctx);
@ -322,6 +325,7 @@ function buildFederation(): Federation<void> {
// Spec 4.4: a remote accepted our outgoing Follow — settle the
// Pending row and trigger the first outbox poll for that actor.
if (accept.actorId == null) return;
if (await isBlockedIri(accept.actorId.href)) return; // blocked instance: silent 202 drop
if (accept.id != null && !(await markInboundActivityProcessed(accept.id.href)).fresh) return; // replay: drop
// Capture the raw object reference BEFORE dereferencing:
// getObject() memoizes the fetched document, after which objectId
@ -362,6 +366,7 @@ function buildFederation(): Federation<void> {
.on(Reject, async (ctx, reject) => {
// Spec 4.5: remote refused our Follow — drop the Pending row.
if (reject.actorId == null) return;
if (await isBlockedIri(reject.actorId.href)) return; // blocked instance: silent 202 drop
if (reject.id != null && !(await markInboundActivityProcessed(reject.id.href)).fresh) return; // replay: drop
const objectId = reject.objectId; // capture before dereference (see Accept)
const object = await reject.getObject(ctx);

View file

@ -140,9 +140,35 @@ document; deliveries signed with the old key fail until then).
(`federationSourceHost`); 429s show in journal logs.
- Outbound is paced (1 req/s delivery, 1 req/5 s polling per host).
- Watch `docker logs <journal>` for `fedify·federation` lines; Loki
picks them up. No per-instance blocklist yet — see
`docs/ideas/instance-administration.md` (manual emergency lever:
block the source IP/host in Caddy).
picks them up.
### Blocking an instance
Blocking a domain makes it inert in **both** directions: its inbound
activities are silently dropped (a 202, no error oracle), we send it no
deliveries, and we never fetch its actors/outboxes. Matching is
exact-host. v1 management is a SQL insert/delete against
`journal.federation_blocked_instances` (the table is the seam a future
admin UI can sit on); protocol/moderation semantics are in
[`FEDERATION.md`](../FEDERATION.md).
```sql
-- Block a hostile instance (exact host, no scheme, no path):
INSERT INTO journal.federation_blocked_instances (domain, reason)
VALUES ('bad.example', 'spam / harassment')
ON CONFLICT (domain) DO NOTHING;
-- List current blocks:
SELECT domain, reason, created_at FROM journal.federation_blocked_instances ORDER BY created_at DESC;
-- Unblock:
DELETE FROM journal.federation_blocked_instances WHERE domain = 'bad.example';
```
The block takes effect immediately (checked per-request/per-job, no
cache). Already-queued deliveries to the domain drain from Fedify's
queue; new fan-outs filter it out. An IP/host block in Caddy remains the
harder emergency lever for a flood that shouldn't reach the app at all.
### Troubleshooting deliveries

View file

@ -12,9 +12,9 @@
## 3. Blocklist
- [ ] 3.1 Add `federation_blocked_instances` table + migration; exact-host check helper
- [ ] 3.2 Enforce at inbox (silent 202 drop + counter), delivery enqueue (filter recipients), and outbox poll/actor fetch (refuse)
- [ ] 3.3 Document the operator procedure (SQL insert/delete) in the ops docs; tests for all three boundaries
- [x] 3.1 Add `federation_blocked_instances` table + migration; exact-host check helper
- [x] 3.2 Enforce at inbox (silent 202 drop + counter), delivery enqueue (filter recipients), and outbox poll/actor fetch (refuse)
- [x] 3.3 Document the operator procedure (SQL insert/delete) in the ops docs; tests for all three boundaries
## 4. Protocol doc & observability

View file

@ -439,6 +439,18 @@ export const federationProcessedActivities = journalSchema.table("federation_pro
receivedAtIdx: index("federation_processed_activities_received_at_idx").on(t.receivedAt),
}));
// Instance blocklist (spec: federation-operations "Instance blocklist").
// One row per blocked domain, matched exactly by host. Enforced at three
// boundaries: inbox (activities silently dropped), delivery enqueue
// (recipients filtered), and outbox poll / actor fetch (refused). v1
// management is a documented SQL insert/delete (see FEDERATION.md); the
// table is the API a future admin UI can sit on.
export const federationBlockedInstances = journalSchema.table("federation_blocked_instances", {
domain: text("domain").primaryKey(),
reason: text("reason"),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
});
// Cache of remote ActivityPub actors we interact with (spec:
// social-federation). One row per actor IRI: display fields for feed
// cards, inbox/outbox URLs for delivery and polling, the public key for