feat(journal): durable Fedify message queue over pg-boss
Task group 1 of federation-hardening. Fedify was configured with `InProcessMessageQueue`, so every queued outbound delivery, its pending retry state, and inbox processing task was lost on a container restart (routine here: deploys, OOM history) — directly contradicting the social-federation spec's promise that fan-out survives a deploy. - `federation-queue.server.ts`: `PgBossMessageQueue` implementing Fedify's `MessageQueue` over the pg-boss instance the journal already runs, mirroring the `PostgresKvStore` adapter. `nativeRetrial = false` keeps Fedify the retry-policy owner; pg-boss supplies durability + delayed jobs (delay → whole-second `startAfter`, `retryLimit: 0`). - Swap it in for `InProcessMessageQueue` in `federation.server.ts`; document the two intentional queueing layers (our fan-out jobs feed Fedify; Fedify's sends now durable underneath). - `server.ts`: create the durable queue at startup when federation is on. - Broaden the structural `BossLike` in `boss.server.ts` with the work/offWork/createQueue/getQueue methods the adapter needs. - Tests: enqueue maps retry/delay correctly, consume roundtrip, restart durability (fresh listener drains a prior instance's backlog), abort stops the worker, depth reports ready vs delayed. Verified: journal typecheck + lint clean, 7/7 new unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
1ea2559246
commit
b8fc8fadff
6 changed files with 252 additions and 12 deletions
77
apps/journal/app/lib/federation-queue.server.ts
Normal file
77
apps/journal/app/lib/federation-queue.server.ts
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
// Postgres-backed Fedify MessageQueue, implemented over the pg-boss
|
||||
// instance the journal already runs (packages/jobs). Fedify uses this
|
||||
// queue for outbound delivery (with its own retry scheduling) and inbox
|
||||
// processing; the previous InProcessMessageQueue lost every queued
|
||||
// message on restart (spec: federation-operations "Durable federation
|
||||
// queue"). pg-boss gives us durable storage + delayed jobs; Fedify stays
|
||||
// the retry-policy owner (see `nativeRetrial` below).
|
||||
//
|
||||
// This mirrors the PostgresKvStore adapter pattern (federation-kv.server.ts):
|
||||
// a thin Fedify-interface shim over infrastructure we already operate,
|
||||
// resolved lazily via getBoss() so it works in both the server-bootstrap
|
||||
// module and the request-path bundle (see boss.server.ts on the two-copy
|
||||
// module situation).
|
||||
//
|
||||
// Fedify fans a single MessageQueue instance out to three logical roles
|
||||
// (inbox / outbox / fanout) and calls listen() once per role with an
|
||||
// equivalent type-dispatching handler. Backing all three with one pg-boss
|
||||
// queue is correct: any worker can process any message because Fedify
|
||||
// routes by message type internally.
|
||||
|
||||
import type {
|
||||
MessageQueue,
|
||||
MessageQueueEnqueueOptions,
|
||||
MessageQueueListenOptions,
|
||||
MessageQueueDepth,
|
||||
} from "@fedify/fedify";
|
||||
import { getBoss } from "./boss.server.ts";
|
||||
|
||||
/** pg-boss queue name backing Fedify's inbox/outbox/fanout message queue. */
|
||||
export const FEDERATION_QUEUE_NAME = "federation.fedify";
|
||||
|
||||
export class PgBossMessageQueue implements MessageQueue {
|
||||
// Fedify owns retry scheduling: on a failed delivery it re-enqueues with
|
||||
// its own backoff policy, so pg-boss must NOT also retry (retryLimit 0
|
||||
// on enqueue) or every failure would be attempted twice.
|
||||
readonly nativeRetrial = false;
|
||||
|
||||
async enqueue(message: unknown, options?: MessageQueueEnqueueOptions): Promise<void> {
|
||||
// Fedify passes delay as a Temporal.Duration; pg-boss startAfter is
|
||||
// whole seconds. Round up so a sub-second backoff still defers.
|
||||
const seconds = options?.delay ? Math.ceil(options.delay.total("seconds")) : 0;
|
||||
await getBoss().send(FEDERATION_QUEUE_NAME, message as object, {
|
||||
retryLimit: 0,
|
||||
...(seconds > 0 ? { startAfter: seconds } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
async listen(
|
||||
handler: (message: unknown) => Promise<void> | void,
|
||||
options?: MessageQueueListenOptions,
|
||||
): Promise<void> {
|
||||
const boss = getBoss();
|
||||
await boss.work(FEDERATION_QUEUE_NAME, async (jobs) => {
|
||||
for (const job of jobs) await handler(job.data);
|
||||
});
|
||||
// Fedify's contract: listen() resolves when the signal aborts and never
|
||||
// rejects; with no signal it never resolves (runs for the process
|
||||
// lifetime, consuming the queue).
|
||||
return new Promise<void>((resolve) => {
|
||||
const signal = options?.signal;
|
||||
if (signal == null) return;
|
||||
const stop = () => void boss.offWork(FEDERATION_QUEUE_NAME).finally(() => resolve());
|
||||
if (signal.aborted) stop();
|
||||
else signal.addEventListener("abort", stop, { once: true });
|
||||
});
|
||||
}
|
||||
|
||||
async getDepth(): Promise<MessageQueueDepth> {
|
||||
const queue = await getBoss().getQueue(FEDERATION_QUEUE_NAME);
|
||||
if (queue == null) return { queued: 0, ready: 0, delayed: 0 };
|
||||
return {
|
||||
queued: queue.queuedCount,
|
||||
ready: queue.readyCount,
|
||||
delayed: queue.deferredCount,
|
||||
};
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue