feat(journal): outbox-poll ingestion of remote trails activities (§7)

Tasks 7.1–7.3, 7.5 (+7.4 pacing; Retry-After-duration backoff noted as
remaining). Resolves the activities.owner_id open question.

Schema (design decision from the open question):
- activities.owner_id nullable + check constraint enforcing exactly
  one of (owner_id, remote_actor_iri) — the follows pattern again.
- remote_published_at carries the origin's publish time for the §8
  feed sort; index on remote_actor_iri for the feed join.
- Compiler-audited fallout: notification fan-out, the Note object
  dispatcher, and the activity detail loader explicitly skip/404
  remote rows (their canonical page is the origin; feed links there).
  Every other surface joins users on owner_id and excludes remote rows
  structurally.

Ingestion:
- federation-ingest.server.ts: parseOutboxItem targets exactly our own
  outgoing Create(Note) shape (PropertyValue stats, first-paragraph
  name, audience from to/cc); unknown items skipped, never fatal.
- ingestRemoteActivities: replay-safe via unique remote_origin_iri,
  conflict-streak early exit (outboxes are newest-first).
- pollRemoteActor: signed fetch (Authorized Fetch via a local
  follower's key), outbox resolution via remote_actors cache with
  actor-doc fallback (which refreshes the cache), 1 req/5 s per-host
  pacing, last_polled_at stamping. Network + pacing injectable.

Jobs: poll-remote-actor (per-actor; the §4 Accept listener already
enqueues it as the first poll) + poll-remote-outboxes (5-min cron
sweep over accepted remote follows not polled within the hour).

Tests: parse-shape units; integration suite for ingestion provenance,
audience→visibility mirroring, replay safety, the DB author invariant,
poll flow (actor-doc → collection → page), signer requirement, and
due-polling selection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-06-07 12:05:32 +02:00
parent 9ada6ab7f3
commit b96bef91a9
13 changed files with 628 additions and 21 deletions

View file

@ -144,9 +144,12 @@ export type Audience = "public" | "followers-only";
export const activities = journalSchema.table("activities", {
id: text("id").primaryKey(),
ownerId: text("owner_id")
.notNull()
.references(() => users.id),
// Local author. NULL for activities ingested from a remote trails
// actor's outbox, where `remoteActorIri` identifies the author —
// exactly one of the two is set (check constraint below; same
// pattern as follows.follower_id / follower_actor_iri). Resolves
// the owner_id open question in social-federation design.md.
ownerId: text("owner_id").references(() => users.id),
routeId: text("route_id").references(() => routes.id),
name: text("name").notNull(),
description: text("description").default(""),
@ -164,16 +167,25 @@ export const activities = journalSchema.table("activities", {
// Federation provenance (spec: social-federation). NULL for local
// activities. For rows ingested from a remote trails actor's outbox:
// `remoteOriginIri` is the activity's IRI on the origin instance
// (unique → replay-safe `ON CONFLICT DO NOTHING` ingestion), and
// `remoteActorIri` keys into `remote_actors`.
// (unique → replay-safe `ON CONFLICT DO NOTHING` ingestion),
// `remoteActorIri` keys into `remote_actors`, and
// `remotePublishedAt` carries the origin's publish time (feed sort
// uses COALESCE(remote_published_at, created_at)).
remoteOriginIri: text("remote_origin_iri").unique(),
remoteActorIri: text("remote_actor_iri"),
remotePublishedAt: timestamp("remote_published_at", { withTimezone: true }),
audience: text("audience").$type<Audience>().notNull().default("public"),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
}, (t) => ({
// Hot paths: listActivities (sort by started_at) and listPublicActivitiesForOwner.
ownerStartedIdx: index("activities_owner_started_idx").on(t.ownerId, t.startedAt.desc()),
ownerCreatedIdx: index("activities_owner_created_idx").on(t.ownerId, t.createdAt.desc()),
// Feed join for remote rows (spec §8).
remoteActorIdx: index("activities_remote_actor_idx").on(t.remoteActorIri),
hasAuthorCheck: check(
"activities_has_author_check",
sql`(${t.ownerId} IS NOT NULL) <> (${t.remoteActorIri} IS NOT NULL)`,
),
}));
// --- OAuth2 PKCE (mobile app auth) ---