Tasks 7.1–7.3, 7.5 (+7.4 pacing; Retry-After-duration backoff noted as
remaining). Resolves the activities.owner_id open question.
Schema (design decision from the open question):
- activities.owner_id nullable + check constraint enforcing exactly
one of (owner_id, remote_actor_iri) — the follows pattern again.
- remote_published_at carries the origin's publish time for the §8
feed sort; index on remote_actor_iri for the feed join.
- Compiler-audited fallout: notification fan-out, the Note object
dispatcher, and the activity detail loader explicitly skip/404
remote rows (their canonical page is the origin; feed links there).
Every other surface joins users on owner_id and excludes remote rows
structurally.
Ingestion:
- federation-ingest.server.ts: parseOutboxItem targets exactly our own
outgoing Create(Note) shape (PropertyValue stats, first-paragraph
name, audience from to/cc); unknown items skipped, never fatal.
- ingestRemoteActivities: replay-safe via unique remote_origin_iri,
conflict-streak early exit (outboxes are newest-first).
- pollRemoteActor: signed fetch (Authorized Fetch via a local
follower's key), outbox resolution via remote_actors cache with
actor-doc fallback (which refreshes the cache), 1 req/5 s per-host
pacing, last_polled_at stamping. Network + pacing injectable.
Jobs: poll-remote-actor (per-actor; the §4 Accept listener already
enqueues it as the first poll) + poll-remote-outboxes (5-min cron
sweep over accepted remote follows not polled within the hour).
Tests: parse-shape units; integration suite for ingestion provenance,
audience→visibility mirroring, replay safety, the DB author invariant,
poll flow (actor-doc → collection → page), signer requirement, and
due-polling selection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
10.1 Privacy manifest (legal/privacy):
- German legal half: Föderation entry under Empfänger/Drittanbieter —
what a public profile exposes, what accepted followers' servers
receive, the loss-of-control over delivered copies, Art. 6(1)(a)
basis, private profiles don't federate.
- English manifest half: dedicated Federation (ActivityPub) section —
actor object contents, push delivery + retraction semantics (incl.
the tombstone caveat), encrypted-at-rest signing keys, remote actor
cache, ingested remote content with the followers-only viewer gate,
inbox logging/rate limits. PRIVACY_LAST_UPDATED bumped.
10.2 docs/deployment.md federation runbook: enabling per environment,
key rotation posture, abuse monitoring, and the troubleshooting
checklist distilled from the 2026-06-06/07 soak (IP families first,
no outbox backfill, tombstones, the 10s delivery timeout, actor
re-fetch).
10.3 Home marketing blurb (en+de) aligned to what actually ships:
Mastodon can follow you + trails-to-trails outbound — no more 'follow
friends anywhere' overstatement.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tasks 6.1–6.6. A local user can follow a user on another trails
instance: WebFinger-resolve the handle, verify the target runs
trails.cool via NodeInfo — checked against the ACTOR IRI's host, never
the handle's domain (split-domain constraint) — record a Pending
follow row, deliver a signed Follow. The §4 inbox listeners already
settle (Accept) or drop (Reject) the row.
- federation-outbound.server.ts: followRemoteActor / cancelRemoteFollow
(Undo delivery) / listOutgoingRemoteFollows. Network steps (lookup,
NodeInfo, delivery) injectable for offline integration tests.
Software allowlist: trails-cool.
- /follows/outgoing: follow-by-handle form + pending/accepted list +
cancel; linked from the feed empty state; i18n en+de. Remote actors
have no local profile page, so the remote Pending state lives here
(the profile Pending button from locked accounts covers local).
- /.well-known/trails-cool now publishes software: trails-cool (6.2).
- Clear 4xx codes: invalid_handle, local_handle, remote_resolve_failed,
not_trails (6.3).
- Tests: handle-parser + allowlist units; integration suite for the
Pending lifecycle (create/idempotent/refuse-non-trails/refuse-
unresolvable/own-host/cancel+Undo) with injected network deps; e2e
anonymous-redirect guard for the new page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 2.2.x line was uninstallable when federation landed
(@fedify/webfinger@2.2.x wasn't published); fixed upstream, so move to
latest. Exact pin stays deliberate — a federation library defines the
wire shapes remote instances see.
Verified: typecheck, lint, full unit + integration suites (245 tests,
including the wire-shape assertions: JRD, actor attachment arrays,
PropertyValue fields, dereferenceable Note ids) all green on 2.2.5.
Lockfile diff is a clean 24-line swap of the fedify package family.
Note: a transient cross-suite flake surfaced when running both
integration suites concurrently (shared journal.follows state) —
pre-existing, unrelated to the upgrade; clean re-run green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
countFollowers counts every accepted follow row, but listFollowers
inner-joined users on follower_id — so a federated follower (NULL
follower_id, follower_actor_iri set) bumped the count while never
appearing in the list. Observed live: profile said '1 follower', list
below was empty. listFollowing had the same latent bug for outbound
trails-to-trails follows.
Both lists now left-join users + remote_actors: local entries link to
the local profile as before; remote entries display cached actor data
(IRI parsing as fallback) and link out to the remote profile.
Integration test pins count/list consistency for a remote follower.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A received Delete is recorded by Mastodon as a permanent tombstone —
later Creates for the same URI are silently refused forever. The old
code sent Delete on every non-public save 'just in case' (the comment
even called over-sending harmless); on the 2026-06-07 soak this
tombstoned an unlisted activity's URI before its first real publish,
making its later flip to public invisible on the remote with no error
anywhere.
- visibilityTransitionAction(previous, next): Create on any transition
to public (re-publish doubles as back-delivery; remotes dedupe by
id), Delete only when leaving public, nothing for
non-public→non-public.
- updateActivityVisibility reads the previous visibility and acts on
the transition.
- design.md documents the tombstone permanence + the user-facing
consequence (un-publish then re-publish won't resurrect the post on
remotes that processed the retraction — same as Mastodon's own
delete-and-redraft).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mastodon's PropertyValue parser requires the actor's attachment to be
a JSON *array* and silently ignores a bare object — and Fedify
compacts single-element arrays to bare objects, so the 🥾 trails.cool
field shipped in #464 never rendered (verified: Mastodon stored
fields = {} after a forced actor refresh on the soak instance).
Ship two fields so the array survives serialization: the 🥾 profile
link (rel=me) plus an Instance link — the latter is genuinely useful
for self-hosted instances anyway. Test now asserts the array shape
explicitly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In production there are two copies of boss.server.ts in the process:
server.ts imports the TypeScript source directly (node
--experimental-strip-types) while route handlers live in the bundled
build/server/index.js with its own module instance. setBoss() from
server.ts wrote a module-local variable the bundle never saw, so EVERY
request-path enqueue failed with 'pg-boss not initialized' and was
swallowed by enqueueOptional's best-effort catch:
- notifications fan-out on activity publish: silently dropped
- komoot bulk-import kick-off: silently dropped
- federation activity push delivery: silently dropped (how we found
it — flipping an activity public on staging delivered nothing)
Dev never reproduces this: vite serves one module graph. Caught live
during the social-federation staging soak.
Fix: store the instance on globalThis under a Symbol.for() key so both
module copies resolve the same boss.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Notes we emit (outbox + push delivery) use /activities/{id} as
their id, but that URL only served HTML — so Mastodon's search-fetch
of an activity URL failed, and strict instances that re-fetch pushed
objects to verify them would drop our deliveries.
- Fedify object dispatcher for Note at /activities/{id}: serves the
same activityToNote mapping used everywhere else; 404 unless the
activity is public AND the owner's profile is public (private owners
don't federate, mirroring the actor).
- Content-negotiation middleware on the activity detail route, same
pattern as the actor route: AP Accept headers short-circuit to
Fedify, browsers get HTML.
Found during the staging soak (Mastodon showed the outbox post count
but couldn't fetch the posts).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mastodon gives inbox deliveries a 10s read timeout; without a queue,
Fedify runs inbox listeners AND outbound deliveries (the Accept we
push back after a Follow) inside the inbound request. A slow or
unreachable remote then blows the budget, the sender times out and
its circuit breaker (Stoplight on Mastodon) cuts us off — observed
live during the staging soak.
InProcessMessageQueue + manuallyStartQueue: the queue consumer starts
explicitly on first federation use (skipped under vitest so tests
don't leak timers). In-process queueing is acceptable for the
single-process journal: queued work lost on restart is recoverable
(remotes retry Follows; activity push delivery is owned by our own
pg-boss jobs). Revisit with a Postgres-backed MessageQueue if that
changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mastodon renders PropertyValue attachments as the profile metadata
table — adds a '🥾 trails.cool' field linking to the user's canonical
profile, so it's human-visible that an account is a trails profile.
(The machine-readable signal stays NodeInfo; this is flair.) The link
carries rel=me so Mastodon can verify it once our HTML profile links
back.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Inbound Mastodon Follows are being rejected 401 (signature
verification) on staging and Fedify's diagnostics were invisible:
it logs through LogTape, which is silent until configured.
- Configure LogTape with a console sink for the 'fedify' category when
the federation instance is built; level via FEDERATION_LOG_LEVEL
(default info).
- Staging deploy sets FEDERATION_LOG_LEVEL=debug for the soak —
signature-verification detail per request; dial down once proven.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
social-federation tasks 5.1–5.6. Completes the inbound-federation
story: a Mastodon follower now receives a trails user's new public
activities in their home timeline.
Outbox (5.1/5.2):
- /users/:username/outbox — paginated OrderedCollection of public
activities as Create(Note), newest first; unlisted/private never
federate. Private-user 404 enforced at the route layer because
Fedify builds collection-level responses from counter/cursors
without consulting the page dispatcher.
- Note shape: HTML content (escaped name/description/stats + link to
the activity page) with structured PropertyValue attachments
(distance-m, elevation-gain-m, duration-s) — Mastodon renders the
text, trails consumers read the structured fields. Resolves the
design open question toward Create(Note).
- Authorized Fetch: signed and unsigned outbox fetches deliberately
see the same (public-only) content until locked accounts exist.
Push delivery (5.3–5.6):
- createActivity / updateActivityVisibility(→public) enqueue one
deliver-activity job per accepted remote follower; flips away from
public and hard deletes enqueue Delete(Tombstone) retractions
(enqueued before the row disappears).
- deliver-activity job: re-reads the row at delivery time (skips if
gone or no longer public), resolves the recipient inbox via the
remote_actors cache with actor-document fetch fallback (priming the
cache), HTTP-signs via the owner's key, and POSTs. retryLimit 8 +
exponential backoff at enqueue time; outbound paced at 1 req/s per
remote host.
- Actor objects now advertise the outbox IRI.
- @js-temporal/polyfill added (same range Fedify uses) for published
timestamps; Fedify's types want the global esnext.temporal namespace,
bridged with a documented cast.
Tests: 9 unit tests for the AS mapping (escaping, stats, attachments,
published fallback, stable ids, tombstones), 4 outbox integration
tests (collection count, page shape/visibility filtering, private-404,
delivery audience query).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
social-federation tasks 3.1–3.4, 4.1–4.8. With this, a Mastodon user
can follow a public trails user: WebFinger → actor fetch → signed
Follow → recorded + Accept(Follow) pushed back.
Identity surface (section 3):
- Actor objects now carry the user's public key (publicKey +
assertionMethods via Fedify key pairs dispatcher; keys generated
lazily as a fallback to the backfill) and an inbox IRI; url uses
localActorIri (3.1).
- Software discovery shipped as standard NodeInfo
(/.well-known/nodeinfo + /nodeinfo/2.1, software.name trails-cool)
instead of the originally-sketched custom AS actor field — Fedify's
typed vocab can't emit arbitrary actor props and NodeInfo is what
the fediverse reads. Artifacts updated accordingly (3.4).
Inbox (section 4):
- /users/:username/inbox resource route; HTTP Signatures verified by
Fedify before any listener runs (4.1). Rate-limited 60 req/5 min per
source instance (host from Signature keyId) BEFORE verification so
hostile instances can't burn CPU on key fetches (4.8).
- Listeners: Follow → auto-accept for public profiles + Accept pushed
back (4.2); Undo(Follow) → row removed (4.3); Accept(Follow) →
Pending settled + first outbox poll enqueued (4.4); Reject(Follow) →
Pending dropped (4.5; UI notice deferred to 6.6). Unhandled types
are acknowledged + dropped by Fedify (4.6).
- Replay protection via Fedify's KvStore, now Postgres-backed
(journal.federation_kv + daily sweep job) so dedupe survives
restarts (4.7).
Schema (discovered requirement):
- follows.follower_id relaxed to nullable + follows.follower_actor_iri
for inbound remote followers — the proposal's 'follows is already
federation-ready' only held for outbound. Check constraint enforces
exactly one follower identity; partial unique index dedupes remote
follows. Notification fan-out + approve flow now filter local
followers explicitly. Design/proposal updated.
Tests: 7 inbox integration tests (accept/refuse/idempotence/undo/
settle/reject/check-constraint), 6 KvStore integration tests, 2 unit
tests for source-host extraction. All against real Postgres, gated on
FEDERATION_INTEGRATION=1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
social-federation tasks 2.1–2.5:
- users.public_key / users.private_key_encrypted (TEXT NULL): RSA 2048
keypairs as JWK JSON; private key AES-256-GCM encrypted at rest with
FEDERATION_KEY_ENCRYPTION_KEY.
- remote_actors table: cache of remote AP actors (display fields,
inbox/outbox URLs, public key, software discovery field, poll cursor).
- activities.remote_origin_iri (UNIQUE) / remote_actor_iri / audience:
provenance + audience tagging for rows ingested from remote outboxes.
Replay-safe ingestion keys off the unique origin IRI.
- crypto.server.ts: generalized into createAesCipher(envVar, salt)
factory; existing INTEGRATION_SECRET surface unchanged.
- federation-keys.server.ts: generate/ensure/load keypairs.
RSASSA-PKCS1-v1_5 because that's what Mastodon interops on.
- backfill-user-keypairs pg-boss job: enqueued once per server startup
when FEDERATION_ENABLED=true; only touches users with NULL keys, so
re-runs are no-ops. New users get keys at registration (both passkey
and magic-link paths), best-effort with the backfill as safety net.
- design.md: noted open question — activities.owner_id is NOT NULL but
remote-ingested rows have no local owner; decide in task 7.2.
Schema is additive; zero behavior change while FEDERATION_ENABLED is
off. db:push verified clean against local Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
social-federation tasks 1.1–1.3:
- Add @fedify/fedify pinned to exactly 2.1.16: every 2.2.x release
depends on @fedify/webfinger@2.2.x which was never published to npm,
so 2.1.16 is the newest installable version.
- app/lib/federation.server.ts: Federation instance with an actor
dispatcher serving Person objects for public users; private and
unknown users 404 (no existence leak). MemoryKvStore for now.
- /.well-known/webfinger resource route delegating to federation.fetch.
- ActivityPub content negotiation on /users/:username via route
middleware (future.v8_middleware — no loader uses the context arg,
so the flag is a no-op for existing code).
- FEDERATION_ENABLED env flag (default off) gating every federation
surface.
- Unit tests exercise the Fedify dispatcher as a remote AP client:
WebFinger resolution, actor fetch with Mastodon's Accept header,
private-user 404s, flag-off 404s.
Spike verdict: Fedify fits — URL dispatch, JRD/AP serialization, and
visibility gating all work through framework routes without a custom
server layer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Symptom
Grafana's \`app-crash-log\` alert fires on every deploy. The Loki query
that backs it (\`ERR_|FATAL|uncaughtException|…\`) matches:
Error [ERR_MODULE_NOT_FOUND]: Cannot find module
'/app/apps/journal/app/lib/logger.server' imported from
'/app/apps/journal/app/lib/email.server.ts'
A real bug, not a false positive: \`email.server.ts\` line 2 was
\`import { logger } from \"./logger.server\"\` — no extension.
## Why typecheck didn't catch it
\`tsconfig.base.json\` sets \`moduleResolution: \"bundler\"\`. The bundler
resolver accepts extensionless relative imports because Vite / esbuild
/ webpack resolve them at build time. TypeScript was happy.
Production runs \`node --experimental-strip-types server.ts\`, which
uses Node's NodeNext ESM resolver — strict about extensions. The two
resolvers disagree silently for files Node loads directly (server.ts,
\`.server.ts\` modules dynamically imported from it, and jobs).
## Fix
1. **Added \`.ts\` extensions to the 4 broken imports** I could find:
- \`apps/journal/app/lib/email.server.ts\` → \`./logger.server.ts\`
(the actual deploy-blocker)
- \`apps/planner/app/lib/brouter.ts\` → \`./route-merge.ts\`, \`./http.server.ts\`
- \`apps/planner/app/lib/use-nearby-pois.ts\` → \`./overpass.ts\`
- \`packages/map/src/MapView.tsx\` → \`./layers.ts\` (caught by the rule)
2. **Added \`eslint-plugin-import-x\` with \`import-x/extensions: always\`**,
scoped to files Node actually executes raw:
- \`apps/*/server.ts\`
- \`apps/*/app/lib/**/*.server.ts\`
- \`apps/*/app/jobs/**/*.ts\`
- \`packages/*/src/**/*.{ts,tsx}\`
Ignored: route-scoped \`*.server.ts\` files (bundled by Vite into the
React Router build, never loaded by Node), and test files (Vitest's
own resolver).
## What's still possible
- Non-\`.server.ts\` files in \`app/lib\` (e.g. \`legal.ts\`, \`actor-iri.ts\`)
could still ship extensionless imports. They're not in the lint
scope. If one breaks at runtime we can extend the glob — for now
they tend to be tiny utility modules that don't import other
relatives.
- The deeper fix would be \`moduleResolution: nodenext\` for server-side
tsconfig, or bundling the server code so Node never sees raw \`.ts\`.
Bigger surgery; the lint rule covers the failure mode for now.
Full repo: pnpm typecheck / lint / test all green after \`pnpm install\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Followup to #447. The audit ran on \`as unknown as\` first; this PR
closes out \`as any\` separately. After this, \`grep -rn ' as any\\b'
apps/ packages/\` returns 0 (excluding tests and node_modules).
## Sites fixed
**\`apps/journal/server.ts\`** + **\`packages/jobs/src/types.ts\`** —
\`komootBulkImportJob as any\`. The job had a typed payload
(\`JobDefinition<KomootBulkImportData>\`) but the worker's
\`JobDefinition[]\` array forced a contravariance cast at every site
that mixed typed and untyped jobs. Dropped the generic from
\`JobDefinition\` entirely; handlers narrow their own \`job.data\`. Only
one job (komoot-bulk-import) used the generic, so the surface is tiny.
**\`apps/journal/app/lib/connected-services/fit.ts\`** + same pattern in
\`routes/sync.import.\$provider.server.ts\` — \`parser.parse(buffer as any,
(err: unknown, d: any) => ...)\`. fit-file-parser's TypeScript types
require \`Buffer<ArrayBuffer>\` specifically; a generic Node \`Buffer\`
is structurally \`Buffer<ArrayBufferLike>\` (which includes
SharedArrayBuffer). The runtime accepts either, so narrowed the cast
to \`as Buffer<ArrayBuffer>\` — still a cast, but precise about what
we're asserting and why. Removed the \`(err, d: any) => …\` ad-hoc
callback typings; the library exports a proper \`FitParserCallback\`.
**\`apps/mobile/lib/editor/RouteMap.tsx\`** — \`onLongPress\` handler
took \`(event: any)\`. maplibre-react-native v11 generates the event
type via React Native codegen but doesn't re-export it as a public
TypeScript type. Replaced with a local structural slice of the parts
we actually read.
Net: 4 \`as any\` sites in prod code → 0. \`pnpm typecheck\` / \`lint\` /
\`test\` all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- RouteMapThumbnail.client.tsx: `{ type: 'Feature', ... } as unknown
as GeoJsonObject` → `as GeoJsonObject`. The literal already
satisfies the type; the unknown bridge was unnecessary.
- use-yjs.ts:102: keep the coercion (y-websocket's `.on()` signature
doesn't include the legacy `"synced"` event even though the
runtime still fires it), but document why with a comment so future
readers don't try to drop it.
Most of the existing coercions are legitimate (Drizzle raw-SQL row
shapes, linkedom DOMParser interop, fit-file-parser's loose \`any\`
callback API) — leaving those is honest. The ones cleaned up here
were shims around APIs that have real types we just hadn't wired:
- **\`window.__leafletMap\`** (4 sites: MapHelpers.tsx, SessionView.tsx ×3)
Added \`Window\` declaration in \`apps/planner/app/types/global.d.ts\`.
Callers now use plain \`window.__leafletMap\` typed as
\`L.Map | undefined\`.
- **\`L.markerClusterGroup\`** (PoiPanel.tsx) — leaflet.markercluster
augments the global \`L\` namespace at runtime but ships no types.
Added a \`declare module \"leaflet\"\` block in the same global.d.ts
with a minimal signature for what we actually use.
- **\`L.DomEvent.preventDefault / .stop\`** taking a Leaflet event
rather than a native one (PlannerMap.tsx, RouteInteraction.tsx,
NoGoAreaLayer.tsx). Leaflet events carry the native event under
\`.originalEvent\`; using that drops the cast entirely.
- **\`_creds as unknown as OAuthCredentials\`** orphan in
wahoo/webhook.ts — \`_creds\` was unused inside the callback (the
void-cast was a no-op preserving the type for documentation).
Replaced with \`async ()\`, dropped the unused import.
Net: 26 \`as unknown as\` / \`as any\` sites → 19, all remaining ones
gated by external lib interop or Drizzle raw-SQL.
Full repo: pnpm typecheck / lint / test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After Phase A (#442) moved the journal callback token off the browser,
the token was still replayable on the wire until \`exp\` (7 days). This
PR makes each token strictly single-use.
Changes:
- **\`journal.consumed_jwt_jti\` table** — \`jti TEXT PRIMARY KEY,
consumed_at TIMESTAMPTZ, expires_at TIMESTAMPTZ\`. Picked up by
drizzle-kit push on deploy.
- **\`createRouteToken\` now sets a \`jti\` claim** (\`randomUUID()\`).
- **\`verifyRouteToken\` atomically consumes the jti** via
\`INSERT … ON CONFLICT DO NOTHING RETURNING jti\`. Postgres
serializes the insert, so exactly one concurrent caller wins; the
rest see an empty result and throw \`TokenAlreadyConsumedError\`.
Tokens without a \`jti\` claim (i.e. minted before this PR) are
also rejected — the right call: any in-flight legacy token sitting
in a planner session is replayable, and we'd rather fail-loud than
silently grandfather them in.
- **\`consumed-jti-sweep\` job** — daily 03:45 UTC cron that
\`DELETE WHERE expires_at < now()\`. Keeps the table tiny; offset
from the other purge jobs to spread load.
- **e2e replay test** — \`integration.test.ts\` now exercises a
same-token double-submit and asserts the second returns 401 with
\`/consumed|already/i\`.
UX implication worth flagging: a user who clicks \"Save\" twice (or whose
network retries a failed POST) sees an error on the second attempt.
They go back to the journal for a fresh \"Edit in Planner\" link.
Full repo: pnpm typecheck / lint / test all green (177 + 31 integration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The five \`*.integration.test.ts\` files in apps/journal (explore, follow,
demo-bot, notifications, notifications-fanout — 31 tests total) were
gated behind \`EXPLORE_INTEGRATION=1\` / \`FOLLOW_INTEGRATION=1\` /
\`DEMO_BOT_INTEGRATION=1\` / \`NOTIFICATIONS_INTEGRATION=1\`. The unit-test
job doesn't have Postgres, so they correctly skipped there — but no
CI job set the env vars, so they were effectively dead code.
Added a step in the E2E job (which already has Postgres + schema
pushed) that flips all four gates and runs the integration files
serially (\`--no-file-parallelism\` — they share the schema and trip FK
constraints if run in parallel; ~2.5s sequential anyway).
Wiring them up surfaced two real issues, both fixed here:
1. **\`createRoute\` silently dropped \`input.visibility\`** — every
caller passing \`visibility: \"public\"\` (including the demo-bot)
was getting the column's \`private\` default. Spread now mirrors
\`createActivity\`'s pattern. Demo-bot routes have actually been
private in production all this time — they were rendering on the
home feed only via the \`activity_published\` fan-out from the
*activity*, not as visible *routes*.
2. **The demo-bot test's fetch stub was incomplete** — it stubbed
\`.text()\` but the planner-session preflight calls \`.json()\`. The
stub now branches on URL: \`/api/sessions\` returns
\`{ sessionId: 'test-session' }\` JSON, the BRouter call returns the
stub GPX text.
Full repo: pnpm typecheck, pnpm lint, pnpm test all green
(177 unit-test pass + 31 integration pass = 208 total, no skips).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-hosted instances no longer inherit the trails.cool flagship Sentry
DSNs. Code paths now read DSN strictly from env — unset = no Sentry
init, no events sent.
Flagship continues to report unchanged: cd-apps.yml and cd-staging.yml
inject the public DSNs as workflow env vars, which feed into:
- runtime via \`infrastructure/app.env\` / \`staging.env\`
(\`SENTRY_DSN_JOURNAL\` / \`SENTRY_DSN_PLANNER\` → compose env per service)
- build time via docker build-arg \`VITE_SENTRY_DSN\` (journal only;
planner has no client Sentry init).
Sentry DSNs are public-by-design (transmitted unencrypted from the
client JS bundle), so embedding them as plaintext workflow env vars
is no worse than the runtime exposure. Forks should replace these with
their own DSNs or remove the workflow env lines to ship Sentry-free.
Files changed:
- apps/journal/server.ts: \`process.env.SENTRY_DSN\` only, no fallback
- apps/planner/server.ts: same
- apps/journal/app/lib/sentry.client.ts: \`import.meta.env.VITE_SENTRY_DSN\`
only; falsy = skip init
- apps/journal/Dockerfile: new \`ARG VITE_SENTRY_DSN\` baked into build
- infrastructure/docker-compose.yml: pass \`SENTRY_DSN\` per service
- infrastructure/docker-compose.staging.yml: same
- .github/workflows/cd-apps.yml: workflow env + build-arg + app.env echo
- .github/workflows/cd-staging.yml: same
Full repo: pnpm typecheck, pnpm lint, pnpm test, journal build all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous handler opened a fresh postgres client (max: 1) on every
call to /api/health and tore it down in the finally block. Under the
prod monitoring cadence (probes every few seconds), that's a fresh
TCP + TLS + auth handshake on every probe, plus connection-table
churn on the Postgres side — fine for the trickle of curl-ish manual
checks, slow-bleed under blackbox monitoring.
Now we cache a module-level singleton postgres client dedicated to
/api/health (max: 2, idle_timeout: 30) and reuse it across calls.
Separate from the app's main DB pool (via @trails-cool/db's createDb)
on purpose — so a starvation event on the main pool doesn't fail the
liveness check and trigger a restart loop.
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Every HTTP request now gets a requestId (inbound X-Request-Id header is
honored, otherwise a fresh UUID is minted) and the value is echoed on
the response. The server wraps the request in
\`requestContext.run({ requestId }, ...)\` — an AsyncLocalStorage scope —
so pino's \`mixin\` callback can read it on every log call without the
caller threading it through.
Net effect: \`logger.info({ ... }, \"db error\")\` from a loader, action,
or downstream lib now lands in JSON with a \`requestId\` field, making
cross-handler debugging trivial (\`grep requestId=abc-123\` returns the
full request trace).
Out of scope here:
- Planner gets the same treatment (separate, smaller PR after this lands).
- BRouter / Fedify outbound calls don't propagate the requestId yet —
those are HTTP boundaries where we'd add it as a header, but the
audit value was the in-process trace.
Tests:
- logger.server.test.ts (2 cases — als-bound info tags requestId; no
context = no tag).
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Sentry DSNs were hardcoded in journal/server.ts, planner/server.ts,
and journal/app/lib/sentry.client.ts. Self-hosted instances inheriting
the trails.cool flagship DSN would silently ship their errors to our
Sentry account.
Now each init site reads its DSN from env:
- journal/server.ts: SENTRY_DSN (server runtime env)
- planner/server.ts: SENTRY_DSN (server runtime env)
- journal/app/lib/sentry.client.ts: VITE_SENTRY_DSN (build-time bake)
The flagship DSN is kept as the fallback so the production deploy keeps
reporting without an infra change — but self-hosters can:
- set SENTRY_DSN=\"\" / VITE_SENTRY_DSN=\"\" to ship their own builds
without Sentry, or
- set SENTRY_DISABLED=true to skip init entirely at runtime, or
- set SENTRY_DSN=/VITE_SENTRY_DSN= to their own DSN.
Follow-up: a future PR can remove the hardcoded fallbacks once
infrastructure/docker-compose.yml + the cd-apps.yml workflow are wired
to pass SENTRY_DSN explicitly. That requires SOPS edits + workflow
changes I want isolated from this purely-code change.
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
verifyLoginCode, verifyMagicToken, and verifyEmailChange previously did
SELECT WHERE used_at IS NULL → UPDATE … SET used_at = now. Two concurrent
verifications could both pass the SELECT and both succeed, accepting the
same single-use token twice.
Collapsed each to a single UPDATE … WHERE … RETURNING * statement.
Postgres serializes row-level locks within an UPDATE, so exactly one
concurrent caller observes a returned row; the rest see an empty array
and get \"Invalid or expired\". The token is also marked used as part of
the same statement — no second write needed.
verifyEmailChange's tertiary email-availability check now runs *after*
the consume; we keep the original semantics where the token is burned
on a clash (the previous code explicitly did the same with a separate
UPDATE).
No behavior change on the happy path. Closes a credential-reuse
window that mattered most for the 6-digit login codes (small search
space, more likely to race).
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous \`importOne\` only fetched page 1 of /v1/workouts and errored
with \"not found on page 1\" if the workout wasn't there — silently
breaking import for any workout older than roughly the most recent 30
entries (per_page default). Webhook-driven imports happen to land on
page 1 by definition, so this only bit on user-initiated catch-up
imports of older workouts.
Now we paginate forward, using the \`total / per_page\` returned by page 1
to compute a stop condition, with a \`MAX_PAGES=100\` ceiling so a
misbehaving API can't loop us. We also stop early on an empty page.
Tests:
- new pagination case (workout on page 2, expect 2 fetch calls)
- new \"not found on any page\" case
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rollup was warning on 5 modules that were both dynamically and statically
imported. With static importers in the same chunk, the dynamic forms
buy no chunking benefit — they were leftovers from earlier
cycle-avoidance workarounds that no longer apply.
Converted to static:
- @trails-cool/gpx (routes.\$id.server.ts, sync.import.\$provider.server.ts)
- logger.server (boss.server.ts — comment claimed test cycles, but tests pass)
- boss.server (activities.server.ts at two sites)
- connected-services/manager (komoot/importer.ts, wahoo/importer.ts)
- notifications.server (root.tsx)
Kept dynamic: fit-file-parser in sync.import.\$provider.server.ts — it's
heavy and only the FIT ingestion path needs it, no other static
importers exist, so the dynamic actually does chunk-split it.
Build is now warning-free.
Full repo: pnpm typecheck, pnpm lint, pnpm test, pnpm build all green.
192 tests passed (up from 181 — rate-limit test from #424 + others).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
E2E suite drives registrations from one IP at parallel volume;
production limits trip and flake tests. Same opt-out pattern as the
fail-loud secret/DB-URL guards.
Adds per-process in-memory fixed-window rate limiting (lib/rate-limit.server.ts)
and applies it across /api/auth/login and /api/auth/register:
- All login steps: 30 attempts per IP per minute (defense against
step-fanout flooding).
- finish-passkey: 10 per IP per minute (assertions don't usefully retry
faster).
- magic-link generation: 3 per email per 5 min + 10 per IP per 5 min
(defeats inbox-spam-the-victim + cross-email IP fanout).
- verify-code: 10 per email per 15 min — makes 6-digit code brute force
(10^6 search) infeasible before code expiry.
- /api/auth/register: 10 per IP per hour (legitimate signup completes
in 2-3 requests; sustained churn from one IP is account spam).
Single-instance is fine for the flagship's current topology (one journal
container). When we horizontally scale we revisit with a Postgres- or
Redis-backed store. Buckets self-clean on next read and a 5-minute
background sweep drops stale entries.
Client IP: honors X-Forwarded-For first entry (Caddy in front of the
journal sets it), falls back to a stable "unknown" bucket so the
limiter still bites when the header is missing.
Tests: rate-limit.server.test.ts (8 cases — exhaustion, independent
scopes/keys, remaining countdown, reset window, XFF parsing, fallback,
trimming). Existing auth tests still pass; limits sized to not trip
during the ~7 test cases that all share the "unknown" IP bucket.
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Playwright runs the server via `react-router serve` with
NODE_ENV=production but against a local dev Postgres and local cookie
secrets. The guards added in 5a7bb76 refused to start under that
configuration. `E2E=true` (already set by the CI E2E job) is now the
explicit opt-out: in real production this env var is never set, so the
guard still bites.
Adds `requireSecret(name, devFallback)` in lib/config.server.ts and
`getDatabaseUrl()` in @trails-cool/db. Both:
- return the env var when set,
- fall back to the dev default in non-production,
- throw at boot in production if the env var is missing OR matches the
known dev fallback (which would otherwise silently ship a public
secret / point at localhost).
Applied to:
- JWT_SECRET (lib/jwt.server.ts) — was `?? "dev-jwt-secret-change-in-production"`
- SESSION_SECRET (lib/auth/session.server.ts) — was `?? "dev-secret-change-in-production"`
- DATABASE_URL (server.ts health + boss; packages/db migrate-data) — was
`?? "postgres://trails:trails@localhost:5432/trails"`
Why: these strings are in the repo and known to attackers. A
misconfigured prod deploy that forgot to set them would either run with
guessable signing keys (full session/JWT forgery) or connect to a
non-existent localhost DB. Better to refuse to start than to silently
operate insecurely.
Tests:
- packages/db/src/get-database-url.test.ts (5 cases)
- lib/config.server.test.ts gains `requireSecret` cases (4 new)
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Completes the .server.ts split started in #418. Every route that mixes
a default-export component with a server-only loader/action now has a
sibling <route>.server.ts holding the data-fetching helpers; the route
.tsx is a thin delegator.
Routes converted (21):
activities._index, activities.$id, activities.new, auth.accept-terms,
auth.verify, explore, feed, notifications, routes._index, routes.$id,
routes.$id.edit, routes.new, settings, settings.account,
settings.connections.komoot, settings.profile, settings.security,
sync.import.$provider, sync.import.komoot, users.$username.followers,
users.$username.following
Pattern (same as home.tsx / users.$username.tsx / settings.connections.tsx):
- loader → `return data(await loadX(request, params?))`
- action → `return await xAction(request, params?)`
- All `getDb` / Drizzle schema / `~/lib/*.server` imports move to the
.server.ts sibling.
- `throw redirect(...)` and `throw data(...)` propagate through the
delegator unchanged.
No behavior changes — pure module-graph cleanup. Component modules no
longer transitively import the DB client; Vite's tree-shake of
server-only code is now backed by an explicit, file-local contract.
Verified:
- pnpm typecheck — green
- pnpm lint — green
- pnpm test — 181 passed, 31 integration-gated skipped
- pnpm --filter @trails-cool/journal build — succeeds
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to PR #406 — addresses the two items deferred from the audit:
#7 — Centralize auth helpers
- New `requireSessionUser(request)` in lib/auth/session.server.ts that
returns the user or throws a redirect to /auth/login.
- New `requireSessionUserJson(request)` companion that throws a 401 JSON
response (for fetcher/JSON endpoints).
- Replace the repeated
const user = await getSessionUser(request);
if (!user) return redirect("/auth/login");
pattern across 18 route loaders/actions. Removes the duplicated guard
preamble and gives a single chokepoint to evolve later (e.g., for
terms-version gating).
#8 — Extract heavy loaders into .server.ts siblings
- routes/home.tsx → home.server.ts (DB count query + listActivities +
listRecentPublicActivities)
- routes/users.$username.tsx → users.$username.server.ts (user lookup +
follow state + counts + listPublicRoutes/Activities + persona check)
- routes/settings.connections.tsx → settings.connections.server.ts
(connected_services join + manifest merge)
Each route file shrinks to a thin delegator: `loader` calls
`loadXxx(request)`. The component module no longer transitively pulls
`getDb` and Drizzle schema into its import graph — Vite's tree-shake
already strips server-only code from the client bundle, but the
explicit `.server.ts` suffix makes that contract local and auditable.
Other 17 routes that mix loader/action with components are left as-is
for now: they're each small enough that the split adds churn without
buying much clarity. The pattern is documented by the three examples;
the rest can convert opportunistically when they grow.
Tests:
- lib/auth/session.server.test.ts (4 cases — redirect for missing
cookie, redirect for ghost userId, success path, JSON 401 variant)
Full repo: pnpm typecheck, pnpm lint, pnpm test all green
(181 passed | 31 integration-gated skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses 8 issues from the Journal architecture audit:
1. DB indexes on routes.ownerId + activities.ownerId. Listing queries on
these tables were full table scans; adds composite indexes matching
the order-by columns (updatedAt/startedAt/createdAt).
2. Zod validation on /api/auth/register body. Previously the action
destructured request.json() with zero schema validation.
3. N+1 GeoJSON batch fetch collapsed to a single ANY($1::text[]) query
in both routes.server and activities.server.
4. Webhook envelope validation in /api/sync/webhook/:provider.
5. AbortSignal.timeout(30s) on all external fetches (Komoot, Wahoo) via
a new fetchWithTimeout helper in lib/http.server.ts.
6. .limit(100) on listPublicRoutesForOwner / listPublicActivitiesForOwner.
9. Welcome email moved off fire-and-forget onto a pg-boss job with
retryLimit: 3 (send-welcome-email).
10. process.env.ORIGIN ?? "http://localhost:3000" centralized into
lib/config.server.ts::getOrigin() across 14 call sites.
Issues 7 (centralized apiError/auth guards across 60+ route files) and
8 (split .server.ts boundaries across 20+ route files) intentionally
deferred — both are pure refactors that would balloon this PR past
reviewability and warrant their own focused PRs.
Tests added:
- lib/config.server.test.ts (2 cases)
- lib/http.server.test.ts (3 cases — timeout abort, success passthrough,
caller-signal composition)
- routes/api.sync.webhook.$provider.test.ts (6 cases)
- routes/api.auth.register.test.ts (7 cases — schema rejection paths +
the new welcome-email enqueue assertion)
Full repo: pnpm typecheck, pnpm lint, pnpm test all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Use ImportBatchStatus type instead of raw string literals in sweep job
- Add a COUNT pre-check so the sweep UPDATE is skipped when no stale
batches exist (avoids an unconditional write every minute)
- Remove comment in disconnect route that explained what the code does
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously always redirected to /settings (which resolves to /settings/profile).
Now reads the Referer header and redirects back to the originating /settings/*
page, defaulting to /settings/connections if no valid referer.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Batches stuck in pending or running for more than 10 minutes (server
restart mid-import, pg-boss job dropped) are marked failed with a
user-visible message. Runs every minute via pg-boss cron with a 55s
expiry so overlapping runs are dropped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Same sort toggle as the user profile page (#399): default is activity
date (startedAt), switchable to "Date added" (createdAt) via ?sort=addedAt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the per-tour import UI with a fire-and-forget background job:
- Add `import_batches` DB table tracking status, found/imported/duplicate counts
- Add `runKomootBulkImport` server function that pages all Komoot tours,
fetches GPX, creates activities, and deduplicates via sync_imports
- Add `komoot-bulk-import` pg-boss job registered at server startup
- Add POST /api/sync/komoot/import to enqueue the job
- Add GET /api/sync/komoot/import-status to return the latest batch
- Replace the Komoot import page with a progress UI that polls every 2s
while a batch is running and shows found/imported/skipped counts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Default sort is by activity date (startedAt); users can switch to
"Date added" (createdAt) via a URL query param (?sort=addedAt).
Activities without a startedAt fall to the bottom when sorted by date.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a workout is imported and the loader revalidates, importableWorkouts
shrinks. The snapshot-update useEffect was overwriting the ref with the
shorter list, causing the index to point to the wrong item and skip every
alternate workout. Remove the snapshot update so the original list is used
throughout the import-all session.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
