Commit graph

220 commits

Author SHA1 Message Date
Ullrich Schäfer
2106d345cf docs(openspec): social-federation 12.4 soaked live — change complete (58/58)
Flagship ⇄ staging, both directions, 2026-06-07: follows settled to
Accepted in seconds, first polls ingested each side's public
activities, both feeds render remote attribution. The cross-origin
Accept fix (#490) verified working in production.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 15:36:01 +02:00
Ullrich Schäfer
f3a17cf78c feat(infra): enable federation on the flagship (social-federation 12.5)
Wiring follows the IS_FLAGSHIP pattern: the compose env defaults every
FEDERATION_* var to empty (off — the safe self-host default; all
federation surfaces 404), and cd-apps.yml appends
FEDERATION_ENABLED=true + FEDERATION_LOG_LEVEL=info to the flagship's
app.env. FEDERATION_KEY_ENCRYPTION_KEY was already in SOPS.

Rollout 12.2/12.3 soaked on staging against a real Mastodon
(2026-06-06/07); the trails↔trails Accept bug found by the §11 harness
is fixed and deployed. Rollback: drop the two echo lines, merge, rerun
cd-apps — instant off, follow rows persist.

First boot with the flag enqueues backfill-user-keypairs (idempotent)
and registers the federation jobs. Applied via manual cd-apps dispatch
after merge since workflow-file changes don't trigger the path filter.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 14:46:32 +02:00
Ullrich Schäfer
819a5d2976 docs(openspec): social-federation §12 rollout status notes
12.1/12.2/12.3/12.6 marked with provenance (additive schema behind the
off flag on prod; the 2026-06-06/07 staging soak against a real
Mastodon covered inbound + push delivery; rollback documented in the
runbook). 12.4 annotated: protocol verified by the two-instance
harness, live staging⇄preview soak queued now that previews federate.
12.5 (prod flag flip) explicitly left as an operator decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 13:46:14 +02:00
Ullrich Schäfer
1eceb6f1f7 feat(journal): two-instance federation harness + trails↔trails Accept fix (§11)
The 11.4 harness — e2e/federation/: postgres (two DBs) + a Caddy with
an internal CA terminating real HTTPS between two complete journal
containers (journal-a.test / journal-b.test), driven by an opt-in
integration test (FEDERATION_TWO_INSTANCE=1, via run.sh). The driver
seeds users straight into each DB, mints a signed session cookie
(known SESSION_SECRET), follows across instances through the real
/follows/outgoing route, and asserts the full pipeline: Follow →
auto-Accept → settle → first outbox poll → bob's public activity
rendered in alice's /feed with @bob@journal-b.test attribution. Plus a
wire-level 11.3 check: unsigned Create(Note) → 4xx, no DB writes.

And the harness immediately earned its keep — it caught a real
trails↔trails bug Mastodon interop never could: our Follow ids are
fragment URIs on OUR domain, so the Follow embedded in another trails
instance's Accept is cross-origin and Fedify rightly distrusts it;
re-fetching the id returns the actor document, getObject() yields a
Person, and the Accept/Reject/Undo listeners bailed silently — follows
stayed Pending forever. The listeners now fall back to the wire
objectId (captured BEFORE getObject(), which memoizes the fetched
document and changes what objectId reports) validated against our
Follow-id shape + the personal inbox's ctx.recipient. Forgery-safe:
settleOutgoingFollow only matches a Pending row toward the
HTTP-Signature-authenticated sender.

Supporting changes:
- federation.server.ts: env-gated allowPrivateAddress
  (FEDERATION_ALLOW_PRIVATE_ADDRESS=true) so the harness's RFC 1918
  Docker network is fetchable — testing only, loudly documented.
- Root .dockerignore: local builds shipped a 1GB+ context and
  overlaid host (darwin) node_modules over the image's own install
  via COPY . . — CI never noticed because it builds from clean
  checkouts. Context is now ~5MB.
- tasks.md: 11.1–11.7 marked with provenance notes; design.md gains
  the cross-origin embedded-object lesson.

Gate: typecheck ✓ lint ✓ unit+integration (FEDERATION_INTEGRATION=1) ✓
e2e 71/72 + known komoot flake green isolated ✓ harness run clean from
scratch (2/2, teardown included) ✓

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 13:36:35 +02:00
Ullrich Schäfer
3c2bdfd2bd feat(journal): honor 429/Retry-After in outbox polling (§7.4)
Completes the 7.4 remainder. Fedify's FetchError carries the failed
Response, so a 429 from a remote's outbox now reads Retry-After
(delta-seconds or HTTP-date per RFC 9110), arms a per-host backoff
(15 min default, capped at the 1 h poll interval), and pollRemoteActor
skips backing-off hosts before doing any DB or network work. State is
in-process like the pacing map — a restart costs at most one extra
request that re-arms the backoff. last_polled_at is deliberately not
stamped on a rate-limited poll so the hourly sweep retries.

Unit tests cover the header parser (incl. the V8 footgun where
Date.parse reads bare negative integers as years), default/cap
behavior, and window expiry; an integration test drives the full
FetchError path and asserts the second poll never touches the network.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:35:59 +02:00
Ullrich Schäfer
bf2d8bfbd4 feat(journal): audience-aware social feed with remote activities (§8+§9)
8.1/8.2: listSocialFeed is now a UNION ALL of local and remote
branches, sorted on COALESCE(remote_published_at, created_at):
- local rows: visibility='public' from accepted local follows — and
  the previously missing accepted_at filter is added (the spec's
  'Pending follows contribute nothing' scenario)
- remote rows: gated structurally by joining the viewer's OWN accepted
  follow against the originating actor — which is exactly the
  followers-only audience rule (a row reaches only viewers whose
  follow brought it in); attribution from the remote_actors cache;
  cards link outward to the canonical origin page (no local detail
  page for remote rows)

9.1: annotated — local Pending button shipped with locked accounts;
remote Pending lives on /follows/outgoing.
9.2: already enforced + tested since §3 (actor/webfinger 404).
9.3 hardening: deliver-activity now re-checks the OWNER's profile
visibility at send time, closing the enqueue→delivery flip window
(enqueue-side and inbound-side gates already existed).

Integration tests: the §8 audience-leak guard (A sees followers-only,
B does not), public remote attribution + outward link, pending
contributes nothing, mixed local/remote COALESCE ordering.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:32:11 +02:00
Ullrich Schäfer
b96bef91a9 feat(journal): outbox-poll ingestion of remote trails activities (§7)
Tasks 7.1–7.3, 7.5 (+7.4 pacing; Retry-After-duration backoff noted as
remaining). Resolves the activities.owner_id open question.

Schema (design decision from the open question):
- activities.owner_id nullable + check constraint enforcing exactly
  one of (owner_id, remote_actor_iri) — the follows pattern again.
- remote_published_at carries the origin's publish time for the §8
  feed sort; index on remote_actor_iri for the feed join.
- Compiler-audited fallout: notification fan-out, the Note object
  dispatcher, and the activity detail loader explicitly skip/404
  remote rows (their canonical page is the origin; feed links there).
  Every other surface joins users on owner_id and excludes remote rows
  structurally.

Ingestion:
- federation-ingest.server.ts: parseOutboxItem targets exactly our own
  outgoing Create(Note) shape (PropertyValue stats, first-paragraph
  name, audience from to/cc); unknown items skipped, never fatal.
- ingestRemoteActivities: replay-safe via unique remote_origin_iri,
  conflict-streak early exit (outboxes are newest-first).
- pollRemoteActor: signed fetch (Authorized Fetch via a local
  follower's key), outbox resolution via remote_actors cache with
  actor-doc fallback (which refreshes the cache), 1 req/5 s per-host
  pacing, last_polled_at stamping. Network + pacing injectable.

Jobs: poll-remote-actor (per-actor; the §4 Accept listener already
enqueues it as the first poll) + poll-remote-outboxes (5-min cron
sweep over accepted remote follows not polled within the hour).

Tests: parse-shape units; integration suite for ingestion provenance,
audience→visibility mirroring, replay safety, the DB author invariant,
poll flow (actor-doc → collection → page), signer requirement, and
due-polling selection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:27:10 +02:00
Ullrich Schäfer
f4d2cf027c docs(journal): federation privacy manifest + runbook + honest home blurb (§10)
10.1 Privacy manifest (legal/privacy):
- German legal half: Föderation entry under Empfänger/Drittanbieter —
  what a public profile exposes, what accepted followers' servers
  receive, the loss-of-control over delivered copies, Art. 6(1)(a)
  basis, private profiles don't federate.
- English manifest half: dedicated Federation (ActivityPub) section —
  actor object contents, push delivery + retraction semantics (incl.
  the tombstone caveat), encrypted-at-rest signing keys, remote actor
  cache, ingested remote content with the followers-only viewer gate,
  inbox logging/rate limits. PRIVACY_LAST_UPDATED bumped.

10.2 docs/deployment.md federation runbook: enabling per environment,
key rotation posture, abuse monitoring, and the troubleshooting
checklist distilled from the 2026-06-06/07 soak (IP families first,
no outbox backfill, tombstones, the 10s delivery timeout, actor
re-fetch).

10.3 Home marketing blurb (en+de) aligned to what actually ships:
Mastodon can follow you + trails-to-trails outbound — no more 'follow
friends anywhere' overstatement.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:21:51 +02:00
Ullrich Schäfer
62b40a25f3 feat(journal): outbound trails-to-trails follows (social-federation §6)
Tasks 6.1–6.6. A local user can follow a user on another trails
instance: WebFinger-resolve the handle, verify the target runs
trails.cool via NodeInfo — checked against the ACTOR IRI's host, never
the handle's domain (split-domain constraint) — record a Pending
follow row, deliver a signed Follow. The §4 inbox listeners already
settle (Accept) or drop (Reject) the row.

- federation-outbound.server.ts: followRemoteActor / cancelRemoteFollow
  (Undo delivery) / listOutgoingRemoteFollows. Network steps (lookup,
  NodeInfo, delivery) injectable for offline integration tests.
  Software allowlist: trails-cool.
- /follows/outgoing: follow-by-handle form + pending/accepted list +
  cancel; linked from the feed empty state; i18n en+de. Remote actors
  have no local profile page, so the remote Pending state lives here
  (the profile Pending button from locked accounts covers local).
- /.well-known/trails-cool now publishes software: trails-cool (6.2).
- Clear 4xx codes: invalid_handle, local_handle, remote_resolve_failed,
  not_trails (6.3).
- Tests: handle-parser + allowlist units; integration suite for the
  Pending lifecycle (create/idempotent/refuse-non-trails/refuse-
  unresolvable/own-host/cancel+Undo) with injected network deps; e2e
  anonymous-redirect guard for the new page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:50:35 +02:00
Ullrich Schäfer
bf907976c4 docs: split-domain handles idea + interop constraint for the trails check
Prompted by Hollo's split-domain setup. Two parts:

- docs/ideas/split-domain-handles.md: offering split handle/server
  domains to self-hosters is config-level work — Fedify's origin
  option natively accepts { handleHost, webOrigin } — and handles are
  permanent identity, so apex handles matter. Single-domain stays the
  default (Decision #18); post-launch polish.
- Interop constraint pinned in social-federation task 6.1 and
  route-federation's gating decision: the trails-to-trails NodeInfo
  check must run against the actor IRI's host after WebFinger
  resolution, never the handle's domain — split-domain remote
  instances would otherwise be wrongly refused.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:18:13 +02:00
Ullrich Schäfer
5d31563008
Merge branch 'main' into fix/tombstone-aware-retraction 2026-06-07 10:57:26 +02:00
Ullrich Schäfer
bf9787e56a docs: capture remaining architecture vision; draft route-federation change
Gap analysis of docs/architecture.md against openspec/ and docs/ideas/
found the envisioned-but-uncaptured remainder. This commit captures it:

New OpenSpec change (validated):
- route-federation — the collaboration half of the federation vision:
  routes as dereferenceable trails:Route objects, Create/Update
  fan-out, Invite/Accept collaboration mirroring (arch decision #2),
  cross-instance Planner edits via HTTP-Signature-requested scoped
  tokens (decisions #3/#12), mirror sync healing (#16). Depends on
  social-federation §6 + route-sharing; carries the 2026-06-07 soak
  lessons as design constraints.

New docs/ideas/ explorations:
- instance-administration — registration toggle, suspend/ban,
  federation blocklists, reports (moderation now gates the federated
  comments idea)
- social-interactions — local likes + comments (don't exist even
  locally; the foundation federated kudos/comments attach to)
- activity-participants — group tagging with confirm/decline +
  federated mentions (the participants jsonb column is an untyped stub)
- multi-day-collections — architecture open question #1, directions
  evaluated

architecture.md cleanup:
- Mastodon-compat section annotated with shipped/captured state + the
  live-soak interop lessons (attachment arrays, tombstones, 10s
  timeout, no backfill)
- api.trails.cool removed (contradicted resolved decision #18)
- Phase 1 ticked (shipped); Phase 2/3 items annotated with where each
  is tracked; specs/ → openspec/, activity.* → journal.*, cx21 → cx23
- brouter-web open question marked resolved-in-practice; new section
  pointing to where the unshipped vision is tracked

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:31:50 +02:00
Ullrich Schäfer
20be961177 fix(journal): retract federated activities only on public→non-public
A received Delete is recorded by Mastodon as a permanent tombstone —
later Creates for the same URI are silently refused forever. The old
code sent Delete on every non-public save 'just in case' (the comment
even called over-sending harmless); on the 2026-06-07 soak this
tombstoned an unlisted activity's URI before its first real publish,
making its later flip to public invisible on the remote with no error
anywhere.

- visibilityTransitionAction(previous, next): Create on any transition
  to public (re-publish doubles as back-delivery; remotes dedupe by
  id), Delete only when leaving public, nothing for
  non-public→non-public.
- updateActivityVisibility reads the previous visibility and acts on
  the transition.
- design.md documents the tombstone permanence + the user-facing
  consequence (un-publish then re-publish won't resurrect the post on
  remotes that processed the retraction — same as Mastodon's own
  delete-and-redraft).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 08:52:23 +02:00
Ullrich Schäfer
bc233e03e5 feat(journal): federation outbox + push delivery to remote followers
social-federation tasks 5.1–5.6. Completes the inbound-federation
story: a Mastodon follower now receives a trails user's new public
activities in their home timeline.

Outbox (5.1/5.2):
- /users/:username/outbox — paginated OrderedCollection of public
  activities as Create(Note), newest first; unlisted/private never
  federate. Private-user 404 enforced at the route layer because
  Fedify builds collection-level responses from counter/cursors
  without consulting the page dispatcher.
- Note shape: HTML content (escaped name/description/stats + link to
  the activity page) with structured PropertyValue attachments
  (distance-m, elevation-gain-m, duration-s) — Mastodon renders the
  text, trails consumers read the structured fields. Resolves the
  design open question toward Create(Note).
- Authorized Fetch: signed and unsigned outbox fetches deliberately
  see the same (public-only) content until locked accounts exist.

Push delivery (5.3–5.6):
- createActivity / updateActivityVisibility(→public) enqueue one
  deliver-activity job per accepted remote follower; flips away from
  public and hard deletes enqueue Delete(Tombstone) retractions
  (enqueued before the row disappears).
- deliver-activity job: re-reads the row at delivery time (skips if
  gone or no longer public), resolves the recipient inbox via the
  remote_actors cache with actor-document fetch fallback (priming the
  cache), HTTP-signs via the owner's key, and POSTs. retryLimit 8 +
  exponential backoff at enqueue time; outbound paced at 1 req/s per
  remote host.
- Actor objects now advertise the outbox IRI.
- @js-temporal/polyfill added (same range Fedify uses) for published
  timestamps; Fedify's types want the global esnext.temporal namespace,
  bridged with a documented cast.

Tests: 9 unit tests for the AS mapping (escaping, stats, attachments,
published fallback, stable ids, tombstones), 4 outbox integration
tests (collection count, page shape/visibility filtering, private-404,
delivery audience query).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 15:32:52 +02:00
Ullrich Schäfer
b17685d58c feat(journal): federation inbox — Mastodon follows land here
social-federation tasks 3.1–3.4, 4.1–4.8. With this, a Mastodon user
can follow a public trails user: WebFinger → actor fetch → signed
Follow → recorded + Accept(Follow) pushed back.

Identity surface (section 3):
- Actor objects now carry the user's public key (publicKey +
  assertionMethods via Fedify key pairs dispatcher; keys generated
  lazily as a fallback to the backfill) and an inbox IRI; url uses
  localActorIri (3.1).
- Software discovery shipped as standard NodeInfo
  (/.well-known/nodeinfo + /nodeinfo/2.1, software.name trails-cool)
  instead of the originally-sketched custom AS actor field — Fedify's
  typed vocab can't emit arbitrary actor props and NodeInfo is what
  the fediverse reads. Artifacts updated accordingly (3.4).

Inbox (section 4):
- /users/:username/inbox resource route; HTTP Signatures verified by
  Fedify before any listener runs (4.1). Rate-limited 60 req/5 min per
  source instance (host from Signature keyId) BEFORE verification so
  hostile instances can't burn CPU on key fetches (4.8).
- Listeners: Follow → auto-accept for public profiles + Accept pushed
  back (4.2); Undo(Follow) → row removed (4.3); Accept(Follow) →
  Pending settled + first outbox poll enqueued (4.4); Reject(Follow) →
  Pending dropped (4.5; UI notice deferred to 6.6). Unhandled types
  are acknowledged + dropped by Fedify (4.6).
- Replay protection via Fedify's KvStore, now Postgres-backed
  (journal.federation_kv + daily sweep job) so dedupe survives
  restarts (4.7).

Schema (discovered requirement):
- follows.follower_id relaxed to nullable + follows.follower_actor_iri
  for inbound remote followers — the proposal's 'follows is already
  federation-ready' only held for outbound. Check constraint enforces
  exactly one follower identity; partial unique index dedupes remote
  follows. Notification fan-out + approve flow now filter local
  followers explicitly. Design/proposal updated.

Tests: 7 inbox integration tests (accept/refuse/idempotence/undo/
settle/reject/check-constraint), 6 KvStore integration tests, 2 unit
tests for source-host extraction. All against real Postgres, gated on
FEDERATION_INTEGRATION=1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 14:33:43 +02:00
Ullrich Schäfer
9a90b6db55 feat(journal): federation schema + per-user signing keypairs
social-federation tasks 2.1–2.5:

- users.public_key / users.private_key_encrypted (TEXT NULL): RSA 2048
  keypairs as JWK JSON; private key AES-256-GCM encrypted at rest with
  FEDERATION_KEY_ENCRYPTION_KEY.
- remote_actors table: cache of remote AP actors (display fields,
  inbox/outbox URLs, public key, software discovery field, poll cursor).
- activities.remote_origin_iri (UNIQUE) / remote_actor_iri / audience:
  provenance + audience tagging for rows ingested from remote outboxes.
  Replay-safe ingestion keys off the unique origin IRI.
- crypto.server.ts: generalized into createAesCipher(envVar, salt)
  factory; existing INTEGRATION_SECRET surface unchanged.
- federation-keys.server.ts: generate/ensure/load keypairs.
  RSASSA-PKCS1-v1_5 because that's what Mastodon interops on.
- backfill-user-keypairs pg-boss job: enqueued once per server startup
  when FEDERATION_ENABLED=true; only touches users with NULL keys, so
  re-runs are no-ops. New users get keys at registration (both passkey
  and magic-link paths), best-effort with the backfill as safety net.
- design.md: noted open question — activities.owner_id is NOT NULL but
  remote-ingested rows have no local owner; decide in task 7.2.

Schema is additive; zero behavior change while FEDERATION_ENABLED is
off. db:push verified clean against local Postgres.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 14:15:51 +02:00
Ullrich Schäfer
4ef86e4dc2 feat(journal): Fedify spike — WebFinger + actor objects behind FEDERATION_ENABLED
social-federation tasks 1.1–1.3:

- Add @fedify/fedify pinned to exactly 2.1.16: every 2.2.x release
  depends on @fedify/webfinger@2.2.x which was never published to npm,
  so 2.1.16 is the newest installable version.
- app/lib/federation.server.ts: Federation instance with an actor
  dispatcher serving Person objects for public users; private and
  unknown users 404 (no existence leak). MemoryKvStore for now.
- /.well-known/webfinger resource route delegating to federation.fetch.
- ActivityPub content negotiation on /users/:username via route
  middleware (future.v8_middleware — no loader uses the context arg,
  so the flag is a no-op for existing code).
- FEDERATION_ENABLED env flag (default off) gating every federation
  surface.
- Unit tests exercise the Fedify dispatcher as a remote AP client:
  WebFinger resolution, actor fetch with Mastodon's Accept header,
  private-user 404s, flag-off 404s.

Spike verdict: Fedify fits — URL dispatch, JRD/AP serialization, and
visibility gating all work through framework routes without a custom
server layer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 14:15:27 +02:00
Ullrich Schäfer
81ba2d5de3 docs: fix stale route-sharing → social-federation dependency
The roadmap claimed social-federation depends on route-sharing for
public routes, but that prerequisite shipped independently via
public-content-visibility (2026-04-24) and social-feed (2026-04-25):
visibility columns, public profiles, and IRI-keyed follows are all in
place. Reorder Phase 1 to reflect that federation can start now.

Also flag route-sharing's proposal as needing re-scoping: tasks
1.1-1.3 and 3.1 duplicate already-shipped visibility work, and the
proposed enum contradicts the shipped text-column design.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 13:16:06 +02:00
Ullrich Schäfer
0448a58e19
Fix remaining OpenSpec validation failures (medium-severity specs)
- komoot-import: add SHALL to Credential storage requirement body
- local-dev-environment: add SHALL to Mobile app dev requirement body
- multi-day-routes: demote ### Note heading to plain paragraph

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 11:26:32 +02:00
Ullrich Schäfer
bbb729ffdd
Fix OpenSpec validation failures on high-severity specs
- road-type-coloring: add proper ## Purpose/## Requirements structure (redirect file)
- planner-journal-handoff: add inline #### Scenario: blocks to each requirement; add SHALL to JWT token requirement
- osm-tile-overlays: add SHALL keyword to profile-aware requirement body
- shared-packages: add #### Scenario: blocks to ui, api, db, jobs, sentry-config requirements

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 11:20:14 +02:00
Ullrich Schäfer
0cf87b72ab
Fix medium-severity spec drift across 17 specs
- authentication-methods: document completeAuth mode param ("redirect"|"json");
  clarify add-passkey nudge (no dismiss mechanism, disappears on passkey add)
- journal-auth: session maxAge is 30 days; terms allow-list uses /legal/ prefix
  matching (broader than fixed list of paths)
- session-notes: mark awareness isolation and UndoManager isolation as not yet
  implemented (shared instances in current code)
- activity-feed: add fan-out scenario for visibility change to public
- explore: note that ?perPage is not yet implemented (hardcoded page size)
- multi-day-routes: add per-day GPX track split scenario (splitByDays option);
  document overnight vs isDayBreak naming gap
- osm-poi-overlays: debounce is 800ms + 2000ms min interval (not 500ms);
  retry is not automatic (fires on next viewport change)
- brouter-integration: rate limit corrected to 300/hour; add segment-cache
  requirement (client caches per-pair segments)
- wahoo-route-push: OAuth state shape uses camelCase (returnTo, pushAfter
  object) not snake_case with push_after boolean
- komoot-import: document noop adapter / ConnectedServiceManager bypass;
  note four Komoot-specific routes that bypass the generic OAuth framework
- background-jobs: exponential backoff not wired (retryLimit only); add SIGINT
- connected-services: add revoked status; name ConnectionNotActiveError
- infrastructure: add INTEGRATION_SECRET and SENTRY_DSN to env var lists;
  split secret decryption scenario by workflow (cd-apps vs cd-infra)
- secret-management: correct CD decryption — cd-apps only decrypts app.env;
  cd-infra decrypts both
- transactional-emails: welcome email is async (pg-boss job); magic-link email
  includes 6-digit numeric code
- journal-route-detail: websites are https: links (not mailto:); opening_hours
  is also displayed
- local-dev-environment: add mobile app (Expo) dev commands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 11:11:31 +02:00
Ullrich Schäfer
47eb2615ec
Fix high-severity spec drift across 10 specs
- rate-limiting: correct BRouter limit to 300/hour (was 60); add Overpass
  rate-limit requirement (120/min per IP)
- security-hardening: BROUTER_AUTH_TOKEN lives in secrets.app.env, not infra.env
- account-management: email-change verification does not re-auth; existing
  session stays valid
- planner-journal-handoff: full rewrite — documents the actual JWT callback
  architecture (edit-in-planner → POST /api/sessions → callback endpoint),
  token claims, notes round-trip via GPX <metadata><desc>, session lifecycle
- route-drag-reshape: rewrite to describe permanent segment midpoint handles
  (not proximity hover ghost marker); click-to-insert + waypoint drag model
- route-splitting: rewrite to match midpoint handle model; notes geometric
  midpoint placement (not cursor-snapped)
- road-type-coloring: redirect to route-coloring (all requirements already
  covered there)
- osm-tile-overlays: mark profile-aware auto-enable as not yet implemented
  (profileOverlayDefaults exported but not wired)
- osm-poi-overlays: zoom threshold is 10 not 12; user override persistence
  marked as not yet implemented
- shared-packages: add all 7 missing packages (map-core, fit, api, db, jobs,
  sentry-config, correct map description); document map vs map-core boundary

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 11:07:03 +02:00
Ullrich Schäfer
771ae57931
Archive komoot-import openspec change and update SOPS secrets
- Stage deletions of openspec/changes/komoot-import/ (files were moved
  to archive via shell mv, not git mv, so deletions weren't staged)
- Include SOPS secrets.app.env update adding INTEGRATION_SECRET

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 21:33:53 +02:00
Ullrich Schäfer
ae23d31a5f
Wire INTEGRATION_SECRET into docker-compose and CI; archive komoot-import
- Add INTEGRATION_SECRET to journal service in docker-compose.yml with
  :? guard so a missing value fails loudly at compose-up time
- Add INTEGRATION_SECRET to E2E test step in ci.yml via GitHub secret
  (unit tests already set their own value in the test file)
- Archive openspec/changes/komoot-import → archive/2026-05-23-komoot-import
- Sync delta specs: new openspec/specs/komoot-import/spec.md,
  updated openspec/specs/route-management/spec.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 21:10:08 +02:00
Ullrich Schäfer
5c299ab9f3
Mark all komoot-import tasks complete 2026-05-23 10:19:28 +02:00
Ullrich Schäfer
fef4051838
Add public import mode to Komoot import spec
Extends the Komoot import change to support two connection modes:

- Public mode: user places their trails.cool profile URL in their Komoot
  bio field; trails.cool verifies ownership via the unauthenticated public
  API (content_text field), then imports public tours with no credentials stored
- Authenticated mode: existing email + password flow, imports all tours
  including private ones

The profile URL verification doubles as cross-platform discovery — the
link stays in the Komoot bio permanently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 09:59:51 +02:00
Ullrich Schäfer
d1a3701720
Fix e2e-testing spec: add missing Purpose section
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 22:01:33 +02:00
Ullrich Schäfer
0b146f9b32
Split planner E2E tests into focused feature files
Breaks the monolithic planner.test.ts (581 lines, 25 tests) into five
focused files grouped by feature area, with BRouter mocked by default
via test.beforeEach in files that need routing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 21:58:10 +02:00
Ullrich Schäfer
b4dd01db41
Fix spec format and E2E test robustness
- Fix openspec/specs/waypoint-notes/spec.md: replace delta-spec header
  with proper ## Purpose + ## Requirements structure so validation passes
- Fix GPX export E2E assertion: check gpxText directly instead of regex
  matching on lat coordinate; use click-elsewhere-to-blur instead of
  .blur() for more reliable note save timing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 00:10:34 +02:00
Ullrich Schäfer
41217ef658
Add E2E tests for waypoint notes and nearby POI snap; archive waypoint-notes
- E2E: type a note, blur, verify it persists and appears in GPX <desc> on <wpt>
- E2E: mock Overpass, select waypoint, verify Nearby list, snap to POI,
  verify note prefix prepended
- Sync waypoint-notes delta spec → openspec/specs/waypoint-notes/spec.md
- Archive waypoint-notes change

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 00:06:07 +02:00
Ullrich Schäfer
c2abb64ee0
Add per-waypoint notes and nearby POI snap to Planner
- `note` field on Waypoint type, stored in Yjs Y.Map and exported as
  `<desc>` inside `<wpt>` in GPX
- Inline textarea note editing in WaypointSidebar with auto-resize,
  character counter (500 max), save-on-blur, Escape cancel
- Note indicator dot on map markers; note tooltip on hover
- `useNearbyPois` hook: fetches POIs within 500m of selected waypoint
  via Overpass proxy, 500ms debounce, AbortController, 60s rate-limit
  suppression
- NearbyPoiMarkers component: renders POI markers on map for selected
  waypoint with snap-to-POI on click
- Nearby section in WaypointSidebar: list with snap buttons, spinner,
  empty/rate-limited states, "Show more" toggle (5 → all)
- Unit tests for fetchNearbyPois bbox geometry and snap-to-POI Yjs
  transaction behaviour

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 23:58:21 +02:00
Ullrich Schäfer
faf2227896
Archive journal-poi-details change; sync journal-route-detail spec
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 23:33:37 +02:00
Ullrich Schäfer
861701e881
Show POI details (phone, website, opening hours) on Journal route detail page
Waypoints snapped to OSM POIs in the Planner now carry their metadata all
the way through to the Journal:

- Extend Waypoint type with osmId and poiTags fields
- Extract osmId/poiTags from Yjs Y.Map in ExportButton and SaveToJournalButton
- Encode POI metadata as <trails:poi> extensions in GPX <wpt> elements
- Parse <trails:poi> extensions back in the GPX parser
- Display phone, website, opening hours, address on Journal route detail
- E2E test for the full roundtrip; seed endpoint now defaults to public visibility

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 23:32:38 +02:00
Ullrich Schäfer
970e0a0755
Archive local-dev-stack change; sync delta spec to main
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 23:12:24 +02:00
Ullrich Schäfer
03791e981d
Mark verification tasks complete
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 23:12:24 +02:00
Ullrich Schäfer
10e0a8d41a
Local dev stack improvements
- Extend docker-compose.dev.yml with optional monitoring profile
  (Prometheus, Grafana, Loki) via --profile monitoring
- Align dev PostgreSQL with production: pg_stat_statements + init scripts
- Add scripts/seed.ts with idempotent Berlin test data (user, route, activity)
- Add pnpm db:seed and pnpm dev:reset scripts
- Simplify CI e2e job: replace manual docker run + BRouter setup with
  docker compose up --wait; add db:seed step
- Improve scripts/dev.sh: Docker check, --wait health checks, --monitoring
  flag, seed step
- Add scripts/reset-dev.sh to wipe and restart the local stack
- Add .env.development.example with documented local defaults

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 23:12:22 +02:00
Ullrich Schäfer
61b787ca84
Fix staging-environment spec: add missing Purpose section
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 20:43:00 +02:00
Ullrich Schäfer
0e77ac7831
Update local-dev-stack proposal: staging now exists
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 20:41:03 +02:00
Ullrich Schäfer
9d4c7d2b41
Archive staging-environments change; update local-dev spec
Staging environments are live and fully operational. Marks the remaining
verification tasks complete, syncs the delta spec to main specs, and
archives the change.

Also adds a "When to Use Local vs. Staging" reference table to the
local-dev-environment spec so the boundary between the two environments
is documented in one place.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 20:38:07 +02:00
Ullrich Schäfer
6c5fb6df0e
Spec drift catch-up: FIT location, notification payload fields, shared-packages index
- wahoo-import: clarify fitToGpx lives at connected-services/fit.ts (shared
  across providers) not inside the wahoo directory
- notifications: correct follow payload field names to followerUsername,
  followerDisplayName, targetUsername, targetDisplayName (matches code)
- shared-packages: add @trails-cool/fit package entry + CAPABILITIES.md index

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 16:02:13 +02:00
Ullrich Schäfer
abc3fbaa5b
Archive atomic-gpx-save + sync gpx-save spec to main
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 15:09:15 +02:00
Ullrich Schäfer
b17f8eb02a
Revert archive of atomic-gpx-save (will redo via skill) 2026-05-10 15:07:54 +02:00
Ullrich Schäfer
b4ef8b0e0e
Archive atomic-gpx-save change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 15:06:55 +02:00
Ullrich Schäfer
78b8b8f55f
Atomic GPX save: validate + persist row + geometry in one transaction
Eliminates the silent-failure pattern where a route/activity row could
be committed with gpx IS NOT NULL but geom IS NULL if the PostGIS write
failed after the row insert.

- New gpx-save.server.ts owns all GPX validation (GpxValidationError,
  validateGpx) and PostGIS geometry writes (writeGeom, tx-aware)
- createRoute, updateRoute, createActivity, createRouteFromActivity all
  wrapped in db.transaction() covering row + geom + version snapshot
- demo-bot uses createRoute/createActivity instead of raw inserts;
  errors propagate loudly
- Callback endpoint returns 400 for GpxValidationError instead of 401
- ADR-0006 documents the invariant for future explorers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 15:02:56 +02:00
Ullrich Schäfer
e268aeec10
Archive unify-auth-completion + sync spec delta (task 5.1)
Applies the ADDED requirement from
openspec/changes/unify-auth-completion/specs/authentication-methods/
into openspec/specs/authentication-methods/spec.md:

- Single web auth completion chokepoint (completeAuth at
  apps/journal/app/lib/auth/completion.server.ts) with five scenarios
  covering passkey register/login finish, magic-link verify-code,
  magic-link click-through, and returnTo sanitization.

Path in the synced requirement is .server.ts (the post-rename name),
not the .ts the delta originally captured.

Change moved to openspec/changes/archive/2026-05-08-unify-auth-completion/.
15/15 tasks complete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 03:02:51 +02:00
Ullrich Schäfer
b9aac2859a
Drop auth.server.ts re-exports + rename to .server.ts convention (task 5.2)
Two cleanups in one pass:

1. Update import paths app-wide from `~/lib/auth.server` to
   `~/lib/auth/session.server` for the four session helpers
   (sessionStorage, createSession, getSessionUser, destroySession).
   ~40 files: 33 simple path swaps where the file imported only session
   symbols, 5 splits where it also imported per-method auth functions
   (auth.verify.tsx, api.settings.email.ts, activities.\$id.tsx,
   routes.\$id.tsx, auth.accept-terms.tsx) — those keep one import
   from auth.server (for verifyMagicToken, canView, recordTermsAcceptance,
   etc.) and gain a second import from auth/session.server.
   Two more files used relative paths and were missed by the first
   grep pass (lib/oauth.server.ts and routes/oauth.authorize.tsx) —
   migrated too.
   The @deprecated re-exports block in auth.server.ts is gone.

2. Rename the new auth files to follow the project's `.server.ts`
   convention so Vite/React Router treat them as server-only (they
   read process.env.SESSION_SECRET, hit the DB, etc. — must NOT enter
   the client bundle):
   - auth/session.ts → auth/session.server.ts
   - auth/completion.ts → auth/completion.server.ts
   - auth/completion.test.ts → auth/completion.server.test.ts
   Done with `git mv` so blame is preserved.

Verified: typecheck + lint green; 126 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 03:01:30 +02:00
Ullrich Schäfer
fc2f938cf5
Mark task 4.3 (manual smoke) complete
All four flows verified locally over plain HTTP: passkey register,
passkey login, logout, magic-link 6-digit verify-code, magic-link
click-through. Session cookie set + correct redirect every time.

13/14 tasks done. Only 5.1 (spec delta sync at /opsx:archive time)
and 5.2 (optional follow-up to drop the auth.server.ts re-exports)
remain — neither blocks merge.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 02:55:19 +02:00
Ullrich Schäfer
d64c47614d
Implement completeAuth chokepoint + caller migration
Implements all of unify-auth-completion (12/14 tasks done; manual
smoke + archive-time spec sync remain).

Design refinement during implementation: completeAuth supports two
response shapes via a `mode` parameter:
- mode: 'redirect' (loaders / direct browser navigation; auth.verify.tsx)
- mode: 'json' (action handlers called by imperative fetch from
  client forms; api.auth.login, api.auth.register)

Both modes share createSession + safeReturnTo + Set-Cookie. JSON mode
carries `{ ok: true, step: "done", redirectTo }` (the `step` field
preserves the existing client-form check).

Why two modes: passkey ceremonies are inherently imperative
(start → browser WebAuthn API → finish), so action handlers can't
move to <Form>/useFetcher. Picking option (B) from the design grill —
the chokepoint owns destination selection while clients navigate —
required this dual shape. The 3 hardcoded client-side targets
(returnTo ?? "/", "/", "/?add-passkey=1") collapse into 1 server-side
sanitization pass (safeReturnTo) inside completeAuth.

New module:
- apps/journal/app/lib/auth/session.ts: cookie session storage
  (sessionStorage, createSession, getSessionUser, destroySession)
  moved from auth.server.ts. Legacy import path kept via re-exports
  with @deprecated JSDoc.
- apps/journal/app/lib/auth/completion.ts: completeAuth + safeReturnTo.
- apps/journal/app/lib/auth/completion.test.ts: 10 contract tests
  covering both modes, returnTo sanitization (path-relative, protocol-
  relative, absolute-URL, malformed), Set-Cookie attachment, redirect
  status, JSON shape.

Caller migration:
- api.auth.register.ts passkey-finish → completeAuth(json)
- api.auth.login.ts finish-passkey → completeAuth(json)
- api.auth.login.ts verify-code → completeAuth(json)
- auth.verify.tsx magic-link consumer → completeAuth(redirect)

Client form updates:
- auth.login.tsx: pass returnTo in fetch body, read result.redirectTo
  on done.
- auth.register.tsx: pass returnTo: "/?add-passkey=1" for the magic-
  link verify-code path (preserves the post-register passkey prompt
  via the chokepoint's safeReturnTo, instead of hardcoding it
  client-side).

Verified:
- pnpm typecheck && pnpm lint: green across all 15 workspaces.
- pnpm --filter @trails-cool/journal test: 126 passed.
- pnpm test:e2e auth: 4/4 passed without modification — confirms the
  refactor is behaviour-preserving for the user-facing flows that
  matter most (passkey register + login).

Spec delta in openspec/changes/unify-auth-completion/specs/ applies at
/opsx:archive time.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 02:38:15 +02:00
Ullrich Schäfer
7e22f1260b
Add unify-auth-completion architecture artifacts
Extract the post-verify orchestration shared across passkey
register-finish, passkey login-finish, magic-link verify-code, and
magic-link click-through into a single completeAuth function. Two
ADRs record the decision:

- ADR-0004: centralize web auth completion (record terms + create
  session + redirect) in apps/journal/app/lib/auth/completion.ts.
- ADR-0005: explicitly no AuthMethod polymorphism. Passkey + magic-
  link is the entire identity surface; OAuth2/PKCE is session
  transport, not a peer method. Recorded as a negative decision so
  future architecture passes don't re-suggest extracting the
  interface.

CONTEXT.md gains an Authentication section covering completeAuth, the
two methods, the OAuth2-as-transport distinction, and where the Terms
gate enforcement lives (root loader for web, requireApiUser for API
per the just-merged mobile-terms-gate).

OpenSpec change unify-auth-completion captures the proposal, design
(5 decisions including the negative-scope choices), spec delta on
authentication-methods, and 14 tasks. Implementation follows on this
branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 02:18:06 +02:00
Ullrich Schäfer
1e43e96732
Archive mobile-terms-gate + sync spec delta
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 02:02:24 +02:00