Archive demo-activity-bot, pg-boss-background-jobs, configurable-demo-persona
Fold completed deltas into main specs (activity-feed, route-management, infrastructure, planner-session), add new background-jobs and demo-activity-bot capability specs, and move the three change dirs to openspec/changes/archive/. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer
parent
0254b2afd7
commit
91e80ace36
25 changed files with 231 additions and 12 deletions
|
|
@ -0,0 +1,2 @@
|
|||
schema: spec-driven
|
||||
created: 2026-04-19
|
||||
|
|
@ -0,0 +1,108 @@
|
|||
## Context
|
||||
|
||||
The just-merged `demo-activity-bot` spec gave `trails.cool` a synthetic public user — Bruno, a Berlin dog on walkabout. The implementation works but hardcodes everything a host operator would sensibly want to customise: `username = "bruno"`, a fixed display name + bio, two string pools of English/German walk-name copy, and an inner-Berlin bbox as the default region.
|
||||
|
||||
Self-hosted instances are the core of this project's federation story. We don't want every federated instance running a Berlin-themed dog walker; that ranges from confusing (why does the Tokyo instance have a Berlin-named bot?) to actively off-brand (the persona voice is playful — not every community wants playful). If turning on `DEMO_BOT_ENABLED=true` ships an identity that isn't theirs, most operators just won't turn it on.
|
||||
|
||||
The bot itself (decide-to-walk gate, cap, BRouter client, retention, synthetic column, metrics) is generic and has no reason to change. Only the *persona* — the identity + voice — needs to become operator-supplied.
|
||||
|
||||
Region is already configurable via `DEMO_BOT_REGION`. That stays as-is; this proposal does not merge region into the persona blob because regions and personas are orthogonal (an operator may want to run the default Bruno persona but in a different bbox for testing).
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- A single env var — `DEMO_BOT_PERSONA` — that accepts either an inline JSON blob or a `file:` URL pointing at a mounted JSON file, so operators can ship either via SOPS env or via a config volume.
|
||||
- Built-in default remains exactly the current Bruno persona, so `trails.cool` needs no operator action when this ships.
|
||||
- Persona supplies: `username`, `displayName`, `bio`, `locales: ("en" | "de")[]`, and `content: { names: { en?: string[]; de?: string[] }, descriptions: { en?: string[]; de?: string[] } }`.
|
||||
- `/users/<username>` continues to render the 🐕 demo badge, but the gate moves from the hardcoded string `"bruno"` to a loader-supplied `isDemoUser` boolean derived from the persona.
|
||||
- Malformed JSON, missing required fields, or an unreachable `file:` path fall back to the built-in Bruno persona and emit a single warn-level log line at worker boot. The worker never crashes over a bad persona file.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Runtime reload. The persona is read once at worker boot. Operators restart the journal container to pick up a new persona. Live reload adds complexity for no user value (operators change personas monthly at most).
|
||||
- Per-request persona selection. There is exactly one demo user per instance.
|
||||
- Allowing the persona to be edited in the web UI by an admin. That would be a separate `admin-persona-editor` change; out of scope here.
|
||||
- Supporting arbitrary locales beyond EN + DE. The app only has those two locale bundles today, and adding locales is a cross-cutting change tracked elsewhere.
|
||||
- Moving region into the persona blob. Region and persona are orthogonal; two env vars remain two env vars.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1. One JSON blob via env, with `file:` fallback for multi-line content.
|
||||
|
||||
Go with a single `DEMO_BOT_PERSONA` env var. Accept two shapes:
|
||||
|
||||
1. Inline JSON — `DEMO_BOT_PERSONA='{"username":"bruno",...}'`. Fine for short personas.
|
||||
2. `file:`-prefixed path — `DEMO_BOT_PERSONA=file:/etc/trails-cool/persona.json`. For realistic personas the pools are ~10–15 entries each; inlining 50 strings of JSON into a SOPS env file is miserable.
|
||||
|
||||
*Alternatives considered:*
|
||||
- **Separate env vars for each field** (`DEMO_BOT_USERNAME`, `DEMO_BOT_DISPLAY_NAME`, `DEMO_BOT_NAME_POOL_EN`, …). Rejected: 10+ related env vars, and list-valued vars need an ad-hoc separator that invites quoting bugs.
|
||||
- **TOML or YAML file**. Rejected: the rest of the repo uses JSON env blobs (`DEMO_BOT_REGION` is already JSON). Stay consistent.
|
||||
- **`DEMO_BOT_PERSONA_FILE` as a separate env var**. Rejected: two env vars for one concept. The `file:` prefix is a well-known convention (12-factor app config).
|
||||
|
||||
### D2. Zod schema for validation.
|
||||
|
||||
Parse with a Zod schema that enforces: username matches `^[a-z0-9][a-z0-9_-]{1,30}$` (same constraint as registration), `displayName` and `bio` are non-empty strings up to 200 chars, `locales` is a non-empty subset of `["en", "de"]`, each `content.names.<locale>` and `content.descriptions.<locale>` is an array of 3–50 non-empty strings.
|
||||
|
||||
Reason for the 3–50 bound: fewer than 3 makes repeats painfully obvious in the daily feed; more than 50 eats unneeded memory and suggests the operator should move to a content pipeline instead of hand-editing JSON.
|
||||
|
||||
*Alternatives considered:*
|
||||
- **No validation — trust the operator.** Rejected: malformed persona is a silent footgun. A bad regex in `username` is a violation of the DB constraint and would crash `ensureDemoUser()` on boot.
|
||||
- **Hand-rolled runtime type checks.** Rejected: the repo already uses Zod in `@trails-cool/api` for request validation. Reuse the pattern.
|
||||
|
||||
### D3. Persona loading happens once, at worker boot, cached for process lifetime.
|
||||
|
||||
Add `loadPersona()` that returns a `DemoPersona` object. It's called from `demo-bot.server.ts` module init (not from each job handler) and the result is frozen. `ensureDemoUser()` and the job handlers all read from this module-level constant.
|
||||
|
||||
Side effect: if an operator changes the persona JSON and the worker is already running, the change doesn't take effect until the container restarts. This is the intended behaviour (see non-goals) and is the same pattern as `DEMO_BOT_REGION` + `DEMO_BOT_RETENTION_DAYS` today.
|
||||
|
||||
### D4. Built-in default persona lives in code, not a default JSON file.
|
||||
|
||||
The existing Bruno strings become a `DEFAULT_PERSONA: DemoPersona` export in `demo-bot.server.ts`. When no `DEMO_BOT_PERSONA` is set, or validation fails, `loadPersona()` returns the default. This keeps the happy path (no operator config) identical to what just shipped.
|
||||
|
||||
*Alternatives considered:*
|
||||
- **Bundle a `personas/bruno.json` in the image and set `DEMO_BOT_PERSONA=file:...` by default.** Rejected: more moving parts for no benefit. Everyone who reads the code already sees the strings; shipping them as a JSON asset doesn't improve readability and adds a runtime read on every boot.
|
||||
|
||||
### D5. The 🐕 demo badge becomes a loader-supplied flag.
|
||||
|
||||
Today `users.$username.tsx` checks `user.username === "bruno"` client-side. That was fine when Bruno was a literal. Now the check must know which username the running instance chose as its demo user. Two options:
|
||||
|
||||
- **Ship the persona username to the client** so the comparison stays client-side.
|
||||
- **Have the loader compute `isDemoUser` and pass a boolean to the component.**
|
||||
|
||||
Choose the loader path. The persona username is not otherwise needed in the client, and shipping instance config through HTML reads as a small information leak even if it's technically public. `isDemoUser` is crisp.
|
||||
|
||||
### D6. No migration. No backwards-compatibility shim. No deprecation window.
|
||||
|
||||
This change is additive: new env var, default preserves current behaviour. There is nothing to migrate. The hardcoded string constants (`DEMO_USERNAME`, `DEMO_DISPLAY_NAME`, `DEMO_BIO`, `NAME_POOL_*`, `DESCRIPTION_POOL_*`) are removed and folded into `DEFAULT_PERSONA`. Importers of `DEMO_USERNAME` (there are none) would break — but the grep is clean.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[Risk] Operator supplies a persona that conflicts with an existing real user's username on first enable.**
|
||||
→ `ensureDemoUser()`'s insert currently ON CONFLICT DO NOTHING on `username` would silently pick up the existing real user as the demo user's owner — all subsequent synthetic rows would attach to that real user. Fix: before insert, SELECT the username; if it exists AND the row is not already marked demo, log an error and disable the bot for this process. Track via a new `demo-bot persona username clash` log line; the operator sees a loud error on boot.
|
||||
|
||||
**[Risk] Operator supplies a persona with empty content pools.**
|
||||
→ The Zod schema rejects arrays shorter than 3. Worker falls back to the default persona and logs the rejection reason.
|
||||
|
||||
**[Risk] A `file:`-referenced JSON file is unreadable at boot (permissions, wrong mount).**
|
||||
→ `loadPersona()` catches the read error and falls back to the default persona with a warn log. The bot stays functional on the built-in Bruno.
|
||||
|
||||
**[Risk] The demo badge is skipped on a legitimately-chosen username that happens to clash with a real user.**
|
||||
→ Not a real issue in practice: the badge reads `isDemoUser` from the loader, which is derived by `userId === personaUserId`, not by username comparison. Two users can never share an ID.
|
||||
|
||||
**[Trade-off] One JSON blob vs. typed env vars.**
|
||||
→ JSON is harder to lint in SOPS/CI than `KEY=value` pairs. Mitigated by the `file:` mode for nontrivial personas, and by the Zod validation surfacing the parse error at worker boot.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
This is a no-op for `trails.cool` itself:
|
||||
|
||||
1. Merge + deploy. Nothing changes — the built-in Bruno persona is the default, produced by the exact same strings the bot generates today.
|
||||
2. For a self-hosted instance that wants its own persona: write `persona.json`, mount it or inline it via SOPS, set `DEMO_BOT_PERSONA=file:/path/to/persona.json`, set `DEMO_BOT_ENABLED=true`, and restart the journal container.
|
||||
|
||||
Rollback is trivial: unset `DEMO_BOT_PERSONA` and restart. The default persona returns.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should the persona schema include an emoji / glyph override so a cat-themed instance can replace the 🐕 in the badge? Leaning yes but the glyph is i18n'd today (lives in the `demo.badge` key), which complicates things. Defer — ship without it, revisit if a real host asks.
|
||||
- Should we expose a `/api/demo-persona` endpoint for federation peers to fetch the current persona? No real need today; a federating peer doesn't need to know that a remote user is synthetic, only that their content is public. Park.
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
## Why
|
||||
|
||||
The demo-activity-bot is hardcoded to "Bruno the Berlin dog walker" — a persona that reads well for `trails.cool` itself but makes no sense for a self-hosted instance in Tokyo, Denver, or Edinburgh. If we want federated hosts to turn the demo on, each instance needs to choose its own demo identity, region, and copy so visitors see a plausible local-voice feed rather than a stranger's Berlin park patrol.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Make the demo user's identity (username, display name, bio) configurable per instance via env — today `ensureDemoUser()` inserts `bruno` with a fixed display name and bio.
|
||||
- Allow each instance to supply its own generated-content pools (route-name pool and description pool) in EN + DE — today these are hardcoded string arrays in `demo-bot.server.ts`.
|
||||
- Allow each instance to restrict the demo to its preferred locale(s) — today the bot picks EN or DE with a coin flip.
|
||||
- Keep all configuration in a single `DEMO_BOT_PERSONA` JSON blob (env or file path) so operators can supply one artifact instead of threading five env vars.
|
||||
- Retain the current built-in Bruno persona as the default when no override is supplied, so `trails.cool` itself keeps its current behavior with no operator action.
|
||||
- Keep `DEMO_BOT_ENABLED`, `DEMO_BOT_REGION`, and `DEMO_BOT_RETENTION_DAYS` as they are today (region is a separate concern from persona; retention + enable-flag are orthogonal).
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
_None._
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `demo-activity-bot`: the bot's identity and generated copy become a **persona contract** supplied by the operator rather than a fixed Bruno/Berlin combination. The generation gate, retention, cap, and synthetic flag are unchanged.
|
||||
|
||||
## Impact
|
||||
|
||||
- **Code:** `apps/journal/app/lib/demo-bot.server.ts` — replace the four hardcoded `*_POOL_*` arrays and the three `DEMO_*` constants with a loaded-once `DemoPersona` object. `ensureDemoUser()` consumes the persona's `username`/`displayName`/`bio`. `templateName`/`templateDescription` read from the persona's pools.
|
||||
- **UX:** `/users/<username>` — the 🐕 demo badge needs to gate on a runtime-known username, not a literal `"bruno"`. Move the gate to loader data (`isDemoUser` boolean) rather than a client-side string check.
|
||||
- **Ops:** `infrastructure/.env.example`, `docker-compose.yml` — add `DEMO_BOT_PERSONA` passthrough; document the JSON shape. Existing `DEMO_BOT_REGION` et al. stay.
|
||||
- **Docs:** a short persona-authoring note in `docs/` (one page) so host operators know the JSON shape + sensible pool sizes.
|
||||
- **Tests:** new unit tests for `loadPersona()` (env → persona, fallback to built-in Bruno, malformed JSON → fallback + warn). Existing integration + E2E tests continue to work unchanged because the default persona is still Bruno.
|
||||
- **No breaking change for `trails.cool`:** the built-in Bruno persona remains the default and is exactly the current hardcoded strings.
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Persona configuration
|
||||
The Journal SHALL load a demo persona — username, display name, bio, supported locales, and per-locale content pools — from configuration at worker boot, and SHALL fall back to a built-in default persona if no override is supplied or the supplied override fails validation.
|
||||
|
||||
#### Scenario: No override supplied — built-in default applies
|
||||
- **WHEN** the worker starts with `DEMO_BOT_ENABLED=true` and `DEMO_BOT_PERSONA` is unset
|
||||
- **THEN** the bot uses the built-in default persona (`username=bruno`, playful Berlin-flavoured display name and bio, `locales=["en","de"]`, and the shipped Bruno-voiced name/description pools)
|
||||
- **AND** behaviour is identical to the demo bot before this change
|
||||
|
||||
#### Scenario: Inline JSON override
|
||||
- **WHEN** `DEMO_BOT_PERSONA` is set to a valid inline JSON object with `username`, `displayName`, `bio`, `locales`, and `content.names` / `content.descriptions` for each listed locale
|
||||
- **THEN** the bot uses that persona — `ensureDemoUser` inserts a row with the persona's username/displayName/bio/sentinel email, and subsequent generated routes and activities draw names and descriptions from the persona's per-locale pools
|
||||
|
||||
#### Scenario: File-backed override
|
||||
- **WHEN** `DEMO_BOT_PERSONA` is set to `file:<absolute-path>` and the referenced file contains a valid persona JSON object
|
||||
- **THEN** the worker reads the file once at boot and uses its contents as the persona
|
||||
|
||||
#### Scenario: Invalid JSON or schema violation → fall back
|
||||
- **WHEN** `DEMO_BOT_PERSONA` is set but the value is not valid JSON, or fails the persona schema (bad username pattern, empty or too-short pool, unsupported locale, etc.)
|
||||
- **THEN** the worker logs a warn-level entry describing the first validation failure and uses the built-in default persona
|
||||
- **AND** the bot continues to run
|
||||
|
||||
#### Scenario: File-backed path unreadable → fall back
|
||||
- **WHEN** `DEMO_BOT_PERSONA=file:<path>` but the file cannot be read (missing, permission denied, not a file)
|
||||
- **THEN** the worker logs a warn-level entry and uses the built-in default persona
|
||||
|
||||
### Requirement: Persona username clash detection
|
||||
The Journal SHALL refuse to attach the demo bot to a pre-existing non-demo user account when the supplied persona username collides with a human user already registered on the instance.
|
||||
|
||||
#### Scenario: Configured username belongs to a real user
|
||||
- **WHEN** the worker starts, the persona's username matches an existing `users` row, and that row has no marker identifying it as a prior demo user (i.e. it was registered via the normal signup flow)
|
||||
- **THEN** the worker logs an error-level "demo persona username clash" entry naming the colliding username
|
||||
- **AND** the generation + prune jobs are not scheduled for this process — the bot stays disabled until the operator picks a different username
|
||||
- **AND** the rest of the Journal continues to serve requests normally
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Demo user bootstrap
|
||||
The Journal SHALL ensure a dedicated bot user exists when the demo bot starts, creating it on first run if missing. The user's identity (username, display name, bio, sentinel email local-part) is derived from the active persona — either the operator-supplied persona or the built-in default.
|
||||
|
||||
#### Scenario: Bot user created on first run
|
||||
- **WHEN** the Journal worker starts with `DEMO_BOT_ENABLED=true` and no `users` row matches the persona's username
|
||||
- **THEN** a new `users` row is inserted with that username, the persona's display name, the persona's bio, a sentinel email `<username>@<domain>`, no passkey credentials, and `terms_accepted_at` + `terms_version` populated at the current version
|
||||
- **AND** subsequent worker startups are idempotent — no second row is inserted
|
||||
|
||||
#### Scenario: Demo user has no usable credentials
|
||||
- **WHEN** any request attempts to authenticate as the demo user via passkey or magic-link
|
||||
- **THEN** authentication fails because no passkey is registered and no mailbox receives magic-link mails
|
||||
|
||||
### Requirement: Synthetic content generation job
|
||||
The Journal SHALL run a recurring background job that generates one public route and one linked public activity for the demo user per run, subject to an env flag and a rate cap. The generated name and description are drawn from the active persona's per-locale content pools.
|
||||
|
||||
#### Scenario: Disabled in non-production environments
|
||||
- **WHEN** the `DEMO_BOT_ENABLED` env var is absent or any value other than `"true"`
|
||||
- **THEN** the job body is a no-op: no BRouter calls, no inserts, no errors
|
||||
|
||||
#### Scenario: Enabled generation flow (decide-to-walk fires)
|
||||
- **WHEN** `DEMO_BOT_ENABLED=true`, the local hour is within 07:00–21:00, the per-tick Bernoulli roll fires, and the hard cap has not been reached
|
||||
- **THEN** the job picks a random start and end point within the configured seed region, calls BRouter with the `trekking` profile, persists the returned GPX as a new route with `visibility='public'` and `synthetic=true`, and inserts a linked activity with the same GPX, `visibility='public'`, `synthetic=true`, a plausible `started_at` and `duration`, and a persona-voiced name + description sampled from one of the persona's supported locales
|
||||
- **AND** the route and activity are attributed to the demo user
|
||||
|
||||
#### Scenario: Locale restricted to a single language
|
||||
- **WHEN** the persona's `locales` list is `["en"]`
|
||||
- **THEN** every generated route's name and description come from the persona's English pool — the German pool is never sampled
|
||||
|
||||
#### Scenario: Decide-to-walk does not fire
|
||||
- **WHEN** the local hour is outside 07:00–21:00, or the Bernoulli roll does not fire
|
||||
- **THEN** the job returns without inserting anything
|
||||
|
||||
#### Scenario: BRouter failure is tolerated
|
||||
- **WHEN** the BRouter call returns no route, a rate-limit, or an error
|
||||
- **THEN** the job logs the failure, inserts nothing, and exits without throwing — the next scheduled tick retries
|
||||
|
||||
#### Scenario: Hard cap prevents runaway growth
|
||||
- **WHEN** there are already 40 or more synthetic items created in the last 14 days
|
||||
- **THEN** the job skips generation for that tick
|
||||
|
||||
#### Scenario: Singleton scheduling prevents overlap
|
||||
- **WHEN** a tick fires while the previous run is still executing
|
||||
- **THEN** the new tick is skipped (pg-boss singleton semantics) so the job cannot overlap itself
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
## 1. Persona type + schema
|
||||
|
||||
- [x] 1.1 Define `DemoPersona` TypeScript interface in `apps/journal/app/lib/demo-bot.server.ts` with `username`, `displayName`, `bio`, `locales: ("en"|"de")[]`, `content: { names: { en?: string[]; de?: string[] }, descriptions: { en?: string[]; de?: string[] } }`
|
||||
- [x] 1.2 Add a Zod schema mirroring the interface: username `^[a-z0-9][a-z0-9_-]{1,30}$`, displayName 1–200 chars, bio 0–200 chars, locales a non-empty subset of `["en","de"]`, each content array 3–50 non-empty strings, ensure every listed locale has both `names` and `descriptions` populated
|
||||
- [x] 1.3 Move the current hardcoded Bruno strings into a `DEFAULT_PERSONA: DemoPersona` export — confirm it round-trips through the Zod schema before export
|
||||
|
||||
## 2. Loader
|
||||
|
||||
- [x] 2.1 Add `loadPersona(): DemoPersona` that reads `DEMO_BOT_PERSONA`: if unset → `DEFAULT_PERSONA`; if it starts with `file:` → read the rest as an absolute path via `fs.readFileSync`; otherwise treat as inline JSON
|
||||
- [x] 2.2 Parse with the Zod schema; on validation failure or file-read failure, log a warn with the first error message and return `DEFAULT_PERSONA`
|
||||
- [x] 2.3 Cache the result in a module-level `Object.freeze`d constant so every caller sees the same instance
|
||||
- [x] 2.4 Remove the now-unused `DEMO_USERNAME`, `DEMO_DISPLAY_NAME`, `DEMO_BIO`, `NAME_POOL_EN`, `NAME_POOL_DE`, `DESCRIPTION_POOL_EN`, `DESCRIPTION_POOL_DE` constants — every reference must flow through the persona
|
||||
|
||||
## 3. Wire into bootstrap + generation
|
||||
|
||||
- [x] 3.1 `ensureDemoUser()` takes the loaded persona and writes `username`, `displayName`, `bio`, and sentinel email `${persona.username}@<domain>`
|
||||
- [x] 3.2 Before insert, `SELECT` the row matching the username; if it exists and was not inserted by a prior demo-bot boot (heuristic: email does not match the sentinel pattern), throw `DemoPersonaUsernameClashError`; server startup catches this and declines to schedule the demo jobs for the process
|
||||
- [x] 3.3 `templateName(startedAt, locale)` reads from `persona.content.names[locale]` with the existing seeded indexing; same for `templateDescription`
|
||||
- [x] 3.4 Locale selection in `generateOneWalk` picks randomly from `persona.locales` via `pickLocale()` (dropping the hardcoded 50/50 EN/DE coin flip)
|
||||
|
||||
## 4. UX — demo badge via loader flag
|
||||
|
||||
- [x] 4.1 In `users.$username.tsx` loader, compute `isDemoUser` by comparing `user.username` against `loadPersona().username`; include `isDemoUser` in the loader return
|
||||
- [x] 4.2 Replace the client-side `user.username === "bruno"` check with the loader-supplied boolean
|
||||
- [x] 4.3 Keep the `demo.badge` i18n key exactly as-is so existing translations continue to work
|
||||
|
||||
## 5. Env + docs
|
||||
|
||||
- [x] 5.1 Add `DEMO_BOT_PERSONA` passthrough in `infrastructure/docker-compose.yml` (journal service)
|
||||
- [x] 5.2 Add a commented `DEMO_BOT_PERSONA` example to `infrastructure/.env.example` with the inline JSON form AND the `file:/path/to/persona.json` form
|
||||
- [x] 5.3 Write `docs/demo-persona.md` — a one-page guide covering: schema shape, inline vs `file:` modes, required pool sizes, how the built-in default looks, and an example persona for a non-Berlin instance
|
||||
|
||||
## 6. Tests
|
||||
|
||||
- [x] 6.1 Unit: `loadPersona()` — unset env returns default; valid inline JSON parses to the persona; invalid JSON falls back + warns; valid JSON that violates the Zod schema falls back + warns
|
||||
- [x] 6.2 Unit: `loadPersona()` with `file:` — reads a real file via a temp path; unreadable file falls back + warns
|
||||
- [x] 6.3 Unit: `templateName`/`templateDescription` draw from the supplied persona's pool and never return entries outside it
|
||||
- [x] 6.4 Integration (`DEMO_BOT_INTEGRATION=1`): `ensureDemoUser` with the default persona is idempotent; clash with a pre-existing real user throws `DemoPersonaUsernameClashError`
|
||||
- [x] 6.5 E2E: no regression — `/users/bruno` still shows the 🐕 badge for the built-in Bruno persona (re-ran `e2e/demo-bot.test.ts` unchanged, both pass)
|
||||
|
||||
## 7. Rollout
|
||||
|
||||
- [x] 7.1 Merge + deploy — nothing changes because default persona equals current behaviour
|
||||
- [x] 7.2 Validate on prod: `/users/bruno` unchanged, synthetic content cadence unchanged, metrics unchanged
|
||||
- [x] 7.3 (Optional demo) On a staging or second instance, ship a non-Bruno persona via `DEMO_BOT_PERSONA=file:...` and confirm the demo user renders with the new identity + voice
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
schema: spec-driven
|
||||
created: 2026-04-19
|
||||
154
openspec/changes/archive/2026-05-03-demo-activity-bot/design.md
Normal file
154
openspec/changes/archive/2026-05-03-demo-activity-bot/design.md
Normal file
|
|
@ -0,0 +1,154 @@
|
|||
## Context
|
||||
|
||||
The Journal runs on a single Hetzner Cloud container with Postgres + PostGIS, BRouter as a sibling compose service, and pg-boss already configured as the background-job queue (`apps/journal/server.ts` boots a worker on startup). `public-content-visibility` introduces a `visibility` column on routes + activities; once that lands, a piece of content marked `public` is viewable by logged-out visitors at its permanent URL and on the owner's `/users/:username` profile.
|
||||
|
||||
This change layers a synthetic-content generator on top. The motivation is narrow — a non-empty feed for prospective users we're demoing to. The design matches that: **cheap, single-region, single-user, single-purpose**. Nothing here should generalise to "simulate a real community"; the bot is allowed to look slightly repetitive because it exists to fill an empty demo, not to fake scale.
|
||||
|
||||
Adjacent context:
|
||||
- BRouter runs internally and already rate-limits per-session at the Planner, but here we're inside the Journal server, so the bot controls its own cadence rather than sharing a limiter with user-driven routing. Load is low — at most a handful of routes per day.
|
||||
- pg-boss supports cron-style recurring schedules and singleton jobs (no concurrent duplicates), which are exactly what we need.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- A logged-out visitor landing on `trails.cool/users/demo` sees a recent-looking public profile with several routes and activities.
|
||||
- The content is generated entirely on-host, with no third-party data calls.
|
||||
- Synthetic content is trivially separable from real content (single boolean flag).
|
||||
- The bot can be disabled without a deploy (env flag) and its output wiped with a single DELETE.
|
||||
- Cadence is boring. One content item every few hours is enough — we are not trying to pretend a Strava-scale community.
|
||||
|
||||
**Non-Goals:**
|
||||
- Realistic social signals (likes, comments, follower counts).
|
||||
- Multiple bot personas or multi-region coverage.
|
||||
- Route topology tricks beyond "point-to-point via BRouter" (no multi-day, no loops, no no-go areas).
|
||||
- Photos or media on activities.
|
||||
- Running the bot in local dev / CI / staging — disabled by default everywhere except prod.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Single bot user: Bruno the trail dog
|
||||
|
||||
**Decision:** One account with:
|
||||
|
||||
- Username: `bruno` (reserved; a later real registration with that username is blocked)
|
||||
- Display name: `Bruno`
|
||||
- Bio: short, whimsical, something like *"Professional park inspector. Currently accepting tennis balls."*
|
||||
- Sentinel email: `bruno@<DOMAIN>` — unroutable, so magic-link login cannot succeed
|
||||
- No passkey credentials
|
||||
- `terms_accepted_at` / `terms_version` populated to the current values at bootstrap
|
||||
|
||||
The persona is a park-walking dog whose "human" logs the walks. This choice:
|
||||
|
||||
- Makes the bot legibly fictional at a glance — a dog account with emoji-laced names reads as *charming*, not as a real-user-you're-being-tricked-into-believing-in.
|
||||
- Narrows the generator's parameter space (trekking profile only, short distances, urban parks) so output is consistent and failure-prone edge cases get pruned.
|
||||
- Keeps the copy bank small and writable in one sitting.
|
||||
|
||||
**Alternatives considered:**
|
||||
- **"Finn" — realistic commuter-cyclist.** Fills the feed convincingly, but deception risk if a visitor later realises. Rejected.
|
||||
- **"trails.cool test pilot" — explicitly diagnostic.** Honest but dead boring; kills the "lively feed" goal the bot exists for. Rejected.
|
||||
- **Multi-persona community simulation.** Out of scope for v1.
|
||||
|
||||
### Bruno's generation parameters
|
||||
|
||||
**Decision:** The persona constrains the generator more tightly than the generic v1 draft:
|
||||
|
||||
- **Profile pool**: `["trekking"]` only (dogs don't ride bikes).
|
||||
- **Distance band**: 2–12 km crow distance between start and end — a realistic walk, not a hike.
|
||||
- **Region**: Berlin inner (bbox roughly `13.25,52.45,13.55,52.60`), biased toward parks/green areas: Grunewald, Tiergarten, Tempelhofer Feld, Volkspark Friedrichshain, Treptower Park, Müggelsee shoreline. The bbox itself is permissive; the copy just reads as park-flavoured.
|
||||
- **Started-at window**: 07:00–20:00 local, with a slight bias toward morning/evening ("walkies" times).
|
||||
- **Copy templates**: a mix of serious-sounding audit language ("Grunewald north-loop patrol") and comic ("Bruno found three sticks today 🐕"). Bilingual EN + DE.
|
||||
|
||||
**Why narrower:** the persona is a feature, not a limitation. Not every dog-walk needs to be in a park, but pretending Bruno walked 40 km across Brandenburg would break the illusion the whole persona exists to prop up.
|
||||
|
||||
### Route generation: BRouter point-to-point in a seed region
|
||||
|
||||
**Decision:** A configured seed region is a bounding box. For the Bruno persona:
|
||||
|
||||
- Region: Berlin inner (`13.25,52.45,13.55,52.60`) — narrower than "Berlin + Brandenburg"; a dog-walk spanning Brandenburg breaks the illusion.
|
||||
- Profile: `trekking` only (see the Bruno persona decision above).
|
||||
- Start point: random within the box.
|
||||
- End point: 2–12 km crow distance from start.
|
||||
- Query BRouter with those two waypoints and `trekking`; reject and retry if BRouter returns no route or the computed distance is outside a sanity band (say, 1.5–18 km real distance).
|
||||
- Persist the returned GPX as the route's source of truth and let existing enrichment compute the rest (PostGIS geom, distance, elevation).
|
||||
|
||||
The bbox stays env-configurable; changing `DEMO_BOT_REGION` relocates Bruno without a code change.
|
||||
|
||||
**Alternatives considered:**
|
||||
- **Pre-recorded GPX files in a corpus.** Chosen by user to be (b) — synthesised via BRouter — in the conversation that led here. Keeps variety cheap and relocatable (change the bbox, new city).
|
||||
- **Komoot import** — reject, external dependency.
|
||||
- **Multi-waypoint (3+ stops)** — slight realism gain, much higher generation failure rate (more chances for BRouter to return no-route). Skip.
|
||||
|
||||
### Activity generation follows the route
|
||||
|
||||
**Decision:** When a route is generated, immediately derive an activity from it:
|
||||
|
||||
- `route_id` links to the new route
|
||||
- `name` and `description` drawn from a small templated set (e.g., "Weekend ride through the Havelland", "Evening loop around Müggelsee")
|
||||
- `started_at` = today between 06:00 and 20:00 local (random), `duration` = distance ÷ an average speed per profile ± jitter
|
||||
- `distance`, `elevation_gain`, `elevation_loss` = the route's computed values
|
||||
- `gpx` = the route's GPX verbatim (the activity "happened" along the planned route)
|
||||
- `visibility = 'public'`, `synthetic = true`
|
||||
|
||||
**Why bundled:** keeping route + activity generation in one transaction means the feed item is coherent the moment it appears. Separating them just to pretend the user planned then rode feels like theatre — we're not hiding the bot, we're just filling the surface.
|
||||
|
||||
### Cadence and guards
|
||||
|
||||
**Decision:** A dog walks twice a day, sometimes three times, never six. The schedule should look like Bruno's day, not like a cron job.
|
||||
|
||||
- A recurring pg-boss job `demo-bot:generate` fires **every 90 minutes** as a singleton.
|
||||
- Most ticks decide to *not* walk: the handler rolls a per-tick probability and skips if it doesn't hit. The probability is **0.12 per tick during 07:00–21:00 local, 0 otherwise** — which yields roughly 2–3 walks per day across the day, biased to waking hours.
|
||||
- This both looks human (walks aren't on a fixed cron) and makes the generation load tiny: one BRouter call per walk, so ~2–3 per day.
|
||||
- The job skips itself entirely when `process.env.DEMO_BOT_ENABLED !== "true"`. Dev / CI / tests: no-op.
|
||||
- Hard cap: if there are already ≥ 40 synthetic items in the last 14 days (≈ 3/day × 14 rounded up), skip for that tick. Stops runaway growth if the retention job fails.
|
||||
- BRouter call per generation: 1. Bot traffic is rounding-error compared to real user traffic.
|
||||
|
||||
**Alternatives considered:**
|
||||
- **Fixed cron every 4 hours** — boring, too regular, a demo visitor with sharp eyes notices.
|
||||
- **Poisson process with mean 3/day** — more faithful but more code. The 90-minute-tick-with-probability scheme is close enough.
|
||||
- **Walk at specific realistic times** (morning, lunch, evening) — adds temporal pattern. Probably worth revisiting once we see the feed in practice, but not worth coding up front.
|
||||
|
||||
### Retention
|
||||
|
||||
**Decision:** A second recurring job `demo-bot:prune` runs daily and deletes synthetic routes + activities whose `created_at` is older than `DEMO_BOT_RETENTION_DAYS` (default 14). Route-version rows cascade-delete via existing FK.
|
||||
|
||||
**Why daily, not weekly:** smaller blast radius if the prune gets something wrong, and it keeps the feed visibly "recent" rather than stable.
|
||||
|
||||
### Flagging synthetic content at the DB level
|
||||
|
||||
**Decision:** Add `synthetic boolean NOT NULL DEFAULT false` to `journal.routes` and `journal.activities`. Set `true` only when the bot inserts.
|
||||
|
||||
**Why:** makes the "delete all bot content" operation a one-liner and lets future listing code exclude synthetic if we decide to, without introspecting content shape or owner. `owner_id = demo_user_id` would almost work as a proxy, but the dedicated flag decouples identity from status — if we later delete and re-seed the demo user we don't lose the signal.
|
||||
|
||||
### Env surface
|
||||
|
||||
**Decision:**
|
||||
- `DEMO_BOT_ENABLED`: `"true"` turns the generator + prune on; anything else is off. Absent means off.
|
||||
- `DEMO_BOT_RETENTION_DAYS`: integer, defaults to 14.
|
||||
- `DEMO_BOT_REGION`: JSON `{ "bbox": [w,s,e,n] }`, defaults to inner Berlin (`13.25,52.45,13.55,52.60`) to suit Bruno's park-walker persona.
|
||||
- No secrets. The whole feature is public-facing by design.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **BRouter outages halt the feed** → acceptable; the job logs and skips. The feed will stop refreshing but existing items remain visible.
|
||||
- **Uncanny repetition** (same start neighbourhoods, template names) → mitigated by varying the start point randomly and the templated copy, but still acceptable for a demo. A reviewer will obviously figure out it's synthetic if they look — we're not hiding it.
|
||||
- **Mistaking bot content for real content during analytics** → mitigated by the `synthetic` flag; analytics queries filter `WHERE synthetic = false` when they want real usage.
|
||||
- **Runaway insert if retention breaks** → mitigated by the hard cap (50 items in last 14 days) inside the generator.
|
||||
- **GDPR / privacy concerns** → none new: the bot has no real PII, its content is first-party, and `demo@trails.cool` is a reserved sentinel address.
|
||||
- **Someone logs in as `demo`** → the user has no credentials (no passkey, no magic-token). Email sentinel means a magic-link request can't succeed either (no inbox). Safe.
|
||||
- **Accidentally enabled in dev** → mitigated by the explicit env opt-in; default-off in every environment but prod.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Merge schema + code; `drizzle-kit push --force` adds `synthetic` columns.
|
||||
2. Set `DEMO_BOT_ENABLED=true` on the prod Journal container; leave every other environment off.
|
||||
3. On next worker restart, the bootstrap step inserts the `demo` user.
|
||||
4. First generation run produces one route + activity. Verify at `/users/demo`.
|
||||
5. After a few ticks, verify the feed looks plausible and the prune doesn't fire unexpectedly (it won't, nothing is 14 days old).
|
||||
|
||||
**Rollback:** `DEMO_BOT_ENABLED=false` and redeploy. To wipe all generated content: `DELETE FROM journal.activities WHERE synthetic = true; DELETE FROM journal.routes WHERE synthetic = true;`. The `demo` user row can stay — it's cheap and keeps the URL stable if we re-enable later.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **Do we want the profile to also get a "synthetic content" badge** so honest readers (and us, on demos) can tell at a glance? A tiny "demo account" pill on `/users/demo` is cheap. Decide during implementation.
|
||||
- **Should the bot post a few backfilled items on first enablement** so the feed isn't just one item for the first four hours? Probably yes — a small one-time bootstrap that generates 3–5 items immediately if the synthetic-item count is 0. Proposing it as a task.
|
||||
- **Should BRouter calls be recorded as `brouter_request_duration_seconds` the same as user requests**? Probably — same histogram is fine; synthetic traffic is a small addition and it keeps ops alerts meaningful.
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
## Why
|
||||
|
||||
To demo trails.cool to prospective users and contributors we need live-looking content on the public surface introduced by `public-content-visibility`. Today a demo link opens to an empty profile. Even after visibility ships, the Journal has 3 users and 0 activities — the feed shows "nothing happened today," which is the worst possible first impression.
|
||||
|
||||
A synthetic "demo" user that autonomously produces plausible activities closes that gap. It also keeps pressure on the content-creation code paths: if the bot is wedged, the import / routing / geometry pipelines are probably broken for real users too.
|
||||
|
||||
This change is the direct follow-up to `public-content-visibility` and is only useful once that lands.
|
||||
|
||||
## What Changes
|
||||
|
||||
- A dedicated bot user account (username e.g. `demo`) is bootstrapped on first run with a stable ID, a public display name, and `visibility` defaults that mark its content public.
|
||||
- A recurring pg-boss job (the existing background-jobs infrastructure on the Journal host) runs every few hours, chooses a random start + end point within a configured seed region, asks BRouter for a route, persists the route + a derived activity attributed to the demo user with `visibility = 'public'`.
|
||||
- Synthetic rows are explicitly flagged at the database level (`synthetic boolean NOT NULL DEFAULT false`) so a single DELETE can clean them up and listings can optionally exclude them later.
|
||||
- A retention job trims synthetic activities older than a configurable window (default 14 days) so the feed looks fresh, not a swamp of old rides.
|
||||
- Everything is gated behind a `DEMO_BOT_ENABLED` env flag. Local dev, CI, and tests default to disabled. The flag is set on prod only.
|
||||
- Route profiles vary: trekking and fastbike, with plausible distance ranges for each. Start times of day look reasonable. Names and descriptions draw from a small templated copy set (in both EN and DE).
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `demo-activity-bot`: A bot user, a scheduler that seeds plausible public routes + activities using BRouter and a seed region, and a retention job to keep the feed bounded.
|
||||
|
||||
### Modified Capabilities
|
||||
- `route-management`: adds a `synthetic` flag alongside the `visibility` work from `public-content-visibility`, so listings and exports can distinguish bot content if they ever need to.
|
||||
- `activity-feed`: same `synthetic` flag for symmetry.
|
||||
|
||||
## Impact
|
||||
|
||||
- **Code**: new `apps/journal/app/jobs/demo-bot.ts` for the job handlers, small helpers for picking seed coordinates and templating names, a one-time bootstrap that inserts the demo user if missing.
|
||||
- **Data**: a schema change (the `synthetic` column on two tables) plus ongoing inserts into those same tables. Bounded by the retention job.
|
||||
- **Infrastructure**: no new services — pg-boss is already in the stack. `DEMO_BOT_ENABLED` is a new env var on the prod Journal container.
|
||||
- **External calls**: BRouter calls from the bot; throttled at the scheduler level so the bot never competes with real user traffic. No external API beyond BRouter.
|
||||
- **Privacy**: the bot user has no real email, no PII, and its content is clearly author-attributed to `demo`. The legal / privacy pages do not need to change — this is first-party content the operator controls.
|
||||
- **Non-goals (explicit)**: realistic social signals (no likes/comments on demo content), multi-user bot personas (one user for v1), cross-region variety (one seed region), multi-day tours, photos or media attachments.
|
||||
- **Rollback**: flip `DEMO_BOT_ENABLED=false` — the job stops. A single `DELETE FROM ... WHERE synthetic = true` removes all bot content without touching real data.
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Synthetic activity flag
|
||||
The Journal SHALL persist a `synthetic` boolean on every activity so automated / demo content can be distinguished from user-created content.
|
||||
|
||||
#### Scenario: User-created activities default to non-synthetic
|
||||
- **WHEN** an activity is created through any user-facing flow (GPX upload, sync import, Planner handoff)
|
||||
- **THEN** the activity row is persisted with `synthetic = false`
|
||||
|
||||
#### Scenario: Bot inserts flag their rows as synthetic
|
||||
- **WHEN** the demo-activity-bot inserts an activity
|
||||
- **THEN** the row is persisted with `synthetic = true`
|
||||
|
||||
#### Scenario: Synthetic flag is not user-editable
|
||||
- **WHEN** an activity owner edits the activity via any user-facing action
|
||||
- **THEN** the stored `synthetic` value is not changed by the edit
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Demo user bootstrap
|
||||
The Journal SHALL ensure a dedicated bot user exists when the demo bot starts, creating it on first run if missing.
|
||||
|
||||
#### Scenario: Bot user created on first run
|
||||
- **WHEN** the Journal worker starts with `DEMO_BOT_ENABLED=true` and no `users` row matches the reserved demo username
|
||||
- **THEN** a new `users` row is inserted with the reserved username, a public-facing display name, a sentinel email (`demo@<domain>`), no passkey credentials, and `terms_accepted_at` + `terms_version` populated at the current version
|
||||
- **AND** subsequent worker startups are idempotent — no second row is inserted
|
||||
|
||||
#### Scenario: Demo user has no usable credentials
|
||||
- **WHEN** any request attempts to authenticate as the demo user via passkey or magic-link
|
||||
- **THEN** authentication fails because no passkey is registered and no mailbox receives magic-link mails
|
||||
|
||||
### Requirement: Synthetic content generation job
|
||||
The Journal SHALL run a recurring background job that generates one public route and one linked public activity for the demo user per run, subject to an env flag and a rate cap.
|
||||
|
||||
#### Scenario: Disabled in non-production environments
|
||||
- **WHEN** the `DEMO_BOT_ENABLED` env var is absent or any value other than `"true"`
|
||||
- **THEN** the job body is a no-op: no BRouter calls, no inserts, no errors
|
||||
|
||||
#### Scenario: Enabled generation flow (decide-to-walk fires)
|
||||
- **WHEN** `DEMO_BOT_ENABLED=true`, the local hour is within 07:00–21:00, the per-tick Bernoulli roll fires, and the hard cap has not been reached
|
||||
- **THEN** the job picks a random start and end point within the configured seed region, calls BRouter with the `trekking` profile, persists the returned GPX as a new route with `visibility='public'` and `synthetic=true`, and inserts a linked activity with the same GPX, `visibility='public'`, `synthetic=true`, a plausible `started_at` and `duration`, and a templated Bruno-voiced name/description
|
||||
- **AND** the route and activity are attributed to the demo user
|
||||
|
||||
#### Scenario: Decide-to-walk does not fire
|
||||
- **WHEN** the local hour is outside 07:00–21:00, or the Bernoulli roll does not fire
|
||||
- **THEN** the job returns without inserting anything — Bruno just isn't walking right now
|
||||
|
||||
#### Scenario: BRouter failure is tolerated
|
||||
- **WHEN** the BRouter call returns no route, a rate-limit, or an error
|
||||
- **THEN** the job logs the failure, inserts nothing, and exits without throwing — the next scheduled tick retries
|
||||
|
||||
#### Scenario: Hard cap prevents runaway growth
|
||||
- **WHEN** there are already 40 or more synthetic items created in the last 14 days
|
||||
- **THEN** the job skips generation for that tick
|
||||
|
||||
#### Scenario: Singleton scheduling prevents overlap
|
||||
- **WHEN** a tick fires while the previous run is still executing
|
||||
- **THEN** the new tick is skipped (pg-boss singleton semantics) so the job cannot overlap itself
|
||||
|
||||
### Requirement: Synthetic content retention
|
||||
The Journal SHALL run a recurring job that deletes synthetic routes and activities older than a configurable window.
|
||||
|
||||
#### Scenario: Prune removes old synthetic content
|
||||
- **WHEN** the prune job runs with `DEMO_BOT_RETENTION_DAYS=14`
|
||||
- **THEN** every row in `journal.routes` and `journal.activities` with `synthetic=true` and `created_at < now() - 14 days` is deleted
|
||||
- **AND** route-version rows cascade-delete via existing foreign keys
|
||||
- **AND** rows with `synthetic=false` are never touched
|
||||
|
||||
#### Scenario: Prune is a no-op when nothing is old
|
||||
- **WHEN** the prune job runs and no synthetic rows exceed the retention window
|
||||
- **THEN** no DELETE statements execute and the job returns normally
|
||||
|
||||
#### Scenario: Disabled in non-production environments
|
||||
- **WHEN** `DEMO_BOT_ENABLED` is not `"true"`
|
||||
- **THEN** the prune job body is a no-op
|
||||
|
||||
### Requirement: Initial backfill
|
||||
On first enablement (when no synthetic content exists yet) the Journal SHALL populate the demo profile with a small batch of items so the first visitor does not see a single-item feed.
|
||||
|
||||
#### Scenario: First enablement backfills several items
|
||||
- **WHEN** the bot is enabled for the first time and the count of synthetic routes is 0
|
||||
- **THEN** the generation job produces 3–5 items in sequence during its first run
|
||||
- **AND** subsequent runs produce one item at a time as normal
|
||||
|
||||
### Requirement: Seed region is configurable
|
||||
The Journal SHALL read the seed region (bounding box) from an env var so the deployment can change it without a code change.
|
||||
|
||||
#### Scenario: Env-configured region
|
||||
- **WHEN** `DEMO_BOT_REGION` is set to a JSON object containing a `bbox` array of four numbers `[west, south, east, north]`
|
||||
- **THEN** all generated start and end points fall within that box
|
||||
|
||||
#### Scenario: Sensible default
|
||||
- **WHEN** `DEMO_BOT_REGION` is unset
|
||||
- **THEN** the job uses a documented default region (inner Berlin) so that out-of-the-box runs still produce plausible Bruno-style walks
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Synthetic route flag
|
||||
The Journal SHALL persist a `synthetic` boolean on every route so automated / demo content can be distinguished from user-created content.
|
||||
|
||||
#### Scenario: User-created routes default to non-synthetic
|
||||
- **WHEN** a route is created through any user-facing flow (New Route, GPX import, Planner handoff)
|
||||
- **THEN** the route row is persisted with `synthetic = false`
|
||||
|
||||
#### Scenario: Bot inserts flag their rows as synthetic
|
||||
- **WHEN** the demo-activity-bot inserts a route
|
||||
- **THEN** the row is persisted with `synthetic = true`
|
||||
|
||||
#### Scenario: Synthetic flag is not user-editable
|
||||
- **WHEN** a route owner edits a route via any user-facing action
|
||||
- **THEN** the stored `synthetic` value is not changed by the edit
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
## 1. Schema
|
||||
|
||||
- [x] 1.1 Add `synthetic boolean NOT NULL DEFAULT false` to `journal.routes` in `packages/db/src/schema/journal.ts`
|
||||
- [x] 1.2 Add the same column to `journal.activities`
|
||||
- [x] 1.3 Verify existing CRUD helpers (`createRoute`, `updateRoute`, `createActivity`, `listRoutes`, `listActivities`) compile and ignore the new column for default callers
|
||||
|
||||
## 2. Bruno bootstrap
|
||||
|
||||
- [x] 2.1 Reserved username: `bruno`. Sentinel email: `bruno@<DOMAIN>`. Display name: `Bruno`. Bio: a short whimsical line (finalise exact wording during apply — e.g. "Professional park inspector. Currently accepting tennis balls.").
|
||||
- [x] 2.2 Add an `ensureDemoUser()` helper in `apps/journal/app/lib/demo-bot.server.ts` that inserts the `bruno` row only if missing (idempotent via `ON CONFLICT DO NOTHING` on `username`), with `terms_accepted_at` and `terms_version` populated.
|
||||
- [x] 2.3 Call `ensureDemoUser()` from the worker startup sequence when `DEMO_BOT_ENABLED === "true"`.
|
||||
- [x] 2.4 Confirm Bruno has no credentials row; the passkey and magic-link login paths both fail for his email (no mailbox exists for `bruno@<DOMAIN>`).
|
||||
|
||||
## 3. Region + generation helpers
|
||||
|
||||
- [x] 3.1 Add `loadRegion()` that reads `DEMO_BOT_REGION` JSON (with `bbox: [w,s,e,n]`) and falls back to the inner-Berlin default `[13.25, 52.45, 13.55, 52.60]`
|
||||
- [x] 3.2 Add `pickEndpoints(bbox)` — random start + end, 2–12 km crow distance apart (dog-walk scale)
|
||||
- [x] 3.3 Profile is fixed to `trekking` (Bruno doesn't ride bikes); no `pickProfile()` needed for v1
|
||||
- [x] 3.4 Add `templateName(startedAt)` + `templateDescription()` — EN + DE Bruno-voiced copy, mixing serious-audit flavour ("Grunewald north-loop patrol") and comic ("Bruno found three sticks today 🐕"); 10–15 entries in each pool, picked deterministically from the day + started-at to avoid same-day repeats
|
||||
|
||||
## 4. BRouter client for the bot
|
||||
|
||||
- [x] 4.1 Reuse the existing `computeRoute` helper where possible; otherwise add a minimal `brouter-client.server.ts` that POSTs to `process.env.BROUTER_URL` and returns parsed GPX + stats
|
||||
- [x] 4.2 On no-route / 5xx / timeout, return a typed "no route" error — the job handles it, does not throw out of the worker
|
||||
|
||||
## 5. Generation job
|
||||
|
||||
- [x] 5.1 Register a pg-boss recurring job `demo-bot:generate` to fire every 90 minutes, singleton mode
|
||||
- [x] 5.2 Handler: skip immediately if `DEMO_BOT_ENABLED !== "true"`
|
||||
- [x] 5.3 Decide-to-walk gate: compute local hour; if outside 07:00–21:00 → skip. Otherwise roll a Bernoulli with p=0.12 and skip if it doesn't fire. Expected output: ~2–3 walks/day
|
||||
- [x] 5.4 Hard cap: `SELECT count(*) FROM routes WHERE synthetic AND created_at > now() - interval '14 days'`; skip if >= 40
|
||||
- [x] 5.5 Pick region + endpoints (profile is fixed to `trekking`); call BRouter; on failure log and return
|
||||
- [x] 5.6 Insert route with `visibility='public'`, `synthetic=true`, owner = Bruno; insert linked activity with derived `started_at` = now, `duration` = distance ÷ ~4.5 km/h ± jitter, stats, and the same GPX
|
||||
|
||||
## 6. Initial backfill
|
||||
|
||||
- [x] 6.1 At the top of the generation handler, check if `count(synthetic routes) == 0`; if so, loop up to 5 times running the full generation body sequentially (each call independent — if one fails, keep going)
|
||||
- [x] 6.2 After the backfill path, fall through to the normal single-item generation
|
||||
|
||||
## 7. Retention job
|
||||
|
||||
- [x] 7.1 Register a pg-boss recurring job `demo-bot:prune` with a cron like `15 3 * * *` (daily at 03:15 UTC), singleton
|
||||
- [x] 7.2 Handler: skip if `DEMO_BOT_ENABLED !== "true"`
|
||||
- [x] 7.3 Read `DEMO_BOT_RETENTION_DAYS` with default 14
|
||||
- [x] 7.4 `DELETE FROM journal.activities WHERE synthetic AND created_at < now() - interval ?`; same for routes
|
||||
- [x] 7.5 Log the count removed for observability
|
||||
|
||||
## 8. Ops + env surface
|
||||
|
||||
- [x] 8.1 Document `DEMO_BOT_ENABLED`, `DEMO_BOT_RETENTION_DAYS`, `DEMO_BOT_REGION` in `infrastructure/.env.example` + pass-through in `docker-compose.yml` (commented; not enabled in any environment file)
|
||||
- [x] 8.2 Set `DEMO_BOT_ENABLED=true` only in prod (runtime flag — not set anywhere in-repo)
|
||||
- [x] 8.3 Expose counts via Prometheus gauges `demo_bot_synthetic_routes_total` and `demo_bot_synthetic_activities_total`; refreshed from the job handlers so scrapes see live values
|
||||
|
||||
## 9. UX tweaks
|
||||
|
||||
- [x] 9.1 "🐕 demo account" badge on `/users/bruno` so honest readers can tell at a glance; gate on `user.username === 'bruno'`
|
||||
- [x] 9.2 i18n: `demo.badge` key in EN + DE (e.g. "Demo account" / "Demo-Konto")
|
||||
|
||||
## 10. Tests
|
||||
|
||||
- [x] 10.1 Unit: `ensureDemoUser` is idempotent; second call is a no-op and does not duplicate (in `demo-bot.integration.test.ts`, runs under `DEMO_BOT_INTEGRATION=1`)
|
||||
- [x] 10.2 Unit: `pickEndpoints` stays within bbox; distance bands match profile
|
||||
- [x] 10.3 Unit: `templateName` / `templateDescription` return non-empty strings in both locales
|
||||
- [x] 10.4 Integration (tests DB): the generation handler inserts one route + one activity with `synthetic=true, visibility='public'` when enabled, and inserts nothing when disabled
|
||||
- [x] 10.5 Integration: the prune handler deletes only `synthetic=true` rows past the window; real rows are untouched
|
||||
- [x] 10.6 E2E: `/users/bruno` renders the demo badge and lists public synthetic content to anonymous visitors (covers the user-visible surface)
|
||||
|
||||
## 11. Rollout
|
||||
|
||||
- [x] 11.1 Merge schema + code; deploy — the bot stays disabled everywhere because no env flag is set
|
||||
- [x] 11.2 Flip `DEMO_BOT_ENABLED=true` on prod via the SOPS env file + redeploy
|
||||
- [x] 11.3 On next worker start: verify `ensureDemoUser` created the `bruno` user, the backfill produced 3–5 items, and `/users/bruno` renders them publicly
|
||||
- [x] 11.4 After 24 h: verify cadence looks right and the prune job ran without deleting anything yet
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
schema: spec-driven
|
||||
created: 2026-04-13
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
## Context
|
||||
|
||||
Background work in trails.cool currently has no execution framework. The planner has an `expireSessions` function that is never called. The Komoot import feature (#223) implements ad-hoc background processing with no retries or failure visibility. Both apps already connect to PostgreSQL, making a Postgres-backed job queue the natural choice — no Redis or additional infrastructure needed.
|
||||
|
||||
pg-boss is a mature Node.js job queue that stores jobs as PostgreSQL rows, supports cron scheduling, automatic retries with exponential backoff, and exposes job state through queryable tables. Since Grafana already has a PostgreSQL datasource, observability comes for free.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Provide a durable job execution framework for both apps using the existing PostgreSQL database
|
||||
- Wire up planner session expiry as the first scheduled job
|
||||
- Make job queue health visible in Grafana
|
||||
- Establish patterns for adding future jobs (Komoot import, ActivityPub federation, email sending)
|
||||
|
||||
**Non-Goals:**
|
||||
- Refactoring the Komoot import to use pg-boss (separate change, after #223 merges)
|
||||
- Multi-worker scaling or horizontal partitioning — single worker per app is sufficient at current scale
|
||||
- Custom job queue UI — Grafana dashboards are sufficient
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. pg-boss over Graphile Worker
|
||||
**Choice**: pg-boss
|
||||
**Rationale**: Built-in cron scheduling, richer job metadata (completion timestamps, retry counts, output storage), and dead letter queue. Graphile Worker is lighter but lacks cron and has less queryable state for Grafana dashboards.
|
||||
**Alternative considered**: Graphile Worker — faster raw throughput, but we need scheduling and observability more than throughput.
|
||||
|
||||
### 2. Shared package `@trails-cool/jobs`
|
||||
**Choice**: New shared package at `packages/jobs/` that wraps pg-boss initialization and exports job registration helpers.
|
||||
**Rationale**: Both planner and journal need background jobs. A shared package avoids duplicating pg-boss setup, ensures consistent configuration (retry policies, monitoring interval), and provides a single place to register job types.
|
||||
**Alternative considered**: Inline setup in each app's `server.ts` — simpler initially but leads to divergent configuration.
|
||||
|
||||
### 3. Worker lifecycle in server process
|
||||
**Choice**: Start the pg-boss worker inside each app's `server.ts` after the HTTP server is listening.
|
||||
**Rationale**: Keeps deployment simple — no separate worker process or container. pg-boss workers are lightweight (polling loop). The planner and journal already run as long-lived Node.js processes.
|
||||
**Alternative considered**: Separate worker container — better isolation but doubles container count and adds deployment complexity for minimal benefit at current scale.
|
||||
|
||||
### 4. Schema isolation
|
||||
**Choice**: Let pg-boss use its default `pgboss` schema in the `trails` database.
|
||||
**Rationale**: pg-boss auto-creates and migrates its own schema. Keeping it separate from `planner` and `journal` schemas avoids any collision. Grant `grafana_reader` SELECT access for dashboards.
|
||||
|
||||
### 5. Session expiry job
|
||||
**Choice**: Cron job running hourly that calls the existing `expireSessions(7)` function.
|
||||
**Rationale**: 7-day TTL with hourly cleanup is generous enough that no active session gets reaped, and frequent enough that stale sessions don't accumulate. The existing function already handles cleanup correctly (deletes DB rows + removes Yjs docs from memory).
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **pg-boss schema migrations on upgrade**: pg-boss manages its own schema and runs migrations on startup. → Mitigation: Pin pg-boss version, test upgrades in dev before deploying.
|
||||
- **Worker blocks event loop**: A misbehaving job handler could block the HTTP server. → Mitigation: Jobs should be I/O-bound (DB queries), not CPU-bound. Add a timeout to job handlers.
|
||||
- **Single worker per app**: No parallelism for heavy workloads. → Mitigation: Sufficient for current scale. Can add concurrency options per queue or separate workers later if needed.
|
||||
- **Database load from polling**: pg-boss polls for jobs. → Mitigation: Default polling interval (2s) is fine for our scale. The queries are indexed and lightweight.
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
## Why
|
||||
|
||||
Background work (session expiry, Komoot activity imports) currently has no durable execution mechanism. Session cleanup exists as dead code (`expireSessions`) that is never called, and the Komoot import in #223 rolls its own ad-hoc background processing without retries or failure handling. Adding pg-boss gives us a PostgreSQL-backed job queue with scheduling, retries, and observability through our existing Grafana stack — no new infrastructure required.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add `pg-boss` as a dependency to both the Planner and Journal apps
|
||||
- Create a shared background worker setup in `@trails-cool/db` (or a new `@trails-cool/jobs` package) that initializes pg-boss with the existing `DATABASE_URL`
|
||||
- Wire up **planner session expiry** as the first recurring job (hourly cron, 7-day TTL)
|
||||
- Wire up **Komoot import processing** as a durable job with retries (replaces ad-hoc background processing in #223)
|
||||
- Grant `grafana_reader` SELECT on the `pgboss` schema for dashboard visibility
|
||||
- Add a Grafana dashboard panel for job queue health (queue depth, failed jobs, processing duration)
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `background-jobs`: pg-boss job queue setup, worker lifecycle, job registration, cron scheduling, retry policies, and Grafana observability
|
||||
|
||||
### Modified Capabilities
|
||||
- `planner-session`: Session expiry is now handled by a scheduled pg-boss job instead of being uncalled dead code
|
||||
- `infrastructure`: PostgreSQL init script grants `grafana_reader` access to the `pgboss` schema; Grafana gets a job queue health panel
|
||||
|
||||
## Impact
|
||||
|
||||
- **Dependencies**: Adds `pg-boss` npm package
|
||||
- **Database**: pg-boss auto-creates its schema (`pgboss`) in PostgreSQL on first run
|
||||
- **Apps**: Both planner and journal server entry points start a pg-boss worker
|
||||
- **Monitoring**: New Grafana panel on Service Health dashboard for job queue metrics
|
||||
- **Komoot import (#223)**: Will be refactored to use pg-boss instead of ad-hoc background processing
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Job queue initialization
|
||||
The `@trails-cool/jobs` package SHALL initialize a pg-boss instance using the app's `DATABASE_URL` and export a function to start the worker.
|
||||
|
||||
#### Scenario: Worker starts with server
|
||||
- **WHEN** the planner or journal server starts
|
||||
- **THEN** pg-boss connects to PostgreSQL, creates/migrates the `pgboss` schema if needed, and begins polling for jobs
|
||||
|
||||
#### Scenario: Worker stops on shutdown
|
||||
- **WHEN** the server process receives SIGTERM
|
||||
- **THEN** pg-boss completes any in-progress jobs and stops gracefully before the process exits
|
||||
|
||||
### Requirement: Cron job scheduling
|
||||
The system SHALL support registering recurring jobs with cron expressions.
|
||||
|
||||
#### Scenario: Register a cron job
|
||||
- **WHEN** a job is registered with a cron expression (e.g., `0 * * * *` for hourly)
|
||||
- **THEN** pg-boss creates a schedule that enqueues the job at the specified interval
|
||||
|
||||
#### Scenario: Cron job survives restart
|
||||
- **WHEN** the server process restarts
|
||||
- **THEN** existing cron schedules persist and continue firing without re-registration conflicts
|
||||
|
||||
### Requirement: Job retry policy
|
||||
Jobs SHALL support configurable retry policies with exponential backoff.
|
||||
|
||||
#### Scenario: Transient failure retry
|
||||
- **WHEN** a job handler throws an error
|
||||
- **THEN** pg-boss retries the job up to the configured retry limit with exponential backoff
|
||||
|
||||
#### Scenario: Permanent failure
|
||||
- **WHEN** a job exhausts all retries
|
||||
- **THEN** the job moves to the `failed` state and remains queryable for debugging
|
||||
|
||||
### Requirement: Job handler timeout
|
||||
Each job handler SHALL have a configurable execution timeout.
|
||||
|
||||
#### Scenario: Job exceeds timeout
|
||||
- **WHEN** a job handler does not complete within its timeout
|
||||
- **THEN** pg-boss marks the job as failed with a timeout error
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Grafana database access
|
||||
The `grafana_reader` PostgreSQL role SHALL have SELECT access to the `pgboss` schema for job queue observability.
|
||||
|
||||
#### Scenario: Grant access on deploy
|
||||
- **WHEN** the infrastructure deploy runs
|
||||
- **THEN** `grafana_reader` is granted `USAGE` on the `pgboss` schema and `SELECT` on all tables in it
|
||||
|
||||
### Requirement: Monitoring stack
|
||||
The Grafana Service Health dashboard SHALL include a job queue health panel.
|
||||
|
||||
#### Scenario: Job queue panel displays metrics
|
||||
- **WHEN** a user views the Service Health dashboard
|
||||
- **THEN** they see a panel showing job queue depth, completed jobs per hour, and failed jobs
|
||||
- **AND** failed jobs are highlighted for investigation
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Session expiry
|
||||
Open sessions with no activity for 7 days SHALL be automatically deleted by a scheduled background job.
|
||||
|
||||
#### Scenario: Stale session cleanup
|
||||
- **WHEN** the hourly `expire-sessions` cron job runs
|
||||
- **THEN** all sessions with `last_activity` older than 7 days are deleted from the database
|
||||
- **AND** their Yjs documents are removed from memory
|
||||
|
||||
#### Scenario: Active session preserved
|
||||
- **WHEN** the `expire-sessions` job runs
|
||||
- **THEN** sessions with `last_activity` within the last 7 days are NOT deleted
|
||||
|
||||
#### Scenario: Cleanup is observable
|
||||
- **WHEN** the `expire-sessions` job completes
|
||||
- **THEN** the job output includes the count of expired sessions
|
||||
- **AND** the result is visible in the Grafana job queue dashboard
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
## 1. Package Setup
|
||||
|
||||
- [x] 1.1 Create `packages/jobs/` package with `package.json`, `tsconfig.json`, and `pg-boss` dependency
|
||||
- [x] 1.2 Add `@trails-cool/jobs` to planner and journal app dependencies in `pnpm-workspace.yaml`
|
||||
- [x] 1.3 Run `pnpm install` and verify workspace resolution
|
||||
|
||||
## 2. Core Job Queue Module
|
||||
|
||||
- [x] 2.1 Create `packages/jobs/src/boss.ts` — initialize pg-boss with `DATABASE_URL`, export `createBoss()` factory
|
||||
- [x] 2.2 Create `packages/jobs/src/worker.ts` — export `startWorker(boss, jobs)` that registers job handlers and starts processing
|
||||
- [x] 2.3 Create `packages/jobs/src/types.ts` — export `JobDefinition` type (name, handler, cron?, retryLimit?, expireInSeconds?)
|
||||
- [x] 2.4 Add graceful shutdown: listen for SIGTERM, call `boss.stop()` to complete in-progress jobs before exit
|
||||
- [x] 2.5 Export public API from `packages/jobs/src/index.ts`
|
||||
|
||||
## 3. Planner Session Expiry Job
|
||||
|
||||
- [x] 3.1 Create `apps/planner/app/jobs/expire-sessions.ts` — job handler that calls `expireSessions(7)` and returns the count
|
||||
- [x] 3.2 Register the job in planner's `server.ts` with cron `0 * * * *` (hourly), retryLimit 2, expireInSeconds 60
|
||||
- [x] 3.3 Verify job appears in `pgboss.schedule` table after planner starts (manual verification after dev stack is running)
|
||||
- [x] 3.4 Write a test for the expire-sessions handler
|
||||
|
||||
## 4. Journal Worker Setup
|
||||
|
||||
- [x] 4.1 Add pg-boss worker startup to journal's `server.ts` (no jobs yet — placeholder for Komoot import and future federation jobs)
|
||||
- [x] 4.2 Add graceful shutdown handling (handled by startWorker via SIGTERM/SIGINT listeners)
|
||||
|
||||
## 5. Infrastructure & Observability
|
||||
|
||||
- [x] 5.1 Add `GRANT USAGE ON SCHEMA pgboss TO grafana_reader` and `GRANT SELECT ON ALL TABLES IN SCHEMA pgboss TO grafana_reader` to `infrastructure/postgres/init-grafana-user.sql`
|
||||
- [x] 5.2 Add a "Job Queue Health" panel to the Service Health Grafana dashboard — queue depth (`SELECT state, count(*) FROM pgboss.job GROUP BY state`), failed jobs, and completed jobs/hour
|
||||
- [x] 5.3 Add a Grafana alert for failed background jobs (`SELECT count(*) FROM pgboss.job WHERE state = 'failed' AND completedon > now() - interval '1 hour'`)
|
||||
|
||||
## 6. Testing & Verification
|
||||
|
||||
- [x] 6.1 Run `pnpm typecheck` — all packages pass
|
||||
- [x] 6.2 Run `pnpm test` — new and existing tests pass (63 planner tests, including 2 new expire-sessions tests)
|
||||
- [x] 6.3 Start dev stack with `pnpm dev:full`, verify pg-boss tables are created and expire-sessions schedule is registered (manual verification)
|
||||
Loading…
Add table
Add a link
Reference in a new issue