trails-cool/trails
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
trails/infrastructure/scripts
History
Ullrich Schäfer a6f8595c4b
poi-index: stream selectors SQL into psql stdin (fix \i container path) 
poi-import.sh ran the classifier selectors via psql \i, but psql executes
inside the postgres container (docker compose exec), so it couldn't find the
host path /opt/trails-cool/scripts/poi-selectors.sql. Concatenate BEGIN + the
selectors file contents + the build statements on the host and pipe them into
psql stdin instead, keeping the ON COMMIT DROP temp table in one transaction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 00:27:48 +02:00
..
backup-postgres.sh Add infrastructure: Terraform, Docker Compose, BRouter, CD pipeline (#2) 2026-03-22 13:32:01 +01:00
gen-poi-selectors-sql.ts poi-index: extract pipeline (BRouter host) + import job (flagship) + e2e 2026-07-12 22:48:32 +02:00
poi-import.service poi-index: extract pipeline (BRouter host) + import job (flagship) + e2e 2026-07-12 22:48:32 +02:00
poi-import.sh poi-index: stream selectors SQL into psql stdin (fix \i container path) 2026-07-13 00:27:48 +02:00
poi-import.timer poi-index: extract pipeline (BRouter host) + import job (flagship) + e2e 2026-07-12 22:48:32 +02:00
poi-selectors.sql poi-index: extract pipeline (BRouter host) + import job (flagship) + e2e 2026-07-12 22:48:32 +02:00
README.md poi-index: extract pipeline (BRouter host) + import job (flagship) + e2e 2026-07-12 22:48:32 +02:00

README.md

Flagship operational scripts

Deployed to /opt/trails-cool/scripts/ by cd-infra.yml.

backup-postgres.sh

Daily pg_dump + retention sweep. Run via cron (see the header).

POI index import (poi-import.sh)

The "load where the database is" half of the poi-index change. Pulls the artifact the BRouter host published, classifies each element into planner.pois category rows, and swaps it in atomically.

  • poi-import.sh — fetch (vSwitch) → checksum → load → classify → guarded atomic rename swap. --bootstrap skips the 70% row-count guard for the first import (when the live table is legitimately empty).
  • poi-selectors.sql — generated from @trails-cool/map-core selectors; the classifier JOINs on it. Regenerate with npx tsx infrastructure/scripts/gen-poi-selectors-sql.ts (CI sync test: packages/map-core/src/poi-selectors-sql.sync.test.ts).
  • poi-import.service / .timer — monthly, offset a day after the extract.

First-time setup on the flagship

# 1. Create the empty live table from the Drizzle schema (idempotent).
#    (Run from a checkout with DATABASE_URL pointed at production, or via the
#    normal db:push deploy path.)
pnpm db:push

# 2. Install the timer.
cp /opt/trails-cool/scripts/poi-import.{service,timer} /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now poi-import.timer

# 3. Bootstrap the first import (guard disabled — live table is empty).
BROUTER_AUTH_TOKEN=$(grep BROUTER_AUTH_TOKEN /opt/trails-cool/.env | cut -d= -f2-) \
  /opt/trails-cool/scripts/poi-import.sh --bootstrap

Prometheus picks up import outcome from node_exporter's textfile collector (poi_import_last_status, …_last_success_timestamp_seconds, …_last_rows) when NODE_EXPORTER_TEXTFILE_DIR is set (the unit sets it). Index size and age are additionally exposed by the planner at /metrics (poi_index_rows{category}, poi_index_age_seconds).