poi-index: run osmium via Docker (host has no osmium + no sudo)

The BRouter host's trails user is non-root without sudo, so osmium-tool can't
be apt-installed. Ship osmium.Dockerfile (built on first use) and run osmium in
a container with the work dir bind-mounted (native I/O, negligible overhead on
Linux). python3/curl/gzip/sha256sum remain on the host. Correct the README
disk figure to the /home free space (~595 GB).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

infrastructure/brouter-host/poi-extract/README.md

@@ -48,8 +48,12 @@ artifact is unreachable from the public internet.
 
 ## Prerequisites
 
-Install on the host: `osmium-tool`, `python3`, `curl`, `gzip`, `coreutils`
+- **Docker** — osmium runs in a container (`osmium.Dockerfile`, built on first
-(`sha256sum`). On Debian/Ubuntu: `apt install osmium-tool python3`.
+  use as `trails-osmium:local`). The BRouter host's `trails` user is non-root
+  with no sudo, so osmium can't be apt-installed; docker-group rights cover it.
+  On Linux this adds no meaningful overhead — big files are bind-mounted, so I/O
+  is native.
+- **On the host** (already present): `python3`, `curl`, `gzip`, `sha256sum`.
 
 ## Install the timer
 
@@ -75,7 +79,8 @@ poi-extract.service` then `journalctl --user -u poi-extract.service -f`.
   (~4 GB) for a smaller instance.
 - **Disk**: the planet PBF + filtered PBF live transiently in `./work` and are
   removed on exit (even on failure) via a trap. Peak transient usage ≈ size of
-  the PBF. The published artifact is a few hundred MB gzipped.
+  the PBF. The `trails` user's `/home` has ~595 GB free (as of 2026-07) — ample
+  for an ~80 GB planet run. The published artifact is a few hundred MB gzipped.
 - **CPU/time**: a full planet filter is IO-bound and can take a few hours; the
   unit runs at `Nice=10` / idle IO priority so it doesn't disturb BRouter.
 

infrastructure/brouter-host/poi-extract/osmium.Dockerfile

@@ -0,0 +1,15 @@
+# Minimal osmium-tool image for the POI extract pipeline. The BRouter host runs
+# a non-root `trails` user with no sudo, so osmium can't be apt-installed on the
+# host — poi-extract.sh runs it in this container instead (docker-group rights
+# are available). Big files are bind-mounted, so I/O is native; the container
+# adds no meaningful CPU/throughput overhead on Linux.
+#
+# Built on first use by poi-extract.sh (tag: trails-osmium:local). Self-hosters
+# get identical behavior without touching system packages.
+FROM debian:bookworm-slim
+
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends osmium-tool \
+    && rm -rf /var/lib/apt/lists/*
+
+ENTRYPOINT ["osmium"]

infrastructure/brouter-host/poi-extract/poi-extract.sh

@@ -1,16 +1,19 @@
 #!/bin/bash
 # POI extract pipeline — runs on the dedicated BRouter host under the `trails`
-# user (the box with 1.8 TB of disk). Downloads an OSM PBF, filters it to the
+# user. Downloads an OSM PBF, filters it to the planner's POI category selectors
-# planner's POI category selectors with osmium, reduces every element to a
+# with osmium, reduces every element to a centroid, and publishes a compact
-# centroid, and publishes a compact NDJSON artifact + manifest that the
+# NDJSON artifact + manifest that the flagship import job pulls over the vSwitch.
-# flagship import job pulls over the vSwitch.
 #
-# Nothing here is planner/journal code — it's plain osmium + python so the host
+# Nothing here is planner/journal code — the host needs no Node runtime. The tag
-# needs no Node runtime. The tag filter comes from `osmium-filters.txt`, which
+# filter comes from `osmium-filters.txt`, generated from `@trails-cool/map-core`
-# is generated from `@trails-cool/map-core`'s POI selectors (single source of
+# POI selectors (single source of truth; kept in sync by
-# truth; kept in sync by osmium-filters.sync.test.ts).
+# osmium-filters.sync.test.ts).
 #
-# Prerequisites on the host: osmium-tool, python3, curl, gzip, sha256sum.
+# osmium runs in a container: the `trails` user is non-root with no sudo, so
+# osmium-tool can't be apt-installed on the host, but docker-group rights are
+# available. Big files are bind-mounted into the container, so I/O is native and
+# the container adds no meaningful CPU overhead on Linux. The image is built on
+# first use from osmium.Dockerfile. python3/curl/gzip/sha256sum run on the host.
 #
 # Usage:
 #   ./poi-extract.sh
@@ -24,6 +27,8 @@
 #   POI_PUBLISH_DIR  Where the artifact + manifest are written for Caddy to
 #                 serve. Default: <script dir>/publish (mounted into the Caddy
 #                 sidecar at /srv/poi — see docker-compose.yml + Caddyfile).
+#   OSMIUM_IMAGE  osmium container image. Default: trails-osmium:local (built
+#                 on first use from osmium.Dockerfile).
 set -euo pipefail
 
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -33,6 +38,7 @@ TRANSFORMER="$SCRIPT_DIR/to-ndjson.py"
 POI_PBF_URL="${POI_PBF_URL:-https://planet.osm.org/pbf/planet-latest.osm.pbf}"
 POI_WORK_DIR="${POI_WORK_DIR:-$SCRIPT_DIR/work}"
 POI_PUBLISH_DIR="${POI_PUBLISH_DIR:-$SCRIPT_DIR/publish}"
+OSMIUM_IMAGE="${OSMIUM_IMAGE:-trails-osmium:local}"
 
 PLANET_PBF="$POI_WORK_DIR/planet.osm.pbf"
 FILTERED_PBF="$POI_WORK_DIR/filtered.osm.pbf"
@@ -41,6 +47,16 @@ MANIFEST="$POI_PUBLISH_DIR/manifest.json"
 
 mkdir -p "$POI_WORK_DIR" "$POI_PUBLISH_DIR"
 
+# Run osmium inside the container with the work dir bind-mounted at /data, as the
+# host uid/gid so output files are owned by `trails`, not root. Big-file I/O goes
+# through the bind mount (native), never the container's overlay layer.
+osmium_run() {
+  docker run --rm \
+    -u "$(id -u):$(id -g)" \
+    -v "$POI_WORK_DIR:/data" \
+    "$OSMIUM_IMAGE" "$@"
+}
+
 # Working files can be huge (planet PBF ~80 GB); always clean them up, even on
 # failure, so a broken run can't fill the shared host's disk.
 cleanup() { rm -f "$PLANET_PBF" "$FILTERED_PBF"; }
@@ -56,26 +72,35 @@ fi
 mapfile -t FILTERS < "$FILTERS_FILE"
 log "tag filters: ${FILTERS[*]}"
 
-# --- 2. Download the PBF -----------------------------------------------------
+# --- 2. Ensure the osmium image exists (build once from osmium.Dockerfile) ---
+if ! docker image inspect "$OSMIUM_IMAGE" >/dev/null 2>&1; then
+  log "building osmium image $OSMIUM_IMAGE"
+  docker build -t "$OSMIUM_IMAGE" -f "$SCRIPT_DIR/osmium.Dockerfile" "$SCRIPT_DIR"
+fi
+
+# --- 3. Download the PBF -----------------------------------------------------
 log "downloading $POI_PBF_URL"
 curl --fail --location --show-error --silent --output "$PLANET_PBF" "$POI_PBF_URL"
 log "downloaded $(du -h "$PLANET_PBF" | cut -f1)"
 
-# --- 3. Filter to our categories --------------------------------------------
+# --- 4. Filter to our categories --------------------------------------------
+# Paths are inside the container's /data bind mount. tags-filter keeps the nodes
+# referenced by matched ways/relations (default), so export can build geometry.
 log "filtering with osmium tags-filter"
-osmium tags-filter --overwrite --output "$FILTERED_PBF" "$PLANET_PBF" "${FILTERS[@]}"
+osmium_run tags-filter --overwrite --output /data/filtered.osm.pbf /data/planet.osm.pbf "${FILTERS[@]}"
 log "filtered to $(du -h "$FILTERED_PBF" | cut -f1)"
 
-# --- 4. Export to centroid NDJSON -------------------------------------------
+# --- 5. Export to centroid NDJSON -------------------------------------------
-# osmium export emits one GeoJSON feature per line; to-ndjson.py reduces each to
+# osmium export emits one GeoJSON feature per line to stdout (forwarded out of
-# a bbox centre + compact record and reports the row count on stderr.
+# the container); to-ndjson.py reduces each to a bbox centre + compact record on
+# the host and reports the row count on stderr.
 log "exporting + transforming to NDJSON"
 rowcount_file="$(mktemp)"
-osmium export "$FILTERED_PBF" \
+osmium_run export /data/filtered.osm.pbf \
   --output-format=geojsonseq \
   --add-unique-id=type_id \
   --geometry-types=point,linestring,polygon \
-  --overwrite --output - \
+  --output - \
   | python3 "$TRANSFORMER" 2>"$rowcount_file" \
   | gzip -c > "$ARTIFACT"
 ROW_COUNT="$(cat "$rowcount_file")"
@@ -87,7 +112,7 @@ if [ "$ROW_COUNT" -eq 0 ]; then
   exit 1
 fi
 
-# --- 5. Manifest (checksum + timestamp + row count) -------------------------
+# --- 6. Manifest (checksum + timestamp + row count) -------------------------
 SHA256="$(sha256sum "$ARTIFACT" | cut -d' ' -f1)"
 GENERATED_AT="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
 cat > "$MANIFEST" <<EOF