diff --git a/infrastructure/brouter-host/poi-extract/README.md b/infrastructure/brouter-host/poi-extract/README.md index 8871980..4a06f72 100644 --- a/infrastructure/brouter-host/poi-extract/README.md +++ b/infrastructure/brouter-host/poi-extract/README.md @@ -48,8 +48,12 @@ artifact is unreachable from the public internet. ## Prerequisites -Install on the host: `osmium-tool`, `python3`, `curl`, `gzip`, `coreutils` -(`sha256sum`). On Debian/Ubuntu: `apt install osmium-tool python3`. +- **Docker** — osmium runs in a container (`osmium.Dockerfile`, built on first + use as `trails-osmium:local`). The BRouter host's `trails` user is non-root + with no sudo, so osmium can't be apt-installed; docker-group rights cover it. + On Linux this adds no meaningful overhead — big files are bind-mounted, so I/O + is native. +- **On the host** (already present): `python3`, `curl`, `gzip`, `sha256sum`. ## Install the timer @@ -75,7 +79,8 @@ poi-extract.service` then `journalctl --user -u poi-extract.service -f`. (~4 GB) for a smaller instance. - **Disk**: the planet PBF + filtered PBF live transiently in `./work` and are removed on exit (even on failure) via a trap. Peak transient usage ≈ size of - the PBF. The published artifact is a few hundred MB gzipped. + the PBF. The `trails` user's `/home` has ~595 GB free (as of 2026-07) — ample + for an ~80 GB planet run. The published artifact is a few hundred MB gzipped. - **CPU/time**: a full planet filter is IO-bound and can take a few hours; the unit runs at `Nice=10` / idle IO priority so it doesn't disturb BRouter. diff --git a/infrastructure/brouter-host/poi-extract/osmium.Dockerfile b/infrastructure/brouter-host/poi-extract/osmium.Dockerfile new file mode 100644 index 0000000..01e3a6f --- /dev/null +++ b/infrastructure/brouter-host/poi-extract/osmium.Dockerfile @@ -0,0 +1,15 @@ +# Minimal osmium-tool image for the POI extract pipeline. The BRouter host runs +# a non-root `trails` user with no sudo, so osmium can't be apt-installed on the +# host — poi-extract.sh runs it in this container instead (docker-group rights +# are available). Big files are bind-mounted, so I/O is native; the container +# adds no meaningful CPU/throughput overhead on Linux. +# +# Built on first use by poi-extract.sh (tag: trails-osmium:local). Self-hosters +# get identical behavior without touching system packages. +FROM debian:bookworm-slim + +RUN apt-get update \ + && apt-get install -y --no-install-recommends osmium-tool \ + && rm -rf /var/lib/apt/lists/* + +ENTRYPOINT ["osmium"] diff --git a/infrastructure/brouter-host/poi-extract/poi-extract.sh b/infrastructure/brouter-host/poi-extract/poi-extract.sh index 5731b0c..d2f5de3 100755 --- a/infrastructure/brouter-host/poi-extract/poi-extract.sh +++ b/infrastructure/brouter-host/poi-extract/poi-extract.sh @@ -1,16 +1,19 @@ #!/bin/bash # POI extract pipeline — runs on the dedicated BRouter host under the `trails` -# user (the box with 1.8 TB of disk). Downloads an OSM PBF, filters it to the -# planner's POI category selectors with osmium, reduces every element to a -# centroid, and publishes a compact NDJSON artifact + manifest that the -# flagship import job pulls over the vSwitch. +# user. Downloads an OSM PBF, filters it to the planner's POI category selectors +# with osmium, reduces every element to a centroid, and publishes a compact +# NDJSON artifact + manifest that the flagship import job pulls over the vSwitch. # -# Nothing here is planner/journal code — it's plain osmium + python so the host -# needs no Node runtime. The tag filter comes from `osmium-filters.txt`, which -# is generated from `@trails-cool/map-core`'s POI selectors (single source of -# truth; kept in sync by osmium-filters.sync.test.ts). +# Nothing here is planner/journal code — the host needs no Node runtime. The tag +# filter comes from `osmium-filters.txt`, generated from `@trails-cool/map-core` +# POI selectors (single source of truth; kept in sync by +# osmium-filters.sync.test.ts). # -# Prerequisites on the host: osmium-tool, python3, curl, gzip, sha256sum. +# osmium runs in a container: the `trails` user is non-root with no sudo, so +# osmium-tool can't be apt-installed on the host, but docker-group rights are +# available. Big files are bind-mounted into the container, so I/O is native and +# the container adds no meaningful CPU overhead on Linux. The image is built on +# first use from osmium.Dockerfile. python3/curl/gzip/sha256sum run on the host. # # Usage: # ./poi-extract.sh @@ -24,6 +27,8 @@ # POI_PUBLISH_DIR Where the artifact + manifest are written for Caddy to # serve. Default: