refactor(infra): mount config dirs not single files; reload on deploy

Single-file bind mounts (./foo.yml:/etc/foo.yml) pin to the host file's
inode at container-create time. The CD pipeline scp's a replacement file
(new inode), so the running container keeps reading the OLD inode —
`docker compose up -d` won't recreate on a content-only change, and
neither restart/SIGHUP/`caddy reload` re-reads the new file. Net effect:
config-only infra PRs deployed "successfully" but never took effect
(confirmed with PR #500's prometheus.yml; the WAL retention work only
applied because #498 also changed docker-compose.yml, forcing a
recreate). Caddyfile changes had the same latent gap.

Switch the four single-file config mounts to DIRECTORY mounts, which
resolve children live so a reload/restart picks up the new file:

- prometheus, loki, promtail: mount ./<svc> dir (loki/promtail
  --config.file paths updated to the real filenames).
- caddy: move Caddyfile into caddy/ and mount the dir; ./sites overlays
  /etc/caddy/sites (caddy/sites/.gitkeep keeps the mountpoint). Container
  path /etc/caddy/Caddyfile is unchanged, so every `caddy reload --config
  /etc/caddy/Caddyfile` ref still works. scp source paths in cd-infra and
  cd-apps updated to ship the caddy/ dir.

cd-infra now applies config-only changes after `up -d`: SIGHUP prometheus
(zero downtime), restart loki+promtail (no SIGHUP reload), caddy reload
(graceful). Mirrored prometheus/loki mounts in docker-compose.dev.yml.

Validated: docker compose config (both files), caddy validate from the
new path, read-only-parent + sites-overlay mount mechanics, workflow YAML.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-06-09 13:10:35 +02:00
parent 8e41b09ac2
commit e60c9d7057
6 changed files with 48 additions and 16 deletions

View file

@ -14,10 +14,17 @@ services:
environment:
DOMAIN: ${DOMAIN:-trails.cool}
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
# Per-PR staging snippets dropped in by cd-staging.yml. Bind mount
# auto-creates the host directory if it doesn't exist, so a fresh
# server doesn't need pre-provisioning.
# Mount the caddy/ DIRECTORY, not the Caddyfile directly. A single-
# file bind mount pins to the host file's inode at create time, so a
# deploy that replaces the file (scp) leaves the container reading the
# old inode — `caddy reload` then reloads stale config. A directory
# mount resolves children live, so reload picks up the new Caddyfile.
# The container path /etc/caddy/Caddyfile is unchanged.
- ./caddy:/etc/caddy:ro
# Per-PR staging snippets dropped in by cd-staging.yml, overlaid onto
# the read-only caddy dir above (caddy/sites/.gitkeep keeps the
# mountpoint present). Bind mount auto-creates the host directory if
# it doesn't exist, so a fresh server doesn't need pre-provisioning.
- ./sites:/etc/caddy/sites:ro
- caddy_data:/data
- caddy_config:/config
@ -187,8 +194,10 @@ services:
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /var/lib/docker/containers:/var/lib/docker/containers:ro
- ./promtail/promtail-config.yml:/etc/promtail/config.yml:ro
command: ["-config.file=/etc/promtail/config.yml"]
# Directory mount (not the file) so a deploy's replaced config is seen
# live and a SIGHUP reload picks it up — see the caddy note above.
- ./promtail:/etc/promtail:ro
command: ["-config.file=/etc/promtail/promtail-config.yml"]
depends_on:
- loki
@ -196,7 +205,10 @@ services:
image: prom/prometheus:latest
restart: unless-stopped
volumes:
- ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
# Directory mount (not the file) so config changes are seen live and a
# SIGHUP reload applies them without recreating the container — see the
# caddy note above. --config.file path below is unchanged.
- ./prometheus:/etc/prometheus:ro
- prometheus_data:/prometheus
command:
- "--config.file=/etc/prometheus/prometheus.yml"
@ -207,9 +219,12 @@ services:
image: grafana/loki:latest
restart: unless-stopped
volumes:
- ./loki/loki-config.yml:/etc/loki/local-config.yaml:ro
# Directory mount (not the file) so a replaced config is seen live —
# see the caddy note above. Loki only reloads its main config on
# restart (no SIGHUP), so cd-infra restarts loki rather than HUPs it.
- ./loki:/etc/loki:ro
- loki_data:/loki
command: ["-config.file=/etc/loki/local-config.yaml"]
command: ["-config.file=/etc/loki/loki-config.yml"]
# Publish only on the vSwitch IP so Promtail running on the
# dedicated BRouter host can push logs in. Hetzner Cloud firewall
# still blocks 3100 from the public internet. Internal services on