Provision Grafana Pushover contact point

The notification policy and contact points live in
`infrastructure/grafana/provisioning/alerting/alerts.yml`, so
UI-created contact points can't be attached to provisioned alerts.
Provision the Pushover integration instead.

Rather than add a second contact point and do the fan-out in the
notification tree (Grafana's tree only fires one receiver per match
without duplication workarounds), put both email + pushover under a
single `default` contact point. Every alert now fans out to both.

Secrets go to SOPS `secrets.infra.env`. The Grafana container reads
`PUSHOVER_API_TOKEN` and `PUSHOVER_USER_KEY_ULLRICH` from env; the
provisioning YAML references them via `$VAR` and Grafana substitutes
at startup, so no secret lands in git.

Priority 1 on firing (bypass quiet hours), 0 on resolve. User can
tweak in the YAML later.

Post-deploy: delete the UI-created "Pushover Ullrich" contact point
in Grafana to avoid a duplicate. It's unused (no policy references it)
but shows up in the contact-point list.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-04-24 19:07:04 +02:00
parent 79c4e1e457
commit 98341e3603
3 changed files with 26 additions and 5 deletions

View file

@ -201,6 +201,11 @@ services:
GF_SMTP_FROM_ADDRESS: noreply@trails.cool
GF_SMTP_FROM_NAME: trails.cool Grafana
GRAFANA_DB_PASSWORD: ${GRAFANA_DB_PASSWORD:-}
# Pushover app + user keys consumed by alerts.yml provisioning.
# App token identifies the trails.cool Pushover application; per-user
# keys receive the notifications. Keys (un)set in SOPS secrets.infra.env.
PUSHOVER_API_TOKEN: ${PUSHOVER_API_TOKEN:-}
PUSHOVER_USER_KEY_ULLRICH: ${PUSHOVER_USER_KEY_ULLRICH:-}
volumes:
- ./grafana/provisioning:/etc/grafana/provisioning:ro
- ./grafana/dashboards:/var/lib/grafana/dashboards:ro