The notification policy and contact points live in `infrastructure/grafana/provisioning/alerting/alerts.yml`, so UI-created contact points can't be attached to provisioned alerts. Provision the Pushover integration instead. Rather than add a second contact point and do the fan-out in the notification tree (Grafana's tree only fires one receiver per match without duplication workarounds), put both email + pushover under a single `default` contact point. Every alert now fans out to both. Secrets go to SOPS `secrets.infra.env`. The Grafana container reads `PUSHOVER_API_TOKEN` and `PUSHOVER_USER_KEY_ULLRICH` from env; the provisioning YAML references them via `$VAR` and Grafana substitutes at startup, so no secret lands in git. Priority 1 on firing (bypass quiet hours), 0 on resolve. User can tweak in the YAML later. Post-deploy: delete the UI-created "Pushover Ullrich" contact point in Grafana to avoid a duplicate. It's unused (no policy references it) but shows up in the contact-point list. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| dashboards | ||
| provisioning | ||