trails-cool/trails
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix error-rate-high alert firing on no data

Browse source 
The alert was firing with DatasourceNoData because zero 5xx responses
means the numerator returns empty (not 0). Fixed by:
- Adding `or vector(0)` so zero 5xx returns 0% instead of empty
- Setting `noDataState: OK` as a safety net

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ullrich Schäfer 2026-03-29 22:20:14 +02:00
parent d45d6e3eaf
commit 3b6aede461
No known key found for this signature in database
GPG key ID: A32FF691A0F752D9
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
infrastructure/grafana/provisioning/alerting/alerts.yml
View file

@ -55,12 +55,13 @@ groups:
- uid: error-rate-high - uid: error-rate-high
title: Error rate > 5% title: Error rate > 5%
condition: B condition: B
noDataState: OK
data: data:
- refId: A - refId: A
relativeTimeRange: { from: 600, to: 0 } relativeTimeRange: { from: 600, to: 0 }
datasourceUid: prometheus datasourceUid: prometheus
model: model:
expr: sum(rate(http_request_duration_seconds_count{status=~"5.."}[5m])) / clamp_min(sum(rate(http_request_duration_seconds_count[5m])), 0.001) * 100 expr: (sum(rate(http_request_duration_seconds_count{status=~"5.."}[5m])) or vector(0)) / clamp_min(sum(rate(http_request_duration_seconds_count[5m])), 0.001) * 100
instant: true instant: true
- refId: B - refId: B
datasourceUid: __expr__ datasourceUid: __expr__