Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vmalert: vmalert_alerting_rules_error should be a counter #5160

Closed
hagen1778 opened this issue Oct 11, 2023 · 1 comment
Closed

vmalert: vmalert_alerting_rules_error should be a counter #5160

hagen1778 opened this issue Oct 11, 2023 · 1 comment
Assignees
@dmitryk-dk
Labels
enhancement New feature or request vmalert

Comments

@hagen1778
Copy link
Collaborator

Is your feature request related to a problem? Please describe

vmalert exposes a gauge metric vmalert_alerting_rules_error which shows whether the current rule is failing or not. But it seems like a counter vmalert_alerting_rules_error_total would be more sufficient here, as it could show not only the state but also rate of failures. And also capture state changes between the scrapes, if any.

Describe the solution you'd like

Would be great to convert gauge vmalert_alerting_rules_error to counter vmalert_alerting_rules_error_total. The change will require not only changes to vmalert, but also to dashboards and recommended alerting rules.

Describe alternatives you've considered

No response

Additional information

No response
@hagen1778 hagen1778 added enhancement New feature or request vmalert TBD To Be Done labels Oct 11, 2023
@dmitryk-dk dmitryk-dk self-assigned this Oct 20, 2023
@dmitryk-dk dmitryk-dk mentioned this issue Oct 20, 2023
Merged
hagen1778 added a commit that referenced this issue Dec 6, 2023
@hagen1778
app/vmalert: replace error metrics for gauges with counter metrics
fb5a3d2 
See #5160

Signed-off-by: hagen1778 <roman@victoriametrics.com>
hagen1778 added a commit that referenced this issue Dec 6, 2023
@hagen1778
app/vmalert: replace error metrics for gauges with counter metrics
22caac0 
See #5160

Signed-off-by: hagen1778 <roman@victoriametrics.com>
hagen1778 added a commit that referenced this issue Dec 6, 2023
@hagen1778
app/vmalert: replace error metrics for gauges with counter metrics
ddba5f0 
See #5160

Signed-off-by: hagen1778 <roman@victoriametrics.com>
hagen1778 added a commit that referenced this issue Dec 6, 2023
@hagen1778
app/vmalert: replace error metrics for gauges with counter metrics
0eecc45 
See #5160

Signed-off-by: hagen1778 <roman@victoriametrics.com>
hagen1778 added a commit that referenced this issue Dec 6, 2023
@dmitryk-dk @hagen1778
app/vmalert: replace error metrics for gauges with counter metrics (#…
935bec4 
…5217)

See #5160

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
hagen1778 pushed a commit that referenced this issue Dec 6, 2023
@dmitryk-dk @hagen1778
app/vmalert: replace error metrics for gauges with counter metrics (#…
6a41e1e 
…5217)

See #5160

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 935bec4)
@valyala
Copy link
Collaborator

valyala commented Dec 12, 2023

vmalert_alerting_rules_error and vmalert_recording_rules_error metrics at vmalert are replaced with vmalert_alerting_rules_errors_total and vmalert_recording_rules_errors_total at the release v1.96.0. Closing this feature request as done.

@valyala valyala closed this as completed Dec 12, 2023
@valyala valyala removed the TBD To Be Done label Dec 12, 2023
@krehel krehel mentioned this issue Dec 13, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dmitryk-dk dmitryk-dk

Labels
enhancement New feature or request vmalert
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@valyala @hagen1778 @dmitryk-dk