Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show rule evaluation errors on rules page #4446

Closed
matthiasr opened this Issue Aug 1, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@matthiasr
Copy link
Contributor

matthiasr commented Aug 1, 2018

Proposal

On the /rules UI page, show the last evaluation status and any errors for each rule. This is similar to how the status and errors for targets are shown on /targets.

Use case. Why is this important?

Humans are not perfect, and sometimes rule evaluation fails. This cannot be fully solved by testing beforehand, since e.g. many-to-many matching errors and timeouts are data dependent.

Right now, we need to find and look at the logs to see this error. Surfacing the last error would make this much easier. It's not a great help for fleetingly broken rules but most of the time when they break they stay broken for a while.

  • Logs:
2018-08-01_11:42:43.52603 level=warn ts=2018-08-01T11:42:43.525846289Z caller=manager.go:343 component="rule manager" group=<redacted> msg="Evaluating rule failed" rule="record: <redacted> \nexpr: (<redacted> / on(a, b) <redacted>)\n  * 100\n" err="many-to-many matching not allowed: matching labels must be unique on one side"
@matthiasr

This comment has been minimized.

Copy link
Contributor Author

matthiasr commented Aug 1, 2018

This would also implement one aspect of #1481.

@brian-brazil

This comment has been minimized.

Copy link
Member

brian-brazil commented Aug 1, 2018

Sounds like a good idea to me.

@noqcks

This comment has been minimized.

Copy link
Contributor

noqcks commented Aug 4, 2018

/assign moi

@lock

This comment has been minimized.

Copy link

lock bot commented Mar 22, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 22, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.