Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v9.5.x] Alerting: Fix state manager to not keep datasource_uid and ref_id labels in state after Error #77391

Merged
merged 2 commits into from Nov 13, 2023

Conversation

grafana-delivery-bot[bot]
Copy link
Contributor

Backport 78fc3bc from #72216


What is this feature?
This PR fixes a bug in the state manager that is described in #71467. It updates the state manager to clean up the labels sneaked after the cacheID was calculated. I added safety to remove only if the result does not contain these labels.

This is a temporary change to remediate the problem and will be changed in #68142

Before
before-no-sound.mp4
After
after-no-sound.mp4

Why do we need this feature?
This happens only in the case when the result does not have any labels, usually when the rule uses the classic condition. The normal evaluation uses labels from the rule only. When an error happens, the error result does not have labels either, and therefore it is matched to the normal state instead of being a new state (in multi-dimensional alerts when each normal result contains labels). The state manager adds extra labels after the state identifier is calculated. Therefore, this makes the normal result match the state that contains the extra labels, and therefore the extra labels (datasource_uid and ref_id) never go away from the state.

Who is this feature for?
Those who use classic condition and execution of Error as an error.

Which issue(s) does this PR fix?:
Fixes #71467

Special notes for your reviewer:

Please check that:

  • It works as expected from a user's perspective.
  • If this is a pre-GA feature, it is behind a feature toggle.
  • The docs are updated, and if this is a notable improvement, it's added to our What's New doc.

…els in state after Error (#72216)

(cherry picked from commit 78fc3bc)
@armandgrillet armandgrillet merged commit 84729d6 into v9.5.x Nov 13, 2023
10 checks passed
@armandgrillet armandgrillet deleted the backport-72216-to-v9.5.x branch November 13, 2023 17:00
@aangelisc aangelisc modified the milestones: 9.5.x, 9.5.14 Nov 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

3 participants