Alerting: Show alert rule thresholds in panels #79971
Conversation
gillesdemey
added
area/alerting
gillesdemey
requested review from
tskarhed,
JoaoSilvaGrafana,
dprokop,
kaydelaney and
konrad147
and removed request for
a team
| ); | ||
|
|
||
| // combineLatest will allow partial results depending on which API call resolves first | ||
| return combineLatest([fetchPromRules, fetchRulerRules]).pipe( |
| if (isAlertingRule(rule) && rule.annotations && rule.annotations[Annotation.panelID]) { | ||
| this.hasAlertRules[dashboard.uid] = true; | ||
| const panelId = Number(rule.annotations[Annotation.panelID]); | ||
| const state = promAlertStateToAlertState(rule.state); | ||
|
|
||
| // there can be multiple alerts per panel, so we make sure we get the most severe state: | ||
| // alerting > pending > ok | ||
| if (!panelIdToAlertState[panelId]) { | ||
| panelIdToAlertState[panelId] = { | ||
| state, | ||
| id: Object.keys(panelIdToAlertState).length, | ||
| panelId, | ||
| dashboardId: dashboard.id, | ||
| }; | ||
| } else if (state === AlertState.Alerting && panelIdToAlertState[panelId].state !== AlertState.Alerting) { | ||
| panelIdToAlertState[panelId].state = AlertState.Alerting; | ||
| } else if ( | ||
| state === AlertState.Pending && | ||
| panelIdToAlertState[panelId].state !== AlertState.Alerting && | ||
| panelIdToAlertState[panelId].state !== AlertState.Pending | ||
| ) { | ||
| panelIdToAlertState[panelId].state = AlertState.Pending; | ||
| } | ||
| } | ||
| }); |
There was a problem hiding this comment.
I know this is existing code but how about extracting this function for clarity?
|
don't merge this, PanelQueryRunner / old dashboard architecture is in a code freeze condition. Unless there is a matching feature PR for scenes this feature will be removed as we migrate to scenes in the next few weeks |
|
Setting this back to draft |
| try { | ||
| dashData = getDashboardQueryRunner().getResult(panel.id); | ||
| } catch (err) { | ||
| dashData = emptyResult(); |
There was a problem hiding this comment.
why are we joining in the dashboard query result here again? it is already joined with the query result here:
|
@gillesdemey we had an idea of using a plain transformation here, a transformation can be async (it's observable). It can modify the data coming in and set the field config to include the thrresholds config we could start with a manually added transformation (that users add). named "Sync alert rule thresholds" |
|
I think that's a great idea! It would definitely fit better with the scenes model 👍 |
|
This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 2 weeks if no further activity occurs. Please feel free to give a status update or ping for review. Thank you for your contributions! |
|
This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 2 weeks if no further activity occurs. Please feel free to give a status update or ping for review. Thank you for your contributions! |
|
This pull request has been automatically closed because it has not had any further activity in the last 2 weeks. Thank you for your contributions! |
What is this feature?
This feature adds visualisations of thresholds for dashboard panels when a Grafana alert is linked to it.
There are several caveats here worth mentioning;
refIDs for the panel's queries have to match therefIDof the alert rule linked to it. If they don't match it will either not show any thresholds or show incorrect ones.fieldConfig.overridesit will always take precedence over any existing manually configured threshold(s). ThePanelQueryRunnercurrently has no way to check for any manually configured thresholds.Why do we need this feature?
In legacy alerting the threshold of the alert was embedded in the panel's alert rule definition. Since we've decoupled alert rules from panels in the new Grafana alerting users had to manually configure thresholds on the panel.
This was both a duplicated effort and since there is only a weak reference between a panel and alert rules it also introduced the possibility of threshold definitions to drift between the panels visualisation and the alert rule definition.
Which issue(s) does this PR fix?:
Fixes #78748
Special notes for your reviewer:
It will use the
fieldConfig.overridesso the data frames can update the panel's visualisation options and show the thresholds.A user might have existing manual thresholds configured that we don't know about. I really haven't found any other way to update the panel's options from the
PanelQueryRunnerwithout refactoring a bunch here.Setting this to draft for now, still needs some tests added and some need fixing.
Any pointers for alternative implementations are welcome!
Please check that: