Skip to content

Alerts Routed to Default Route Despite Conditional Configuration in Grafana OnCall #5480

Open
@aelogonpin

Description

@aelogonpin

What went wrong?

We are experiencing an issue where, even though a conditional routing rule is configured in Grafana OnCall, all incoming alerts from Alertmanager are always being routed to the default route, ignoring the specified conditions. Despite setting up escalation chains and custom routing rules, they are not being followed, and the alerts keep going to the default route.

Expected Behavior:
The alert should respect the routing condition and go through the correct custom escalation chain, not the default route.
Alerts with matching conditions should trigger the proper escalation paths, and notifications should follow the custom route.
The default route should only be used for alerts that do not match any other specified conditions.-

How do we reproduce it?

Steps to Reproduce:
Alertmanager sends alerts to Grafana OnCall.
Conditional routing is configured based on a specific condition in the route ( alertname, etc.).
Ex: {{ payload.commonLabels.alertname == "HttpServiceUnavailable" }}

Image

Despite the condition being true for the alert, it is always routed to the default route, bypassing the custom escalation chains and routes.
Escalation chains are not triggered for the appropriate route, and the alert ends up in the default route.

Grafana OnCall Version

v1.14.1

Product Area

Alert Flow & Configuration

Grafana OnCall Platform?

Kubernetes

User's Browser?

No response

Anything else to add?

In the picture if you look at it you can see that it points to a group to which I need to put these alerts.

For some reason this does not go through the first “if” and goes to the “default” that directs the alert to another group with other escalation chain, this has no integration, I just want to send alerts without integration.

And it also doesn't pick it up being the default route and sends it to me to the “testing grafana oncall” group.

Real case:

1st It arrives alert -> Grafana oncall picks it up -> For some reason it doesn't meet the IF -> It arrives to the default, but also doesn't route it to the scalation chain I need either.

Case I would like:
2º It arrives alert -> Grafana oncall picks it up -> If the conditionals are fulfilled it goes to the corresponding scalation or corresponding group and in the case that it goes to the default it goes to the corresponding scalation or group.

Image

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions