Add thresholds to Alerts

[sylr]

Hi,

I think Prometheus Alert system is lacking a real thresholds mechanism.

To me duplicating alerts with different conditions and label is a rather ugly workaround.

Something like this would be nice (it's really just to demonstrate, no a request to implement that exact syntax):

ALERT node_cpu_iowait
IF round(avg(rate(node_cpu{mode="iowait"}[2m]) * ON(instance) GROUP_LEFT(nodename) node_uname_info) BY (nodename) * 100)
FOR 5m
THRESHOLDS [
    {ge=20, labels=[severity="critical"]},
    {ge=10, lt=20, labels=[severity="warning"]},
]
LABELS {
}
ANNOTATIONS {
    description="Node {{ $labels.nodename }} has {{ $value }}% of iowait", 
    summary="{{ $labels.nodename }} - High CPU iowait (> 15%)"
}

Cheers.

[brian-brazil]

This is something we're very unlikely to implement. PromQL expressions and thus alerts can be far more sophisticated than a simple threshold would allow for. It's suggested to use configuration management if the repetition is an issue for you.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.