Alerting: /api/prometheus/grafana/api/v1/alerts uses invalid state names

[piotrp]

/api/prometheus/grafana/api/v1/alerts endpoints returns alert's state as Normal/Pending/Alerting, instead of inactive/pending/firing as required by https://github.com/prometheus/compliance/blob/main/alert_generator/specification.md.

Looks like Grafana passes string value of its internal alert state, without any translation:

grafana/pkg/services/ngalert/api/api_prometheus.go

Line 61 in 623255c

State: alertState.State.String(),

grafana/pkg/services/ngalert/eval/eval.go

Lines 111 to 134 in 623255c

	type State int
	const (
	// Normal is the eval state for an alert instance condition
	// that evaluated to false.
	Normal State = iota
	// Alerting is the eval state for an alert instance condition
	// that evaluated to true (Alerting).
	Alerting
	// Pending is the eval state for an alert instance condition
	// that evaluated to true (Alerting) but has not yet met
	// the For duration defined in AlertRule.
	Pending
	// NoData is the eval state for an alert rule condition
	// that evaluated to NoData.
	NoData
	// Error is the eval state for an alert rule condition
	// that evaluated to Error.
	Error
	)

Tested on Grafana 8.5.6.

[piotrp]

After looking at this a bit longer I think it's by design, and this endpoint is only inspired by Alert Generator spec, and doesn't try to follow it

[JohnnyQQQQ]

Thanks for reaching out @piotrp, we already run the alert generator spec against grafana alerting and we are not fully compliant yet. Our goal is to become eventually compliant.

I will keep this issue open to track this part of the effort.

[piotrp]

Thank you, it's good to know where this is going. I'm writing custom integration that puts all active alerts into a table and sends it via email, so I'm looking closely at what is available in current APIs, and what can be inferred by looking at available annotations/labels.

[piotrp]

Small update, for completeness sake:

• health - error is reported by "error" instead of "err" (plus there's additional "nodata" health state)
• there's also "NoData" alert state

For my use it would be ideal to have additional Grafana-specific information present in additional attributes (or private annotations?):

• explicit nodata indicator (because compliance means that this will disappear from places where it's currently present, and would need to be inferred from present labels)
• multiple values coming from firing alert with classic condition, here even having __value_string__ annotation, like in /alertmanager, would work as this can be parsed to extract metric names and values
• some way to tell whether alert is silenced

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last year. It will be closed in 30 days if no further activity occurs. Please feel free to leave a comment if you believe the issue is still relevant. Thank you for your contributions!