[Bug] Unified Alerting Not Working

[celestialorb]

Describe the bug
When enabling unified alerting for a Grafana deployment I still receive one alert for each replica of the deployment.

Version
v4.7.0

To Reproduce
Steps to reproduce the behavior:

1. Enable unified alerting in the Grafana configuration spec.
2. Create the Grafana instance with the operator with multiple replicas.
3. Create an alert that will fire.
4. Verify that multiple instances of the alert are received despite unified alerting being enabled.

Expected behavior
I expect that enabling unified alerting in the Grafana spec would provision a Grafana deployment/configmap with proper unified alerting.

Suspect component/Location where the bug might be occurring
I suspect this is being caused by not providing the ha_peers configuration under unified_alerting in the config map. If the operator can manage a service for the Gossip protocol and set that service to the ha_peers I believe that might resolve the issue. This should be similar to how the Helm chart solves it.

[pb82]

@celestialorb what Grafana version are you using? I believe it has to be at least v8.0.0. You can override the base image if Grafana that the operator deploys.

[celestialorb]

@celestialorb what Grafana version are you using? I believe it has to be at least v8.0.0. You can override the base image if Grafana that the operator deploys.

I'm using Grafana 9.3.0.

[NissesSenap]

@celestialorb wasn't unified alerting deprecated in version 9?

We are not grafana maintainers so we don't know all possible grafana configuration options.
This to me sounds that we are missing a configuration value under: https://github.com/grafana-operator/grafana-operator/blob/master/documentation/api.md#grafanaspecconfigunified_alerting

I look at the docs and it seems like it's missing the ha_peers config that you mention: https://grafana.com/docs/grafana/latest/alerting/set-up/configure-high-availability/#enable-alerting-high-availability-in-grafana

To add this it's a simple fix and we are happy to take PR:s

[celestialorb]

@celestialorb wasn't unified alerting deprecated in version 9?

We are not grafana maintainers so we don't know all possible grafana configuration options. This to me sounds that we are missing a configuration value under: https://github.com/grafana-operator/grafana-operator/blob/master/documentation/api.md#grafanaspecconfigunified_alerting

I look at the docs and it seems like it's missing the ha_peers config that you mention: https://grafana.com/docs/grafana/latest/alerting/set-up/configure-high-availability/#enable-alerting-high-availability-in-grafana

To add this it's a simple fix and we are happy to take PR:s

If I recall correctly, unified alerting was opt-in in Grafana 8 and then made the default in Grafana 9.

I can take a look at crafting together a simple PR for it.

[NissesSenap]

I'm like 100% sure that had read it somewhere but I guess I misunderstood it and after the grafana homepage update I'm unable to find anything about it more or less. Sorry about that.

Looking forward to your pr @celestialorb