Expose resource alarms as Prometheus metrics

[dcorbacho]

Alarms could make it into a core metrics table as binary entries and be reported by Prometheus.
CC @MarcialRosales

[michaelklishin]

Good idea.

[deadtrickster]

I can write a collector that calls rabbit_alarm:get_alarms and exposes alarms as metrics with nodes as labels. Is this what you had in mind?

[deadtrickster]

this is how one of the unofficial exporters handles alarms:

https://github.com/kbudde/rabbitmq_exporter#node---counter

[gerhard]

That sounds like a great idea @deadtrickster!

This comment from a larger 2019 discussion with @brian-brazil explains why node labels should not be used. Has this recommendation changed? This is the reason why we make extensive use of * on(instance) group_left(rabbitmq_cluster, rabbitmq_node) in all our PromQL.

I'm thinking:

rabbitmq_memory_alarm 0
rabbitmq_disk_alarm 0

By the way:

rabbitmq-server/deps/rabbitmq_prometheus/src/collectors/prometheus_rabbitmq_core_metrics_collector.erl

Lines 24 to 29 in a1561ff

	%% Because all metrics are from RabbitMQ's perspective,
	%% cached for up to 5 seconds by default (configurable),
	%% we prepend rabbitmq_ to all metrics emitted by this collector.
	%% Some metrics are for Erlang (erlang_), Mnesia (schema_db_) or the System (io_),
	%% as observed by RabbitMQ.
	-define(METRIC_NAME_PREFIX, "rabbitmq_").

[deadtrickster]

ok right, instance label is enough already. so boolean metrics for each resource type without any labels.

[lhoguin]

Anything else worth importing from the unofficial exporter?

[johanneswuerbach]

Anything else worth importing from the unofficial exporter?

In our setup we also use the rabbitmq_partitions metric, would it also make sense to port this one?

There is another issue open for that though #2508