METRICS.md

Metrics exposed by Kommitted

Namespace

Following Prometheus metric and label naming good practices, Kommitted uses a namespace that identifies the metrics it creates unequivocally. But in the interest of keeping things short, kmtd it is. Each metrics is hence named kmtd_<METRIC NAME>.

Metrics

Below is the list of the current Metrics exposed by Kommitted.

Consumer Metrics

kmtd_kafka_consumer_partition_lag_milliseconds

Description: The time difference (time lag) between when the latest offset was produced and the latest consumed offset was consumed, by the consumer of the topic partition, expressed in milliseconds. NOTE: '-1, -1' means 'unknown'.
Labels: cluster_id, group, topic, partition, member_id, member_host, member_client_id
Type: gauge
Timestamped: true

kmtd_kafka_consumer_partition_lag_offset

Description: The difference (lag) between the last produced offset and the last consumed offset, by the consumer of the topic partition. NOTE: '0, -1' means 'unknown'.
Labels: cluster_id, group, topic, partition, member_id, member_host, member_client_id
Type: gauge
Timestamped: true

kmtd_kafka_consumer_partition_offset

Description: The last consumed offset by the consumer of the topic partition. NOTE: '0, -1' means 'unknown'.
Labels: cluster_id, group, topic, partition, member_id, member_host, member_client_id
Type: gauge
Timestamped: true

Topic Partition Metrics

kmtd_kafka_partition_earliest_available_offset

Description: Earliest offset available to consumers of the topic partition.
Labels: cluster_id, topic, partition
Type: gauge
Timestamped: false

kmtd_kafka_partition_latest_available_offset

Description: Latest offset available to consumers of the topic partition.
Labels: cluster_id, topic, partition
Type: gauge
Timestamped: false

Topic Partition Offset Tracking Metrics

kmtd_kafka_partition_earliest_tracked_offset

Description: Earliest offset tracked to estimate the lag of consumers of the topic partition.
Labels: cluster_id, topic, partition
Type: gauge
Timestamped: true

kmtd_kafka_partition_latest_tracked_offset

Description: Latest offset tracked to estimate the lag of consumers of the topic partition.
Labels: cluster_id, topic, partition
Type: gauge
Timestamped: true

Cluster Metrics

Those are metrics specific to the component in Kommitted that fetches the set of Consumer Groups and Members from the Kafka cluster, and sends it to the rest of the system for further processing (i.e. to know which consumer groups to produce the lag information for).

kmtd_consumer_groups_total

Description: Consumer groups currently in the cluster.
Labels: cluster_id
Type: gauge
Timestamped: false

kmtd_consumer_groups_members_total

Description: Members of consumer groups currently in the cluster.
Labels: cluster_id, group
Type: gauge
Timestamped: false

kmtd_cluster_brokers_total

Description: Brokers currently in cluster.
Labels: cluster_id
Type: gauge
Timestamped: false

kmtd_cluster_topics_total

Description: Topics currently in cluster.
Labels: cluster_id
Type: gauge
Timestamped: false

kmtd_cluster_partitions_total

Description: Partitions currently in cluster.
Labels: cluster_id
Type: gauge
Timestamped: false

kmtd_cluster_topic_partitions_total

Description: Topic's Partitions currently in cluster.
Labels: cluster_id, topic
Type: gauge
Timestamped: false

Kommitted (internal) Metrics

cluster_status module

kmtd_cluster_status_emitter_fetch_time_milliseconds

Description: Time (ms) taken to fetch cluster status metadata.
Labels: cluster_id
Type: histogram
Timestamped: false

kmtd_cluster_status_emitter_channel_capacity

Description: Capacity of internal channel used to send cluster status metadata to rest of the service.
Labels: cluster_id
Type: gauge
Timestamped: false

consumer_groups module

kmtd_consumer_groups_emitter_fetch_time_milliseconds

Description: Time (ms) taken to fetch information about all consumer groups in cluster.
Labels: cluster_id
Type: histogram
Timestamped: false

kmtd_consumer_groups_emitter_channel_capacity

Description: Capacity of internal channel used to send consumer groups metadata to rest of the service.
Labels: cluster_id
Type: gauge
Timestamped: false

partition_offsets module

kmtd_partition_offsets_emitter_fetch_time_milliseconds

Description: Time (ms) taken to fetch earliest/latest (watermark) offsets of a specific topic partition in cluster.
Labels: cluster_id, topic, partition
Type: histogram
Timestamped: false

kmtd_partition_offsets_emitter_channel_capacity

Description: Capacity of internal channel used to send partition watermark offsets to rest of the service.
Labels: cluster_id
Type: gauge
Timestamped: false

kmtd_partition_offsets_register_usage

Description: Amount of offsets tracked per topic partition.
Labels: cluster_id, topic, partition
Type: gauge
Timestamped: false

Labels

Each metrics has some or all of the following labels applied; what labels applies depends on the level of specificity of each metric.

Specificity ⬇️	Name	Definition
Least	cluster_id	Identifier of the Kafka Cluster
More	topic	Name of the Topic
More	partition	(Numeric) identifier of the Topic Partition
More	group	Name of the Consumer Group
Most	member_id	Identifier of a Member in the Consumer Group
Most	member_host	Host of a Member in the Consumer Group
Most	member_client_id	Configured client.id of a Member in the Consumer Group