Kafka ZooKeeper Exporter
A daemon that exposes Kafka cluster state stored in ZooKeeper.
Metrics exported by
kafka_zookeeper_exporter provide cluster level overview
of the entire cluster and can be used along
jmx_exporter which provides broker
jmx_exporter exports what each brokers believes to be true, but
this information can be incorrect in case of a network partition or other split
brain issues. ZooKeeper on the other hand is the source of truth for the entire
cluster configuration and runtime status, so the metrics exported from it are
the best representation of the entire cluster status.
Number of partitions configured for given topic.
Number of replicas configured for given topic.
This metric will have value
1 for the replica that is currently the leader for
Each Kafka partition have a list of replicas, the first replica is the preferred
(default) leader. This metric will have value
1 if the current partition
leader is the preferred one.
This metric will indicate whenever given replica is in sync with the partition leader.
go get -u github.com/cloudflare/kafka_zookeeper_exporter cd $GOPATH/src/github.com/cloudflare/kafka_zookeeper_exporter make
Start the exporter
To see the list of avaiable flags run
Send a request to collect metrics
- zookeeper - required, address of the ZooKeeper used for Kafka, can be multiple addresses separated by comma
- chroot - path inside ZooKeeper where Kafka cluster data is stored. Has to be omitted if Kafka resides in the root of ZooKeeper.
- topics - optional, list of topics to collect metrics for, if empty or missing then all topics will be collected
Example Prometheus scrape job configuration:
- job_name: kafka_zookeeper_exporter_mycluster static_configs: - targets: # hostname and port `kafka-zookeeper-exporter` is listening on - myserver:9381 metrics_path: /kafka scheme: http params: zookeeper: ['zk1.example.com:2181,zk2.example.com:2181'] chroot: ['/kafka/mycluster']
This example uses
static_configs to configure scrape target.
See Prometheus docs for other
ways to configure it.