Kafka

Cluster management:

List cluster members:

<kafka_dir>/bin/zookeeper-shell.sh localhost:2181 ls /brokers/ids

Topic management:

Creating a topic:

<kafka_dir>/bin/kafka-topics.sh --create --bootstrap-server <kafka_ip>:<port> --replication-factor 1 --partitions 1 --topic my-topic

List topics:

<kafka_dir>/bin/kafka-topics.sh --list --bootstrap-server localhost:9092

Check topic details:

<kafka_dir>/bin/kafka-topics.sh --describe --bootstrap-server localhost:9092 --topic my-topic

List kafka consumer groups:

<kafka_dir>/bin/kafka-consumer-groups --all-groups --bootstrap-server <node ip>:9092 --describe

Check consumer group details:

<kafka_dir>/bin/kafka-consumer-groups.sh --group <group name> --bootstrap-server <node ip>:9092 --describe

Check overrides:

<kafka_dir>/bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topics-with-overrides

Set retention:

<kafka_dir>/bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --alter --config retention.ms=43200000 --topic <topic name> --command-config /opt/kafka/config/client-ssl.properties

When One-way SSL is enable:

Add this paramenter in the end of the command:

--command-config /opt/kafka/config/client-ssl.properties

File client-ssl.properties must have:

ssl.truststore.location=/opt/kafka/ssl/kafka-server-172.16.210.113.truststore.p12 ssl.truststore.password=BGHtjekRXBQW8 security.inter.broker.protocol=SSL

Disclaimer:

For the correct execution of the above commands you should consider that:

1. Kafka application directory is /opt/kafka
2. Zookeeper is running on 2181 port
3. Kafka is running on 9092 port

For more details please consult Kafka quickstart page: https://kafka.apache.org/quickstart