Kafka Setup Overview

Kafka Cluster

• The Kafka cluster refers to the collection of Kafka brokers that handle the storage and management of messages.
• In this setup, we are running a single Kafka broker as part of the cluster.
• The Kafka broker is defined by the KAFKA_BROKER_ID=1 environment variable when starting the Kafka container.

Bootstrap Server

• The bootstrap server is the initial Kafka broker address that clients (producers and consumers) connect to in order to discover the rest of the Kafka cluster.
• In this setup, the bootstrap server is configured as localhost:9092, where localhost refers to the host machine, and 9092 is the port exposed by the Kafka container.
• The bootstrap server is used to establish the connection between clients and the Kafka broker:

  podman exec -it kafka kafka-topics --create --topic test-topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

Kafka Topic

• A Kafka topic is a category or stream to which messages are sent by producers and consumed by consumers.
• In this setup, we create a Kafka topic named test-topic:

  podman exec -it kafka kafka-topics --create --topic test-topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

• Topics are managed by Kafka brokers, and messages are stored in partitions within those topics.

Summary

• Kafka Cluster: The Kafka container running as the broker (podman run -d --name=kafka ...).
• Bootstrap Server: The address localhost:9092 specified in the --bootstrap-server flag.
• Topic: The test-topic that is created with the kafka-topics --create command.