Skip to content

Docker container for a single node Kafka broker

Jump to bottom
authorjapps edited this page Mar 10, 2019 · 14 revisions 
 
After following this page you should be successfully able to bring up a dockerized 
Kafka broker and run some tests.
                                                          (Author:Santhosh Kumar)

Table of Contents

How to configure/create a Docker compose file?

See here the docker-compose file

How to bring down the container using the compose file ?

  • $ docker-compose -f kafka-single-node.yml down

If the containers are up, the above command will bring them down and if they are not up, then it does no harm.

How to bring up the container using the compose file?

  • Clone this repo

  • Go to the zerocode-docker-factory/compose dir in a terminal window

  • Then run

    • $ docker-compose -f kafka-single-node.yml up -d

    or

    • $ docker-compose -f kafka-single-node.yml up

This brings up both Zookeeper and Kafka server(server and broker mean the same thing on this page)

How to check the status of Kafka and Zookeeper?

$ docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                          NAMES
9b5001dc2931        confluentinc/cp-kafka:5.0.1       "/etc/confluent/dock…"   3 hours ago         Up 3 hours          0.0.0.0:9092->9092/tcp         compose_kafka_1
037afbb001ac        confluentinc/cp-zookeeper:5.0.1   "/etc/confluent/dock…"   3 hours ago         Up 3 hours          2181/tcp, 2888/tcp, 3888/tcp   compose_zookeeper_1
$ 

Note-
See under the "NAMES" column(right hand ride) above, for `compose_kafka_1` as container name
and the "PORTS" column to see which port the container is listening to for the external listners.

How to verify the port has been exposed outside docker container ?

  • To make sure Kafka broker is running and it has exposed the port 9092 to external client outside the docker,

    • nc -vz localhost 9092

$ nc -vz localhost 9092   
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
        outif lo0
        src ::1 port 51288
        dst ::1 port 9092
        rank info not available
        TCP aux info available

Connection to localhost port 9092 [tcp/XmlIpcRegSvc] succeeded!

Note- If a port is not exposed, you get the following message.
--------------------------------------------------------------
$ nc -vz localhost 9093  
nc: connectx to localhost port 9093 (tcp) failed: Connection refused
nc: connectx to localhost port 9093 (tcp) failed: Connection refused
$

How to verify that a message can be produced to and consumed from a topic?

Get into the container

$ docker exec -it compose_kafka_1 bash
root@9b5001dc2931:/#

For producing a message to a topic(named test)

root@9b5001dc2931:/# echo "test-XYZ" | kafka-console-producer --broker-list kafka:29092 --topic test
>>root@9b5001dc2931:/# 
Note-
If you see no error means, it might been produced correctly

For consuming a message from a topic(named test)

root@9b5001dc2931:/# kafka-console-consumer --bootstrap-server kafka:29092 --topic test --from-beginning
test-XYZ

Press Ctrl+c to stop this process and come to the command prompt

Now, how to run the Zerocode tests?

In your kafka_server.properties upddate the broker host to kafka.bootstrap.servers=localhost:9092 and run the Producer and Consumer tests.

HelloWorld tests are here

How to bring down the containers once you are done with your testing?

Already explained in the above section, see here

Clone this wiki locally