(Extremely) Basic Kafka Tutorial
This repository contains a super simple intro to Kafka tutorial. You will learn how to do two things:
Launch a Kafka broker locally using Docker
Connect to that Kafka broker using the
kafka-pythonlibrary and write to and read from a topic.
We’re going to run a Docker compose template that launches 3 containers:
broker - our Kafka broker
zookeeper - used by Kafka for leader election
jupyter - notebooks for connecting to our Kafka broker
Clone this repository:
git clone firstname.lastname@example.org:mneedham/basic-kafka-tutorial.git && cd basic-kafka-tutorial
Open a terminal window and run the following command:
Near the beginning of the output you should see the following text:
jupyter-tutorial | [I 10:35:27.804 NotebookApp] The Jupyter Notebook is running at: jupyter-tutorial | [I 10:35:27.804 NotebookApp] http://(4a031e4b5326 or 127.0.0.1):8888/?token=7907fef53948168308c829d48d9978b8f9c475b7c621c7d1 jupyter-tutorial | [I 10:35:27.804 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). jupyter-tutorial | [C 10:35:27.811 NotebookApp] jupyter-tutorial | jupyter-tutorial | To access the notebook, open this file in a browser: jupyter-tutorial | file:///home/jovyan/.local/share/jupyter/runtime/nbserver-6-open.html jupyter-tutorial | Or copy and paste one of these URLs: jupyter-tutorial | http://(4a031e4b5326 or 127.0.0.1):8888/?token=7907fef53948168308c829d48d9978b8f9c475b7c621c7d1
Grab that URL and the token and open it in a web browser. Given the above output we’d navigate to http://localhost:8888/?token=7907fef53948168308c829d48d9978b8f9c475b7c621c7d1, but you’ll need to replace the token with your own one.
Once you’ve done that open
Kafka Tutorial.ipynb and go through that.
The notebook installs a Python driver for Kafka, and then shows how to publish and consume a simple JSON message.
The producer and consumer code in the notebooks is adapted from Adnan Siddiqi’s Getting started with Apache Kafka in Python article and accompanying repository.