Kafka Producer produces data from EventStream (SSE - Server Sent Events) and publish to the Topic and the Kafka Consumers can consume that messages
Using the Python the client library sseclient
to iterate over the http Server Sent Event (SSE) streams (also known as EventSource, after the name of the Javascript interface inside browsers).
pip install sseclient
Start the Zookeeper and the Kafka Server
zookeeper-server-start.sh ~/Desktop/kafka_2.13-3.2.1/config/zookeeper.properties
kafka-server-start.sh ~/Desktop/kafka_2.13-3.2.1/config/server.properties
Producer
python3 kafka_producer.py
gets the data from the Wikimedia EventStream and publish the message to the Kafka Topic wikimedia.recentchange
Consumer
python3 kafka_consumer.py wikimedia.recentchange cg1
consumes the message from the topic wikimedia.recentchange and the consumer is in the Consumer Group cg1.
The Producer will produce lots of data since it is reading from the EventSource.