Skip to content
Trying out the Python client with Kafka by ingesting and processing tweets from Twitter
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
tools
tweets_on_kafka
.gitignore
LICENSE
README.md
requirements.txt

README.md

tweets-on-kafka

Stream tweets from Twitter into Kafka and consume them back

Why Kafka and tweets?

Kafka is one of the many distributed message brokers that enables low latency message ingestion. Ingesting data such as tweets in real-time can be very useful. In this case, I could do some analysis in near real-time downstream using Spark or Beam to understand what people are talking about regarding weather and maybe retarget users with offers etc.

Useful reading

  1. Kafka in Docker: https://github.com/wurstmeister/kafka-docker
  2. Python Kafka Producer & Consumer: https://kafka-python.readthedocs.io/en/master/usage.html
  3. Streaming tweets using Tweepy: https://tweepy.readthedocs.io/en/3.7.0/streaming_how_to.html
You can’t perform that action at this time.