Sentiment-analysis-using-kafka-spark

Twitter sentiment analysis using Apache Kafka, Apache Spark structured streaming and dashboard monitoring page.

Table of Contents

1. About The Project
   • Built With
2. Modules
   • Kafka Producer
   • Kafka Consumer
   • Web server
3. Usage
4. Contributing
5. License

About The Project

This project is for Distributed enabling platforms course 21/22 A.Y at unipi given by Prof. Patrizio Dazzi. I implemeted a tweet sentiment analysis using distributed systems. Technologies used will be listed below. In this project, you will find three separate modules (applications).

• Kafka-producer : This module reads tweets from twitter API and publish tweets to kafka topic continously.
• Kafka-consumer : This is a spark application which uses spark structured streaming. It reads tweets from kafka server and make a prediction for each tweets and aggrigate the total tweet predicted for each sentiments(posetive, negative or neutral) and POST result to the web-server
• web-server : web-server is a flask web admin dashboard where all the total amount of tweets predicted and additional information are displayed

Technologies user

I have utilized the following libraries for the development of this project.

• Spark
• Kafka
• Flask
• Johnsnowlab Spark NLP

Getting Started

Loading ...

Usage

Loading . . .

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the MIT License. See LICENSE.txt for more information.

Contact

Website - joetelila.com

Project Link: https://github.com/joetelila/Sentiment-analysis-using-kafka-spark