Skip to content

Data Driven Sentiment Insight into Twitter(X) Trends | Kafka | Spark | Spark MLlib | Docker

Notifications You must be signed in to change notification settings

Twitter-Analytics/trendz-insights

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Trendz-Insights

Uncovering twitter trends through sentiment analysis

We developed a real-time sentiment analysis system for Twitter (X) trends using Kafka and Spark. This project's aim is to leverage data-driven business insights to help companies assess and forecast public sentiment on trending topics.

This project aims to bridge the gap between data and people, offering a unique window into data-driven analytics and enhancing businesses' ability to make informed decisions based on real-time public opinion.

Trendz-Insights is a project that aims to analyze trends on social media, particularly Twitter, using a pipeline of technologies including Kafka, Spark Streaming, PostgreSQL, Power BI, and Spring Boot. The project utilizes external NLTK for NLP based sentiment analysis of tweets and trends, and TF-IDF for trend generation.

System Architecture

image

Features

  • Emulation of twitter API
  • Apache Kafka for high throughput asynchronous message queue.
  • Apache Spark as data processing engine.
  • PostgreSQL: for data storage.
  • NLTK for sentiment generation.
  • Spark MLlib for trend generation usng TF-IDF algoriCthm.

Final Power BI Visualisation

WhatsApp Image 2024-06-09 at 6 32 42 PM

Contributors

Releases

No releases published

Packages

No packages published