Twitter-Live-Stream-Spark

Using Spark streaming to stream live Twitter data and performing different kind of operations.

Opeartion:

• Setting cores of processor and memory limit
• Establishing connection with Twitter developer APIs
• Capturing tweets
• Calculating number of character, word count per tweet
• Displaying hashtags per tweet
• Avereage numbe rof characters and words in tweet per RDD
• Finding top 10 hash tags based on their number of occurrence
• Performing opeartions 6-7 for for last 5 min og tweets, computing every 30 sec

Tags: Java,Spark-Streaming

Twitter Developer portal : https://developer.twitter.com/en (To generate consumerkey,consumersecrets etc)

Reference:

• http://spark.incubator.apache.org/docs/latest/streaming-programming-guide.html
• http://spark.apache.org/docs/latest/index.html
• https://developer.twitter.com
• http://ampcamp.berkeley.edu/big-data-mini-course/realtime-processing-with-spark-streaming.html