Skip to content
/ big-data Public

Big Data coursework using Hadoop/Spark, and concepts of mapreduce, graph theory, text mining, machine learning, etc.

Notifications You must be signed in to change notification settings

mt592/big-data

Repository files navigation

Big Data Course

A compilation of work completed in the Big Data course I took during my master's program.

Projects completed:

  • acm_citation_analysis (final project) - using the weighted page rank algorithm, understand which papers in the ACM database get cited most frequently and are most 'important' in the network. Read more here.
  • common_followers - explore which social media users mutually follow one another.
  • common_movies - identify people with the highest overlap in common movie viewing.
  • wikipedia_and_higgs - find symmetric wikipedia URLs. Create in-degree distribution of the higgs social media data set.
  • yelp_average_stars - calculate the average stars per business on Yelp using MapReduce and Map-Combine-Reduce.
  • yelp_restaurant_ml - Use text mining techniques and machine learning to predict the star rating of restaurants on Yelp using the raw text reviews.

Tech stack:

  • Hadoop
  • Scala
  • Spark/PySpark
  • Java

About

Big Data coursework using Hadoop/Spark, and concepts of mapreduce, graph theory, text mining, machine learning, etc.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published