Specialization: Big Data for Data Engineers

Big Data Essentials: HDFS, MapReduce and Spark RDD

Assignments:

• Hadoop Streaming Assignment 0: Word Count
• Hadoop Streaming Assignment 1: Words Rating
• Hadoop Streaming Assignment 2: Stop Words
• Spark Assignment 1: Pairs
• Spark Assignment 2: Reconstructing the path
• Real-World Applications: TF-IDF

Big Data Analysis: Hive, Spark SQL, DataFrames and GraphFrames

Assignments:

• Hive Assignment 1. DDL: Create Tables
• Hive Assignment 2. DML: Find Most Popular Tags
• Spark Assignment 1: Counting number of the mutual friends
• Spark Assignment 2: Graph based Music Recommender