ITS-ICT BigData

Labs

00 - Prerequisites

01 - Fork and clone this repo

02 - Provision the environment

03 - HFDS file upload using CLI

04 - HFDS file upload using browser

05 - MapReduce word count

06 - Code MapReduce job

07 - Hadoop Streaming

08 - Yarn

09 - Pig

10 - Spark installation

11 - Spark RDDs basics

12 - Spark SQL basics

13 - Spark with RDBMS

14 - Install Kafka

15 - Exchange messages with Kafka

16 - Kafka partition replication

17 - Kafka consumers

18 - Processing Kafka messages with Spark

19 - Apache Nifi MQTT to Kafka

20 - Install Apache Flink

Assignments

01 - PV energy production insights with MapReduce

02 - COVID pandemic insight with MapReduce

03 - PV energy production insights with Pig

04 - COVID pandemic insight with Pig

05 - Counting words using Spark RDDs

06 - Counting succeeded SpaceX launches with Spark

07 - Find SpaceX rocket launch cost

08 - Search in Strava activities

09 - Milano open WiFi insight

Appendices

A - Sync a local copy of forked repo