Skip to content

srjth19/Analysis-of-Stack-Exchange-posts

Repository files navigation

CA675-Assignment-1-20211114

Assignment 1: Data Analysis

Documentation files:

  1. Assignment Documentation.pdf
  2. Complete chronological screenshots.pdf

Source Code files:

Stack Exchange Query:

Data extraction query codes from Stack Exchange.txt

Pig:

Pig ETL Source Code.txt

Hive:

Hive Query Source Code.txt

MapReduce for TF-IDF:

Hadoop source code for TF-IDF.txt

mapper1.py

mapper2.py

mapper3.py

mapper4.py

reducer1.py

reducer2.py

reducer3.py

Source for mapper and reducer scripts : https://github.com/devangpatel01/TF-IDF-implementation-using-map-reduce-Hadoop-python-.git

HDFS Commands:

General HDFS Commands Source Code.txt

Results:

Extracted data from Stack Exchange.zip

ETL Transformations using Pig.zip

Data queried from Hive.zip

TF-IDF calculated using MapReduce.zip

Releases

No releases published

Packages

No packages published

Languages