BPLab

This repository serves as a series of labs in the course Big Data Processing: Comprehensive Experiments (2016 Spring term), Department of Computer Science and Technology, Nanjing University.

authors

Yueqi Chen (Yueqichen.0x0@gmail.com)
Wei Liu (liuwei13cs@smail.nju.edu.cn)
Zibo Zhou (441842096@qq.com)
Jiecai Yang

cluster environment

hadoop 2.7.1 (yarn)
java 1.6.0_17
CPU: Xeon (R) E5-2620
Memory: 64GB
Disk: 6TB
Network: 1 Gbps Ethernet

Lab content

Lab2: InvertedIndex Table
Lab3: Lab about HBase and Hive

file structure

./README.md ..................... file you are opening
./Lab2 .......................... directory for Lab2
	./Lab2/src .................. source directory for Lab2
    ./Lab2/src/InvertedIndex ........ directory for Inverted Index Table job
    ./Lab2/src/ResultSort ........... directory for Result Sort job
./Lab2/report ............... lab report for Lab2
./Lab3 .......................... directory for Lab3
./Lab3/src .................. source directory for Lab3
    ./Lab3/src/InvertedIndex ........ directory for InvertedIndex Index Table job
    ./Lab3/src/ReadHBase ............ read data from HBase
    ./Lab3/src/Optional ............. Optional job for Lab3
./Lab3/report ............... lab report fot Lab3
./Lab4 .......................... directory for Lab3
./Lab4/src .................. source directory for Lab4
    ./Lab4/src/Driver ............... directory for triangles counting with OR logic
    ./Lab4/src/Optional ............. directory for triangles counting with AND logic
./Lab4/report ............... lab report fot Lab4