Hadoop with Docker

• download hadoop-3.1.1.tar.gz.

├── hadoop
│   ├── Dockerfile
│   ├── prepare_hadoop.sh 
│   ├── hadoop-3.1.1.tar.gz
│   └── conf
│       ├── core-site.xml
│       ├── dfs-site.xml
│       ├── mapred-site.xml
│       └── yarn-site.xml

make build-hadoop

make start-hadoop

After that, open in browser:

http://localhost:50070

http://localhost:8088

Inside the hadoop:

/root/hadoop/bin/hdfs dfs -mkdir -p /user/root/tmp

/root/hadoop/bin/hdfs dfs -put ./data/data.csv /user/root/tmp/

cat data/data.csv | python3 mapreducer/mapper.py | sort | python3 mapreducer/reducer.py