Skip to content
Docker file for Hadoop 3
Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore
Dockerfile
LICENSE
README.md
core-site.xml
hdfs-site.xml
hue.ini
mapred-site.xml
ssh_config
start-all.sh
yarn-site.xml

README.md

Docker file for Hadoop 3

Most of the work is coming from : http://bigdatums.net/2017/11/04/creating-hadoop-docker-image/

Just added a few adaptations for Hadoop 3.

For some details about Hadoop 3 (such as new ports), see: https://fr.slideshare.net/HadoopSummit/hadoop-3-in-a-nutshell

Please, read the content of Dockerfile, because it may be possible that you have to update it. See the comments about the tgz of hadoop3.

After starting the container, you can access the web UI:

Warning: hue is not fully functional... Its integration is a work in progess (file browsing is ok) !

How-to

  • Build the image
sudo docker build -t hadoop3 .
  • Run the container
sudo docker run --hostname=hadoop3 -p 8088:8088 -p 9870:9870 -p 9864:9864 -p 19888:19888 \
  -p 8042:8042 -p 8888:8888 --name hadoop3 -d hadoop3
  • Access the container
sudo docker exec -it hadoop3 bash
  • Test a job
yarn jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0.jar pi 10 100
  • Clean
sudo docker stop hadoop3 
sudo docker rm hadoop3 

Next steps

Product/Framework/Env. Version (R)equired/((O)ptional
Hue 4.1 R
Hive 2.3.2 R
Minifi ? O
Druid ? O
Kafka ? O
Storm ? O
Spark 2.2.0 O
Ambari 2.6.1 O
Ambari-metrics 2.6.1 O
HBase ? O

Some notes

You can’t perform that action at this time.