Run Hadoop Custer within Docker Containers
Switch branches/tags
Nothing to show
Clone or download
Latest commit c83992f Jun 7, 2017

Run Hadoop Cluster within Docker Containers

alt tag

3 Nodes Hadoop Cluster

1. pull docker image
sudo docker pull kiwenlau/hadoop:1.0
2. clone github repository
git clone
3. create hadoop network
sudo docker network create --driver=bridge hadoop
4. start container
cd hadoop-cluster-docker
sudo ./


start hadoop-master container...
start hadoop-slave1 container...
start hadoop-slave2 container...
  • start 3 containers with 1 master and 2 slaves
  • you will get into the /root directory of hadoop-master container
5. start hadoop
6. run wordcount


input file1.txt:
Hello Hadoop

input file2.txt:
Hello Docker

wordcount output:
Docker    1
Hadoop    1
Hello    2

Arbitrary size Hadoop cluster

1. pull docker images and clone github repository

do 1~3 like section A

2. rebuild docker image
sudo ./ 5
  • specify parameter > 1: 2, 3..
  • this script just rebuild hadoop image with different slaves file, which pecifies the name of all slave nodes
3. start container
sudo ./ 5
  • use the same parameter as the step 2
4. run hadoop cluster

do 5~6 like section A