Puppet module for deploying Hadoop MapReduce Next Generation (MRv2)
Puppet HTML Other
Latest commit fbe79ff Jun 7, 2015 @viirya Merge pull request #3 from bleuchtang/master
README - add module in puppet install command


Puppet module for deploying Hadoop MapReduce Next Generation on cluster

This module deploys Hadoop MapReduce Next Generation on a cluster of machines. It is tested on Apache Hadoop 2.2.0 under puppet agent/master environment. It is based on bcarpio/hadoop 0.0.3.


Install this module on your puppet master node by:

    sudo puppet module install viirya-hadoop.

In site.pp, define:

    node 'your hadoop slave nodes' {
        include java
        include hadoop::cluster::slave

    node 'your hadoop master node' {
        include java
        include hadoop::cluster::master

For pseudo-distributed mode of Hadoop, put following codes in a .pp file (e.g. hadoop.pp).

    include java
    include hadoop::cluster::pseudomode


    sudo puppet apply hadoop.pp

Hadoop Distribution:

This puppet module will automatically download Apache Hadoop distribution from one pre-defined Apache Mirror site. If you are like to use faster mirror site, please modify the URL in init.pp.


Some parameters are able to modify in params.pp. You should modify the parameters such as 'master', 'resourcemanager' and 'slaves' to reflect your hadoop cluster settings.

SSH keys:

Remember to generate your ssh keys and put the keys in files/ssh/.

Note: Since master deployment will run hadoop scripts to launch hadoop services on slaves nodes, please deploy hadoop slaves first. When all slaves are deployed, then deploy master node.