Puppet module for deploying Hadoop MapReduce Next Generation on cluster

This module deploys Hadoop MapReduce Next Generation on a cluster of machines. It is tested on Apache Hadoop 2.2.0 under puppet agent/master environment. It is based on bcarpio/hadoop 0.0.3.


Usage:

Install this module on your puppet master node by:

    sudo puppet module install viirya-hadoop.

In site.pp, define:

    node 'your hadoop slave nodes' {
        include java
        include hadoop::cluster::slave
    }

    node 'your hadoop master node' {
        include java
        include hadoop::cluster::master
    }

For pseudo-distributed mode of Hadoop, put following codes in a .pp file (e.g. hadoop.pp).

    include java
    include hadoop::cluster::pseudomode

Then,

    sudo puppet apply hadoop.pp


Hadoop Distribution:

This puppet module will automatically download Apache Hadoop distribution from one pre-defined Apache Mirror site. If you are like to use faster mirror site, please modify the URL in init.pp.


Parameters:

Some parameters are able to modify in params.pp. You should modify the parameters such as 'master', 'resourcemanager' and 'slaves' to reflect your hadoop cluster settings.

SSH keys:

Remember to generate your ssh keys and put the keys in files/ssh/.

Note: Since master deployment will run hadoop scripts to launch hadoop services on slaves nodes, please deploy hadoop slaves first. When all slaves are deployed, then deploy master node.