R and Hadoop Integrated Programming Environment
R Java C++ C Other
Pull request Compare This branch is even with delta-rho:master.
Latest commit d3eed56 Dec 11, 2015 @hafen hafen Merge pull request #39 from hafen/master
Failed to load latest commit information.
conf/maven restructured project into standard maven format Nov 25, 2013
.gitignore Almost done moving to rJava, read and write functions to be implemented Jan 22, 2013
NEWS.md Update NEWS.md Dec 12, 2015
README.md Update to be compatible with R 3.2 Apr 19, 2015


RHIPE: R and Hadoop Integrated Programming Environment

RHIPE is an R package that provides a way to use Hadoop from R. It can be used on its own or as part of the Tessera environment.


Installation of RHIPE requires a working Hadoop cluster and several prerequisites. A comprehensive installation guide, as well as other options to get an environment set up (Vagrant, Amazon Web Services, etc.) are discussed here.

If you are interested in installing and using RHIPE, please see the link just provided. The remainder of this README will focus on developers who want to build RHIPE from source or contribute to RHIPE development.

Hadoop Support

Our current development efforts are focused on Hadoop 2 (YARN), although code that works with earlier versions of Hadoop is also available in this repository.

Hadoop 2

The master branch of this repository is focused on Hadoop 2 development, and contains code to build RHIPE v0.75.x. As some aspects of YARN that we have addressed are not backward compatible, packages built from this branch will not work with Hadoop 1.

Hadoop 1

The v0.74 branch of this repository is for Hadoop 1.

Hadoop Distributions

There are several Hadoop distributions avaialble. RHIPE has been successfully built and run for Apache Hadoop 1.x , Cloudera CDH3, CDH4mr1, and CDH5mr2. There are maven profiles setup in the POM that build against Apache Hadoop 1.x, 2.x,CDH3, CDH4, CDH5, HDP 1,2 & 2.2.

Building Rhipe

Probably the easiest way to build RHIPE is to provision a Vagrant machine that has all the prerequisites configured. Another option is to set up a local pseudo-distributed Hadoop cluster, for example see here.

If you set up your own machine, you will need to make sure the following dependencies are met, beyond Hadoop:

Rhipe is built using both Ant and Maven. Maven handles the Java build with the various distro dependencies. Ant drives the build process including the R packaging, running R commands and testing.


with no targets prints help.

To build Rhipe for a specific distro run

ant build-distro -Dhadoop.version=[hadoop-1,hadoop-2,cdh3,cdh4,cdh5,hdp-1,hdp-2]