Skip to content

myHadoop is a simple system for end-users to provision Hadoop instances on traditional supercomputing resources, without requiring any root privileges. Users may use myHadoop to configure and instantiate Hadoop on the fly via regular batch scripts.

master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
bin
 
 
 
 
etc
 
 
 
 
 
 
 
 

README.md

NSF-0844530

UPDATE myHadoop is no longer being updated here. Originally written by Sriram Krishnan at SDSC, it is currently being maintained at https://github.com/glennklockwood/myhadoop/

myHadoop

myHadoop enables the use of Hadoop in a non-dedicated cluster environment, being administered by typical batch scheduler. We are currently supporting the PBS (Moab) and the Sun Grid Engine (SGE) schedulers, although a port to a scheduler such as Condor would be very trivial.

The pbs-example.sh and sge-example.sh provide an example of how to use myHadoop with PBS and SGE respectively. For more details, please read the documentation in the "docs" directory.

Pre-requisites

myHadoop needs Apache Hadoop 0.20.2 and a batch scheduler such as PBS.

Other

Work was funded by a grant from the NSF Cluster Exploratory (CluE) program (Award# IIS-0844530, PI: Baru, Co-PI Krishnan) More info: http://nsf.gov/awardsearch/showAward?AWD_ID=0844530

About

myHadoop is a simple system for end-users to provision Hadoop instances on traditional supercomputing resources, without requiring any root privileges. Users may use myHadoop to configure and instantiate Hadoop on the fly via regular batch scripts.

Resources

License

Releases

No releases published

Packages

No packages published

Languages

You can’t perform that action at this time.