Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Hadoop-ec2 tool written in java.
Java Shell
tree: 13c36635c1

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
bin
config
nbproject
src
test/com/lunabeat/dooper
.gitignore
README
build.xml
manifest.mf

README

Reimplementing some of the hadoop-ec2 tools without so much shell script as a way to get cozy with the AWS SDK.

Building - 
it's currently setup as a Netbeans project, will be getting the Netbeans stuff out soon.

depends on:
--aws-sdk 
http://aws.amazon.com/sdkforjava/

--commons-codec v. 1.4 (aws includes 1.3 but I had issues using it)
http://commons.apache.org/codec/

--log4j
http://logging.apache.org/log4j/

--ganymed (ssh lib)
http://www.cleondris.ch/opensource/ssh2/

setup the required libraries and build.

Configuring --
there's sample config files in dist/config
to launch a cluster you need to choose or build an AMI, preferably with hadoop installed, but you could install hadoop on startup without too much effort.

Running -- 

run pooper.sh in dist/bin after building.

As of this writing, HaddooperPooper 's command-list:
	command
	create-groups
	delete-cluster
	describe-cluster
	get-login-command
	launch-cluster
	launch-master
	launch-slaves
	list-clusters
	login
	push-file
	terminate-cluster
	terminate-slaves

Still to come:
port forwarding, 
automated job submission/termination, 
support for a map-reduce only (non-dfs) slave group that can be scaled separately,
GUI.










Something went wrong with that request. Please try again.