Skip to content
Hadoop-ec2 tool written in java.
Java Shell
Find file
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
bin
config
nbproject
src
test/com/lunabeat/dooper
.gitignore
README
build.xml
manifest.mf

README

Reimplementing some of the hadoop-ec2 tools without so much shell script as a way to get cozy with the AWS SDK.

Building - 
it's currently setup as a Netbeans project, will be getting the Netbeans stuff out soon.

depends on:
--aws-sdk 
http://aws.amazon.com/sdkforjava/

--commons-codec v. 1.4 (aws includes 1.3 but I had issues using it)
http://commons.apache.org/codec/

--log4j
http://logging.apache.org/log4j/

--ganymed (ssh lib)
http://www.cleondris.ch/opensource/ssh2/

setup the required libraries and build.

Configuring --
there's sample config files in dist/config
to launch a cluster you need to choose or build an AMI, preferably with hadoop installed, but you could install hadoop on startup without too much effort.

Running -- 

run pooper.sh in dist/bin after building.

As of this writing, HaddooperPooper 's command-list:
	command
	create-groups
	delete-cluster
	describe-cluster
	get-login-command
	launch-cluster
	launch-master
	launch-slaves
	list-clusters
	login
	push-file
	terminate-cluster
	terminate-slaves

Still to come:
port forwarding, 
automated job submission/termination, 
support for a map-reduce only (non-dfs) slave group that can be scaled separately,
GUI.










Something went wrong with that request. Please try again.