klbostee / dumbo
watch download tarball
public
Forks9Right
Watchers103Right
Description: Python module that allows you to easily write and run Hadoop programs.
Homepage: http://last.fm/dumbo
Clone URL: git://github.com/klbostee/dumbo.git 
version bump
klbostee (author)
Fri Mar 06 04:07:34 -0800 2009
commit  536827ab779da797cc14a99b1e898bd32f8c57b0
tree    419db8ce85746d64d32735d1f22501ad15b911ab
parent  c93545b2c1b600d31742737028b7f8ebb9c8b1ad
dumbo /
name age
history
message
file README Fri Dec 26 03:51:46 -0800 2008 added example, updated README, and added indivi... [Klaas Bosteels]
directory bin/ Fri Dec 26 03:51:46 -0800 2008 added example, updated README, and added indivi... [Klaas Bosteels]
file build-pymod.xml Tue Nov 04 03:28:22 -0800 2008 Fixed ant issues and updated README. [Klaas Bosteels]
file build.xml Fri Dec 26 03:18:58 -0800 2008 fit better into hadoop [Klaas Bosteels]
directory examples/ Thu Feb 26 03:15:53 -0800 2009 automatically add properties to mapper and redu... [klbostee]
directory src/ Fri Mar 06 04:07:34 -0800 2009 version bump [klbostee]
README
DESCRIPTION
"""""""""""

Originally, Dumbo was just a simple Python module that made writing 
and running Streaming programs very easy, but now it also consists 
of some helper code in Java. More generally, Dumbo can be considered
to be a convenient Python API for writing MapReduce programs.


INSTALLATION
""""""""""""

Dumbo should get built together with the rest of Hadoop when the 
"dumbo/" directory is put in Hadoop's "src/contrib/" directory. More
precisely, a "build/hadoop-*/contrib/dumbo/" directory should be
generated when you run "ant package" in Hadoop's root directory.


USAGE
"""""

contrib/dumbo/bin/put examples/brian.txt brian.txt

contrib/dumbo/bin/start examples/wordcount.py \
-input brian.txt -output brian-wc -inputformat text

contrib/dumbo/bin/cat brian-wc > brian-wc.txt


MORE INFO
"""""""""

http://github.com/klbostee/dumbo/wikis