Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Python module that allows one to easily write and run Hadoop programs.
Python

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
examples
src
README
build-pymod.xml
build.xml

README

DESCRIPTION
"""""""""""

Originally, Dumbo was just a simple Python module that made writing 
and running Streaming programs very easy, but now it also consists 
of some helper code in Java. More generally, Dumbo can be considered
to be a convenient Python API for writing MapReduce programs.


INSTALLATION
""""""""""""

The Java code gets build together with the rest of Hadoop when the 
"dumbo/" directory is put in Hadoop's "src/contrib/", and the Python 
module can be installed by running

sudo ant -f build-pymod.xml install_pymod

in the "src/contrib/dumbo" directory. If the dir "dumbo/" is a subdir
of Hadoop's "src/contrib/", then the -f option can be omitted:

sudo ant install_pymod


USAGE
"""""

/usr/local/hadoop/bin/hadoop dfs -put examples/brian.txt brian.txt

python examples/wordcount.py -hadoop /path/to/hadoop \
-file excludes.txt -input brian.txt -output brian-wc

python -m dumbo cat brian-wc > brian-wc.txt


MORE INFO
"""""""""

http://github.com/klbostee/dumbo/wikis
Something went wrong with that request. Please try again.