added gzip and bzip2 support for local (non-hadoop) dumbo job #77

Closed
wants to merge 1 commit into
from

Conversation

Projects
None yet
1 participant

I added gzip and bzip2 file support for mapper input.

The current dumbo seems only support this kind of mapreduce in local mode.
cat input | mapper | sort | reducer > output

What I added works like this.
zcat input | mapper | sort | reducer > output
bzcat input | mapper | sort | reducer > output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment