add support for combiners

[coyotemarin]

dumbo does this clever thing to support combiners; they basically add | sort | python mr_script.py --combiner to the end of the mapper command line. mrjob should do this as well.

We'll need to update our (lame) internal representation of steps, and update the --steps protocol to include combiners.

[irskep]

It also appears that Hadoop 0.20 supports arbitrary scripts for combiners, so we could just declare that we support it for that version and up.

(Let it be known that I am interested in having/doing this, as it brings significant opportunities for optimization.)

[coyotemarin]

Oh, sweet; I knew they added it, but I thought it was in a later version.

It's not a big deal to hack it with | sort | my_your_job.py --combiner for earlier versions, but we should definitely use -combiner in versions where it's available.

[irskep]

My other tasks are done or blocked so I started to think about what needs to be done here.

MRJob:

• Define combiner(), combiner_init(), and combiner_final()
• Add those methods to the step representation, mr(), steps(), and show_steps()
• Write run_combiner() and add it to execute()
• Command line options / is_mapper_or_reducer()
• Integrate with protocols

Other:

• Add some -combiner-vs-| sort... logic to mrjob.compat
• Add | sort... to local and inline runners or have them call combine() directly
• Add | sort... or -combiner to EMRJobRunner and HadoopJobRunner
• Figure out the logging situation (where are combiner failures written? probably just need to update TASK_ATTEMPTS_LOG_URI_RE.)
• Update examples and documentation

Oh, and boto doesn't support combiners at all.