Update README #6

Merged
merged 1 commit into from Feb 10, 2012

Conversation

Projects
None yet
2 participants
Contributor

espringe commented Feb 7, 2012

Scoobi has a combine primitive, which works just like your reduce (using hadoop's combine).

And scoobi also has really great support for transparent serialization. There's a bit of information here: https://github.com/NICTA/scoobi/wiki/Serialization (Note: the explicit stuff is very rarely required, it's just used to allow you to provide your own efficient schemes). And the pretty robust object graph serialization happens here: https://github.com/NICTA/scoobi/blob/master/src/main/scala/com/nicta/scoobi/impl/rtt/ClassBuilder.scala (It's actually using reflection to turn object graph into Java code that would completely reproduce it, then compiles it, and sends the byte code down)

Contributor

espringe commented Feb 7, 2012

Original comment:
https://github.com/espringe/scalding/commit/6efa0ea87ee28589f208c433e94e329b17c4175a#commitcomment-942149

(For some reason, github only attaches the comments to the commit, rather than to the pull request)

@johnynek thanks, I've updated the commit

Contributor

espringe commented Feb 9, 2012

@johnynek Uggh, sorry -- I'm an idiot, I hadn't noticed I spelt that wrong too. Fixed

johnynek merged commit 2f2bed5 into twitter:master Feb 10, 2012

Collaborator

johnynek commented Feb 10, 2012

Thanks! Merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment