See recent commits on https://github.com/dangra/dumbo
Had a quick look at the code and it looks great to me. Definitely some very nice contributions, thanks!
The "dumbo.contrib" package seems a fair enough place to put these reducers, but I wouldn't mind converting "dumbo.lib" in a package either as long as we put everything it already contains now (either directly or indirectly via imports) in "dumbo/lib/init.py" to preserve backwards compatibility. I might even prefer the latter, unless I'm overlooking some potential downsides?
I do have one comment after a quick glance through your code though: The tests you added should probably go in "tests/" instead of "dumbo/contrib".
GH-32: convert dumbo.lib module to a package
My time to apology for delayed answer, finally got some free time to work on this, check recent commits tagged by GH-32 in my fork. thanks!
GH-32: Add generic reducer to output raw files
- It is named RawReducer
- Handles outputing to single or multiples files (requires feathers)
- Include working examples to generate tokyo-cabinets, constant dbs, and jsonlines
- Last but not least, unittests!
Merged in 0094bdb. Thanks Daniel!