New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reducers for outputting raw binary files, tokyo cabinet dbs or constant dbs #32

Closed
klbostee opened this Issue Mar 17, 2011 · 3 comments

Comments

Projects
None yet
2 participants
@klbostee
Owner

klbostee commented Mar 17, 2011

See recent commits on https://github.com/dangra/dumbo

@klbostee

This comment has been minimized.

Show comment
Hide comment
@klbostee

klbostee Mar 17, 2011

Owner

Had a quick look at the code and it looks great to me. Definitely some very nice contributions, thanks!

The "dumbo.contrib" package seems a fair enough place to put these reducers, but I wouldn't mind converting "dumbo.lib" in a package either as long as we put everything it already contains now (either directly or indirectly via imports) in "dumbo/lib/init.py" to preserve backwards compatibility. I might even prefer the latter, unless I'm overlooking some potential downsides?

I do have one comment after a quick glance through your code though: The tests you added should probably go in "tests/" instead of "dumbo/contrib".

Owner

klbostee commented Mar 17, 2011

Had a quick look at the code and it looks great to me. Definitely some very nice contributions, thanks!

The "dumbo.contrib" package seems a fair enough place to put these reducers, but I wouldn't mind converting "dumbo.lib" in a package either as long as we put everything it already contains now (either directly or indirectly via imports) in "dumbo/lib/init.py" to preserve backwards compatibility. I might even prefer the latter, unless I'm overlooking some potential downsides?

I do have one comment after a quick glance through your code though: The tests you added should probably go in "tests/" instead of "dumbo/contrib".

dangra added a commit to dangra/dumbo that referenced this issue Apr 21, 2011

@dangra

This comment has been minimized.

Show comment
Hide comment
@dangra

dangra Apr 21, 2011

Contributor

My time to apology for delayed answer, finally got some free time to work on this, check recent commits tagged by GH-32 in my fork. thanks!

Contributor

dangra commented Apr 21, 2011

My time to apology for delayed answer, finally got some free time to work on this, check recent commits tagged by GH-32 in my fork. thanks!

dangra added a commit to dangra/dumbo that referenced this issue Apr 21, 2011

GH-32: Add generic reducer to output raw files
- It is named RawReducer
- Handles outputing to single or multiples files (requires feathers)
- Include working examples to generate tokyo-cabinets, constant dbs, and jsonlines
- Last but not least, unittests!
@klbostee

This comment has been minimized.

Show comment
Hide comment
@klbostee

klbostee Apr 27, 2011

Owner

Merged in 0094bdb. Thanks Daniel!

Owner

klbostee commented Apr 27, 2011

Merged in 0094bdb. Thanks Daniel!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment