Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bagel: Large-scale graph processing on Spark #48

merged 10 commits into from May 13, 2011


None yet
3 participants
Copy link

ankurdave commented May 3, 2011

Bagel is an implementation of the Pregel graph processing framework on Spark.

Bagel currently supports basic graph computation, combiners, and aggregators. Future work includes support for mutating the graph topology. Tests exist but currently don't run due to a Spark bug.

ankurdave added some commits Apr 13, 2011

Add Bagel test suite
Note: This test suite currently fails for the same reason that the
Spark Core test suite fails: Spark currently seems to have a bug where
any test after the first one fails.
Refactor and add aggregator support
Refactored out the agg() and comp() methods from

Defined an implicit conversion to allow applications that don't use
aggregators to avoid including a null argument for the result of the
aggregator in the compute function.

This comment has been minimized.

Copy link

tjhunter commented May 4, 2011

I would recommend you refactor your code before merging, it is always harder / less tempting to do after.


This comment has been minimized.

Copy link

mateiz commented May 9, 2011

This looks great, Ankur, except for two naming things: can you change the package name from bagel to spark.bagel, and can you rename the Pregel class to Bagel?


This comment has been minimized.

Copy link
Member Author

ankurdave commented May 9, 2011

Sure, I've done so.

mateiz added a commit that referenced this pull request May 13, 2011

Merge pull request #48 from ankurdave/bagel-new
Bagel: Large-scale graph processing on Spark

@mateiz mateiz merged commit 4b1f0f1 into mesos:new-rdds May 13, 2011


This comment has been minimized.

Copy link

mateiz commented May 13, 2011

Looks great, thanks. The one thing I should add is that maybe you should write a README documenting the examples, or a wiki page (and put a comment in the code pointing to this location).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.