-
Notifications
You must be signed in to change notification settings - Fork 385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bagel: Large-scale graph processing on Spark #48
Conversation
Note: This test suite currently fails for the same reason that the Spark Core test suite fails: Spark currently seems to have a bug where any test after the first one fails.
Refactored out the agg() and comp() methods from Pregel.run. Defined an implicit conversion to allow applications that don't use aggregators to avoid including a null argument for the result of the aggregator in the compute function.
I would recommend you refactor your code before merging, it is always harder / less tempting to do after. |
This looks great, Ankur, except for two naming things: can you change the package name from bagel to spark.bagel, and can you rename the Pregel class to Bagel? |
Sure, I've done so. |
Bagel: Large-scale graph processing on Spark
Looks great, thanks. The one thing I should add is that maybe you should write a README documenting the examples, or a wiki page (and put a comment in the code pointing to this location). |
Bagel is an implementation of the Pregel graph processing framework on Spark.
Bagel currently supports basic graph computation, combiners, and aggregators. Future work includes support for mutating the graph topology. Tests exist but currently don't run due to a Spark bug.