Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bagel: Large-scale graph processing on Spark #48

Merged
merged 10 commits into from May 13, 2011
Merged

Bagel: Large-scale graph processing on Spark #48

merged 10 commits into from May 13, 2011

Commits on May 3, 2011

  1. Add Bagel, an implementation of Pregel on Spark

    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    c0736f6 View commit details
    Browse the repository at this point in the history

  2. Clean up Pregel.run, add logging

    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    62ef620 View commit details
    Browse the repository at this point in the history

  3. Add Bagel classpath to run script

    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    45ec9db View commit details
    Browse the repository at this point in the history

  4. Update ShortestPath to work with controllable partitioning

    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    19122af View commit details
    Browse the repository at this point in the history

  5. Clean up Bagel source and interface

    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    c5b3ea7 View commit details
    Browse the repository at this point in the history

  6. Add Bagel test suite 

    Note: This test suite currently fails for the same reason that the
Spark Core test suite fails: Spark currently seems to have a bug where
any test after the first one fails.
    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    1c8ca0e View commit details
    Browse the repository at this point in the history

  7. Package combiner functions into a trait

    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    c18fa3e View commit details
    Browse the repository at this point in the history

  8. Refactor and add aggregator support 

    Refactored out the agg() and comp() methods from Pregel.run.

Defined an implicit conversion to allow applications that don't use
aggregators to avoid including a null argument for the result of the
aggregator in the compute function.
    @ankurdave
    ankurdave committed May 3, 2011
    Configuration menu
    Copy the full SHA
    563c5e7 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2011

  1. Move shortest path and PageRank to bagel.examples

    @ankurdave
    ankurdave committed May 4, 2011
    Configuration menu
    Copy the full SHA
    c110405 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2011

  1. Rename bagel to spark.bagel and Pregel to Bagel

    @ankurdave
    ankurdave committed May 9, 2011
    Configuration menu
    Copy the full SHA
    f40a089 View commit details
    Browse the repository at this point in the history