Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CAY-1139, 1148] Implements Pregel on ET #1170

Merged
merged 25 commits into from
Jun 12, 2017
Merged

[CAY-1139, 1148] Implements Pregel on ET #1170

merged 25 commits into from
Jun 12, 2017

Conversation

JunhoeKim
Copy link
Contributor

@JunhoeKim JunhoeKim commented Jun 7, 2017

Closes #1139 #1148 .
This PR implements a simple version of Pregel on ET. The current version has an environment that only runs Pagerank app. And optimization problems related to graph processing will be covered in future issues.

This program works in the following way:

  1. Set the init state
  • PregelLauncher submits PregelDriver according to the specified configuration.
  • PregelDriver submits PregelWorkerTask to executors. Then all executors starts graph processing app.
  1. Communication with master and workers
  • In a single superstep, PregelMaster communicates with WorkerMsgManager in each executor using CentComm services.
  • SuperstepControlMsg and SuperstepResultMsg determine the termination of current app.
  1. Computation and tables
  • At every superstep, the Computation interface which are specified by app works around the vertices assigned to the executor.
  • ET has a vertex table and two message tables for current and next superstep. These message tables are supervised by MessageManager.

Copy link
Contributor

@wynot12 wynot12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JunhoeKim Overall it looks great :)
I've made some minor commits.
Could you take a look?
Thanks!

@JunhoeKim
Copy link
Contributor Author

@wynot12 It looks good to me. Thank you for refining works!

Copy link
Contributor

@wynot12 wynot12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JunhoeKim Thanks for your work :)
I'll merge it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants