Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gRPC improvements, mTLS, and more #66

Merged
merged 22 commits into from Oct 27, 2020
Merged

gRPC improvements, mTLS, and more #66

merged 22 commits into from Oct 27, 2020

Conversation

mortendahl
Copy link
Member

@mortendahl mortendahl commented Oct 26, 2020

This PR does quite a bit:

  • separates the protobuf services, Executor and ChannelManager; this captures the fact that the former is used by the orchestrator while the latter is simply an implementation detail of (one way of doing) networking
  • makes it possible to use gRPC over mTLS
  • cleans up that way networking is done, avoiding the need to access local buffers through gRPC
  • renames Server to Worker
  • renames KernelBasedExecutor to AsyncExecutor (since they're all likely to be kernel based, but we could have other non-async strategies)
  • moves kernel implementations into isolated files
  • shows how docker compose can be used to spin up an extra "coordinator" service that simplifies how examples can be executed (in python-functions)
  • installs and uses watchexec to automatically restart workers have changes are detected to the source code

TODO:

  • restarting workers using watchexec should be for development only; maybe separate dev-worker image?
  • update all examples to match python-functions (see Update all examples to match python-functions #70)
  • update relevant example READMEs
  • mTLS for runtime as well

@mortendahl mortendahl changed the title gRPC mTLS and more gRPC improvements, mTLS, and more Oct 26, 2020
moose/worker.py Outdated Show resolved Hide resolved
moose/worker.py Outdated Show resolved Hide resolved
Copy link
Member

@yanndupis yanndupis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @mortendahl, excellent work on this 💪 ! I don't have other comments than the one mentioned by Dragos.

As mentioned in your TODOs would be great to have all the examples matching python-functions example but it can be done in a separate PR. Thanks

@mortendahl mortendahl merged commit 7b018c6 into prototype Oct 27, 2020
@mortendahl mortendahl deleted the grpc-mtls branch October 27, 2020 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants