-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[environments] Support PEX executables #15518
Comments
Long term it seems likely we're going to have a long tail of extra features for runtime_env. One way of supporting this to make it easy to add new keys to runtime_env. This would involve at least:
cc @iycheng |
Yes, this is what I'm currently working on. Hopefully, I'll get the PR out soon. |
@iycheng Hi Yi, if you could share some pointers on the approach we would be happy to work on this and contribute back! |
@iycheng Hi Yi, I wonder if you are open for a discussion on this feature request? |
Hi @mingshi-wang sorry, I missed your message. Yes, I'm ok with some discussion of this one. I think right now, @architkulkarni also has some thoughts about this one, please involve him as well. |
PEX support should be very usefully. i see this request open 6 months ago did you have any estimations ? |
any updates on this? or can we contribute to this feature? |
We don't have a timeline for adding this, but it should be possible to write your own "third-party runtime env plugin" for this. We don't have documentation for the plugin API at the moment and it is still experimental, but the following test https://github.com/ray-project/ray/blob/2bedf2490d7b11c2728c3fc8c57cf337a5e296ef/python/ray/tests/test_runtime_env_plugin.py has some example plugins. If the plugin has enough interest we can consider merging it into the Ray repo. cc @scv119 |
Any updates on this? We're working with pants / PEX and would love to see this being supported! |
This is a feature request for supporting PEX in the runtime_env of ray, which is similar to how conda is supported (see issue #14524 2).
For example, we can package an actor “foo” in a PEX file “my_package.pex”:
And then initialize ray with the PEX environment to enable the worker to run “foo” in the PEX environment:
We have an example repo GitHub - leozc/fp20 1 to package ray actors into PEX files and run the actor in a local ray cluster. The commands are:
build ray_demo.pex in dist:
run ray workers in PEX environment:
What we would like to have is to execute the code in a real cluster (e.g. cloud, k8s, etc.).
The text was updated successfully, but these errors were encountered: