Revert "[serve] Replace Ray Client with Ray Job Submission for `serve… #33006
Conversation
ray-project#32976) - Previously, if connecting to a local cluster, `serve run` will directly call `ray.init()` and `serve.run` - With the change in ray-project#30913, even when connecting to a local cluster, `serve run` would submit a Ray Job to the local cluster and then call `ray.init()` and `serve.run()` inside that job. This caused the issue ray-project#32881, where if a user ran `serve run` inside a Job (or any other process with a runtime environment), they should expect to have access to the dependencies installed in that runtime environment. However since `serve run` goes through another layer of Job submission, they don't. Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
zcin
requested review from
edoakes,
simon-mo,
jiaodong,
shrekris-anyscale,
sihanwang41,
architkulkarni and
a team
as code owners
There was a problem hiding this comment.
Do you mind confirming that this PR has the relevant manual changes from #32976 (comment) (like the part about "one of them was made after the 2.3.0 branch cut, so it will be removed when cherry-picked onto the release branch")? Just asking because this PR seems to only have one commit, not sure if the manual change would require another commit or if it was done during the git cherry-pick process
Thanks for checking! Yes the change I was referring to is here, in a change made to master after the branch cut we changed the API from |
…… (#32976)
Why are these changes needed?
Cherry pick for #32976
serve runwill directly callray.init()andserve.runserve run#30913, even when connecting to a local cluster,serve runwould submit a Ray Job to the local cluster and then callray.init()andserve.run()inside that job. This caused the issue [serve]serve runcommand within job entrypoint fails if aworking_diris specified #32881, where if a user ranserve runinside a Job (or any other process with a runtime environment), they should expect to have access to the dependencies installed in that runtime environment. However sinceserve rungoes through another layer of Job submission, they don't.Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.