Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Serve] add blocking to serve.run() #43227

Merged
merged 9 commits into from
Feb 20, 2024

Conversation

GeneDer
Copy link
Contributor

@GeneDer GeneDer commented Feb 16, 2024

Why are these changes needed?

Rename the existing serve.run to serve._run and added blocking option to the new serve.run

Related issue number

Closes #43211

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Manually tested and seeing it behaves correctly

from ray import serve

@serve.deployment
def foo() -> str:
    return "bar"

handle = serve.run(foo.bind(), blocking=True)
xxx = "xxx"

image

Signed-off-by: Gene Su <e870252314@gmail.com>
Signed-off-by: Gene Su <e870252314@gmail.com>
Signed-off-by: Gene Su <e870252314@gmail.com>
Signed-off-by: Gene Su <e870252314@gmail.com>
Signed-off-by: Gene Su <e870252314@gmail.com>
@GeneDer GeneDer marked this pull request as ready for review February 16, 2024 18:53
@GeneDer GeneDer requested a review from a team February 16, 2024 18:53
@GeneDer
Copy link
Contributor Author

GeneDer commented Feb 16, 2024

Copy link
Contributor

@edoakes edoakes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Please also unify the CLI to call into the same flag

Signed-off-by: Gene Su <e870252314@gmail.com>
This reverts commit b8a2fd6.

Signed-off-by: Gene Su <e870252314@gmail.com>
Signed-off-by: Gene Su <e870252314@gmail.com>
@GeneDer GeneDer force-pushed the add-blocking-to-serve-run-sdk branch from 9fb3ccd to 6aff081 Compare February 19, 2024 03:35
Signed-off-by: Gene Su <e870252314@gmail.com>
@@ -590,8 +590,14 @@ def run(
if is_config:
client.deploy_apps(config, _blocking=False)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edoakes do you think as is is good enough or if we also want to add blocking to client.deploy_apps as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm that's an internal API anyways

@edoakes edoakes merged commit 3b9adee into ray-project:master Feb 20, 2024
9 checks passed
@GeneDer GeneDer deleted the add-blocking-to-serve-run-sdk branch February 20, 2024 17:12
khluu pushed a commit that referenced this pull request Feb 21, 2024
Rename the existing serve.run to serve._run and added blocking option to the new serve.run

---------

Signed-off-by: Gene Su <e870252314@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[serve] Add public blocking argument to serve.run
2 participants