Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Bug 4252 Restore functioning benoml.ray.deployment #4257

Merged
merged 1 commit into from
Nov 2, 2023

Conversation

jerryharrow
Copy link
Contributor

@jerryharrow jerryharrow commented Nov 2, 2023

What does this PR address?

Two problems were preventing benoml.ray.deployment from working:

Fixes #4252

Before submitting:

Two problems were preventing benoml.ray.deployment from working
- Incorrect path reference to _internal.utils pkg
  Fixed to reference parent path correctly.
- Missing abstract async_stream_method in the RayRunnerHandle
  Added a stub that throws NotImplementedError like TitanRunnerHandle
@jerryharrow jerryharrow requested a review from a team as a code owner November 2, 2023 19:32
@jerryharrow jerryharrow requested review from aarnphm and removed request for a team November 2, 2023 19:32
Copy link
Member

@aarnphm aarnphm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. We will release a patch promptly.

@aarnphm aarnphm merged commit dde6ac7 into bentoml:main Nov 2, 2023
27 of 44 checks passed
@tozzi21
Copy link

tozzi21 commented Nov 4, 2023

created a facebook--opt-125m-service bentoml container, and attempted to instantiate via RayServe in serve run python_file

(ServeReplica:default:bento-runner-llm-opt-runner pid=852) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::SERVE_REPLICA::default#bento-runner-llm-opt-runner#JhDlMg:ServeReplica:default:bento-runner-llm-opt-runner.init() (pid=852, ip=172.17.85.4, actor_id=e9ed32d116920e8d9f11639e03000000, repr=<ray.serve._private.replica.ServeReplica:default:bento-runner-llm-opt-runner object at 0x7f163f378700>)

RuntimeError: Deploying application default failed: The deployments ['bento-runner-llm-opt-runner', 'bento-svc-llm-opt-service'] are UNHEALTHY.
2023-11-04 20:54:00,043 ERR scripts.py:564 -- Received unexpected error, see console logs for more details. Shutting down...

attached log.

serve run bento_ray-colon-classifier.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: unable to start BentoML container in RayServe
3 participants