-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Core][2/3] Make streaming generator public API #38784 #41436
[Core][2/3] Make streaming generator public API #38784 #41436
Conversation
@@ -544,9 +544,9 @@ def _submit_task( | |||
"""Submit the task with index task_idx. | |||
|
|||
NOTE: When dynamic block splitting is enabled, returns | |||
Tuple[ObjectRef[ObjectRefGenerator], None] instead of | |||
Tuple[ObjectRef[DynamicObjectRefGenerator], None] instead of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc @raulchen is this still relevant with data?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this code path is being deprecated.
@pcmoritz @ericl this PR will have a breaking change to num_returns="dynamic" because it changes the name of the class to ObjectRefGenerator -> DynamicObjectRefGenerator, so that If this is concerning, we can instead use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I remember correctly, we said that dynamic is experimental so it should be fine?
@@ -233,7 +233,7 @@ def _send_query_java(self, query: Query) -> ray.ObjectRef: | |||
|
|||
def _send_query_python( | |||
self, query: Query | |||
) -> Union[ray.ObjectRef, "ray._raylet.StreamingObjectRefGenerator"]: | |||
) -> Union[ray.ObjectRef, "ray._raylet.ObjectRefGenerator"]: | |||
"""Send the query to a Python replica.""" | |||
if query.metadata.is_streaming: | |||
method = self._actor_handle.handle_request_streaming.options( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe you can/should remove the num_returns="streaming"
here
Why are these changes needed?
This is the second PR of 3 steps.
All sort of implementation work [done]
Rename StreamingObjectRefGenerator and annotate it as a public API (and move it to init.py) <---- this PR
Typing support [future]
This PR renames StreamingObjectRefGenerator - > ObjectRefGenerator. This also means the existing dynamic generator's (num_returns="dynamic") return type will become ObjectRefGenerator -> DynamicObjectRefGenerator, which is a breaking change, but we will anyway deprecate this API. I am open for a different name if we'd like to keep the perfect backward compatibility.
Related issue number
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/
under thecorresponding
.rst
file.