-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Data] Remove BulkExecutor
code path
#40200
Conversation
74fff54
to
5d2ebbb
Compare
Signed-off-by: Scott Lee <sjl@anyscale.com>
Signed-off-by: Andrew Xue <andrewxue@anyscale.com>
5d2ebbb
to
a067674
Compare
Signed-off-by: Andrew Xue <andrewxue@anyscale.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LG, thanks @Zandew and @scottjlee!
from ray.data._internal.execution.legacy_compat import ( | ||
execute_to_legacy_block_list, | ||
) | ||
from ray.data._internal.execution.streaming_executor import ( | ||
StreamingExecutor, | ||
) | ||
|
||
if context.use_streaming_executor: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@scottjlee anything preventing us from removing use_streaming_executor
from DataContext
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i'm not sure, but i think we have several related fields like enable_optimizer as well, that we discussed and decided we want to remove together in a separate PR
Why are these changes needed?
Related issue number
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/
under thecorresponding
.rst
file.