-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[data] add flag to skip get_object_locations for metrics #39884
Conversation
Signed-off-by: Andrew Xue <andrewxue@anyscale.com>
Signed-off-by: Andrew Xue <andrewxue@anyscale.com>
Signed-off-by: Andrew Xue <andrewxue@anyscale.com>
python/ray/experimental/locations.py
Outdated
# If this is for purely metrics use, we can skip the call for performance. | ||
if ( | ||
skip_get_locations | ||
and not ray.data.context.DataContext.get_current().enable_get_object_locations_for_metrics |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this function is supposed to be used by all Ray libs, not only Data. So let's not add data-specific logic here.
I guess we don't need to touch this function at all. We can just skip calling it in Data code.
@Zandew @scottjlee btw, do you know why this is causing failures now? what changed? |
It may be the additional calls to |
Signed-off-by: Andrew Xue <andrewxue@anyscale.com>
Signed-off-by: Andrew Xue <andrewxue@anyscale.com>
…#39884) Adds the flag `ENABLE_GET_OBJECT_LOCATIONS_FOR_METRICS` to `DataContext`. If set, this flag skips the `get_object_locations` call if the result is used only for metrics. --------- Signed-off-by: Andrew Xue <andrewxue@anyscale.com> Signed-off-by: Victor <vctr.y.m@example.com>
Why are these changes needed?
Adds the flag
ENABLE_GET_OBJECT_LOCATIONS_FOR_METRICS
toDataContext
.If set, this flag skips the
get_object_locations
call if the result is used only for metrics.Related issue number
Closes #39596
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/
under thecorresponding
.rst
file.