Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tune] change delimiter for results #16573

Merged
merged 6 commits into from Sep 28, 2021
Merged

Conversation

richardliaw
Copy link
Contributor

Why are these changes needed?

We should have a consistent usage of "/" for all dict flattening?

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
@richardliaw
Copy link
Contributor Author

@krfricke lmk what you think about this.

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
"Dataframes will use '/' instead of '.' to delimit "
"nested result keys in future versions of Ray. For forward "
"compatibility, set the environment variable "
"TUNE_RESULT_NEW_DELIM=1.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TUNE_RESULT_DELIM="." instead?

@richardliaw richardliaw marked this pull request as ready for review July 28, 2021 23:53
Comment on lines 544 to 551
def _delimiter(self):
delimiter = os.environ.get("TUNE_RESULT_DELIM", "/")
if delimiter == "/" and log_once("delimiter_deprecation"):
warnings.warn(
"Dataframes will use '/' instead of '.' to delimit "
"nested result keys in future versions of Ray. For forward "
"compatibility, set the environment variable "
"TUNE_RESULT_DELIM='.'")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm confused here, but shouldn't this be the other way around?

I.e. we use

delimiter = os.environ.get("TUNE_RESULT_DELIM", ".")

because it is the current behavior. Then we check

if delimiter == "." and log_once("delimiter_deprecation"):

because the . will be deprecated, and then we suggest


                "nested result keys in future versions of Ray. For forward "
                "compatibility, set the environment variable "
                "TUNE_RESULT_DELIM='/'")

because / will be used in the future?

Copy link
Contributor

@krfricke krfricke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did the changes according to my comments and will merge as current behavior will not be changed. We can then switch to the new delimiter in the 1.9.0 release.

@krfricke krfricke merged commit 227aa9e into ray-project:master Sep 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants