Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[train+tune] Refactor restoration configuration to be centered around storage_path #42853

Merged
merged 37 commits into from
Feb 13, 2024
Merged

[train+tune] Refactor restoration configuration to be centered around storage_path #42853

merged 37 commits into from
Feb 13, 2024

Commits on Jan 30, 2024

  1. update resume config 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    ff6f2af View commit details
    Browse the repository at this point in the history

  2. suppress useless warning driveby 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    62ede46 View commit details
    Browse the repository at this point in the history

  3. pipe resume config through 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    241a8d7 View commit details
    Browse the repository at this point in the history

  4. simplify resume implementation 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    71f624e View commit details
    Browse the repository at this point in the history

  5. resume config not actually needed 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    dbb3ede View commit details
    Browse the repository at this point in the history

  6. update implementation to respect new resume config 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    ba0c70a View commit details
    Browse the repository at this point in the history

  7. pipe resume config from the entrypoint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    67c453a View commit details
    Browse the repository at this point in the history

  8. fix param counting to only consider required positional args 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    743ef95 View commit details
    Browse the repository at this point in the history

  9. remove a duplicate train func validity check 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    b6a8b81 View commit details
    Browse the repository at this point in the history

  10. add test 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    c00f142 View commit details
    Browse the repository at this point in the history

  11. fix lint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    aab6975 View commit details
    Browse the repository at this point in the history

  12. fix legacy config -> resumeconfig translation 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    704201a View commit details
    Browse the repository at this point in the history

  13. allow new config in tune.run_experiments: 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    fc39139 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. support deprecated resume param in tune.run 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    6330eda View commit details
    Browse the repository at this point in the history

  2. fix tune restore tests 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    700cbe1 View commit details
    Browse the repository at this point in the history

  3. fix lint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    f921bc0 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of https://github.com/ray-project/ray into rest… 

    …ore_terminated_trainer
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    91768a4 View commit details
    Browse the repository at this point in the history

  5. Revert "remove a duplicate train func validity check" 

    This reverts commit b6a8b81.

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    fcfd3e7 View commit details
    Browse the repository at this point in the history

  6. fix required param counting for train fn validity check on driver 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    d3a050a View commit details
    Browse the repository at this point in the history

  7. loop through remaining resume str addons 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    fe72616 View commit details
    Browse the repository at this point in the history

  8. fix error -> terminated unintended trial status change 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a30b7fc View commit details
    Browse the repository at this point in the history

  9. remove deprecated usage in tune+rllib test 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    93a47bb View commit details
    Browse the repository at this point in the history

  10. fix lint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    9ff08f8 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'master' of https://github.com/ray-project/ray into rest… 

    …ore_terminated_trainer
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    518885c View commit details
    Browse the repository at this point in the history

  12. fix ckpting integration test 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a123078 View commit details
    Browse the repository at this point in the history

  13. fix callback integration test 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a4f6f0e View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2024

  1. Merge branch 'master' of https://github.com/ray-project/ray into rest… 

    …ore_terminated_trainer

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    ae4819c View commit details
    Browse the repository at this point in the history

  2. rename _ResumeConfig -> ResumeConfig 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    2fb7395 View commit details
    Browse the repository at this point in the history

  3. move ResumeConfig and expose as a developer api 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    3d467a3 View commit details
    Browse the repository at this point in the history

  4. add experimental notice to resume config docstring 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    a2e4359 View commit details
    Browse the repository at this point in the history

  5. fix merge conflict 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    0b6f1d9 View commit details
    Browse the repository at this point in the history

  6. fix tune.run(resume) docstring 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    32a9444 View commit details
    Browse the repository at this point in the history

  7. fix circular import 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    ec6c1c0 View commit details
    Browse the repository at this point in the history

  8. rename IGNORE -> SKIP everywhere 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    72b1194 View commit details
    Browse the repository at this point in the history

  9. don't deprecate tune.run(resume) for now 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    a86c5f2 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'master' of https://github.com/ray-project/ray into rest… 

    …ore_terminated_trainer
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    5b21a07 View commit details
    Browse the repository at this point in the history

  11. Hide resume config in the Tuner API for now to leave it experimental 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    36ef2b2 View commit details
    Browse the repository at this point in the history