Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[train] New persistence mode cleanup: Remove legacy checkpoint management internals #40069

Merged
merged 56 commits into from
Oct 5, 2023
Merged

[train] New persistence mode cleanup: Remove legacy checkpoint management internals #40069

merged 56 commits into from
Oct 5, 2023

Conversation

justinvyu
Copy link
Contributor

@justinvyu justinvyu commented Oct 3, 2023 •
edited
Loading

Why are these changes needed?

This PR removes the unused legacy checkpoint managers (air._internal._CheckpointManager, tune.execution.CheckpointManager). This also involved removing _TrackedCheckpoint dependencies and removing the CheckpointStorage.MEMORY checkpoint type throughout the code.

TODO

In a follow-up PR, add back some of the deleted tests to the train checkpoint manager unit test. (Most of them have already been ported over -- just a NaN test is needed.)

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@justinvyu
remove old path in base trainer
a9f6548 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove usage in data parallel trainer
0ce34b4 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove in backend exec
3300274 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove unused code in training iterator
9de4fc9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove unused train ckpt manager usage
8fe7e87 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove train._internal.checkpoint
c13021d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
9610db9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
cleanup session
8fbb1ed 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove old TrainingResult
c6ab523 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
finish removing training result
44271be 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove ckpt config as an arg of backend exec
c4176bb 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
8583076 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
07f0154 
…nup_old_codepath
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
c1e1b0f 
…nup_old_codepath
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
45db383 
…nup_old_codepath
@justinvyu
remove deprecated arg
457297e 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
3b24621 
…nup_old_codepath
@justinvyu
fix missing arg
7b7088d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
ac8beee 
…nup_old_codepath

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
0c317cf 
…nup_old_codepath
@justinvyu
remove lazy ckpt constant
9cacfaa 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air.Checkpoint
78655bb 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove legacy air framework ckpts
bc53837 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update horovod trainer docstring
909c330 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove some air.checkpoint usage
9a743a8 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air.checkpoint usage in fn traianble + label some code to remove
528b024 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air integrations from serve
0814cf4 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air dict ckpt dependency from trainable
1d89c19 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove more ray.air ckpt related imports
6d7f391 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
ea2c853 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove ckpt storage deps from trial
2e5708e 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove dep in output
7f16696 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove dep in pbt
d78c0a8 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove unused test utils
0eb9a8d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove pbt test that doesn't apply anymore
1481a89 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
clear up pbt + fix mock trial checkpoint
80bd54d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove legacy checkpoint remainders
734824c 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
71988db 
…nup_old_codepath2

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'cleanup_old_codepath2' into cleanup_old_codepath3
6fcbff2
@justinvyu
remove refs to legacy ckpts in notebooks
26cf9b9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove missing tests from build files
fca67c3 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
98a7b28 
…nup_old_codepath3
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
7151479 
…nup_old_codepath2
@justinvyu
fix test_huggingface
c795441 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix hf text classification
f4923cb 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
89f56b8 
…nup_old_codepath2
@justinvyu
Merge branch 'cleanup_old_codepath2' into cleanup_old_codepath3
16b0a4c
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
e8b8ce8 
…nup_old_codepath2
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
5e8e058 
…nup_old_codepath2

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'cleanup_old_codepath2' into cleanup_old_codepath3
3214e4a 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
2d8cfdd 
…nup_old_codepath3

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu justinvyu merged commit 3152779 into ray-project:master Oct 5, 2023
57 of 62 checks passed
@justinvyu justinvyu deleted the cleanup_old_codepath3 branch October 5, 2023 23:43
Zandew pushed a commit to Zandew/ray that referenced this pull request Oct 10, 2023
@justinvyu
[train] New persistence mode cleanup: Remove legacy checkpoint manage…
4a38722 
…ment internals (ray-project#40069)

This PR removes the unused legacy checkpoint managers (`air._internal._CheckpointManager`, `tune.execution.CheckpointManager`). This also involved removing `_TrackedCheckpoint` dependencies and removing the `CheckpointStorage.MEMORY` checkpoint type throughout the code.

---------

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
vymao pushed a commit to vymao/ray that referenced this pull request Oct 11, 2023
@justinvyu
[train] New persistence mode cleanup: Remove legacy checkpoint manage…
8c1c83d 
…ment internals (ray-project#40069)

This PR removes the unused legacy checkpoint managers (`air._internal._CheckpointManager`, `tune.execution.CheckpointManager`). This also involved removing `_TrackedCheckpoint` dependencies and removing the `CheckpointStorage.MEMORY` checkpoint type throughout the code.

---------

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Victor <vctr.y.m@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewdeng matthewdeng matthewdeng approved these changes

@krfricke krfricke Awaiting requested review from krfricke

Assignees

@matthewdeng matthewdeng

@krfricke krfricke

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@justinvyu @matthewdeng @krfricke