Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[train] New persistence mode cleanup: Remove legacy air.Checkpoint #40061

Merged
merged 39 commits into from
Oct 5, 2023
Merged

[train] New persistence mode cleanup: Remove legacy air.Checkpoint #40061

merged 39 commits into from
Oct 5, 2023

Conversation

justinvyu
Copy link
Contributor

Why are these changes needed?

This PR removes air.Checkpoint and other legacy framework checkpoints that depend on it.

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@justinvyu
remove old path in base trainer
a9f6548 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove usage in data parallel trainer
0ce34b4 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove in backend exec
3300274 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove unused code in training iterator
9de4fc9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove unused train ckpt manager usage
8fe7e87 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove train._internal.checkpoint
c13021d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
9610db9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
cleanup session
8fbb1ed 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove old TrainingResult
c6ab523 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
finish removing training result
44271be 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove ckpt config as an arg of backend exec
c4176bb 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
8583076 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
07f0154 
…nup_old_codepath
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
c1e1b0f 
…nup_old_codepath
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
45db383 
…nup_old_codepath
@justinvyu
remove deprecated arg
457297e 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
3b24621 
…nup_old_codepath
@justinvyu
fix missing arg
7b7088d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
ac8beee 
…nup_old_codepath

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
0c317cf 
…nup_old_codepath
@justinvyu
remove lazy ckpt constant
9cacfaa 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air.Checkpoint
78655bb 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove legacy air framework ckpts
bc53837 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update horovod trainer docstring
909c330 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove some air.checkpoint usage
9a743a8 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air.checkpoint usage in fn traianble + label some code to remove
528b024 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air integrations from serve
0814cf4 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove air dict ckpt dependency from trainable
1d89c19 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove more ray.air ckpt related imports
6d7f391 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
ea2c853 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove legacy checkpoint remainders
734824c 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
71988db 
…nup_old_codepath2

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
matthewdeng
matthewdeng approved these changes Oct 3, 2023
View reviewed changes
Copy link
Contributor

@matthewdeng matthewdeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this! Thanks for cleaning this up :)

@justinvyu
remove refs to legacy ckpts in notebooks
26cf9b9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
7151479 
…nup_old_codepath2
@justinvyu
fix test_huggingface
c795441 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix hf text classification
f4923cb 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
89f56b8 
…nup_old_codepath2
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
e8b8ce8 
…nup_old_codepath2
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
5e8e058 
…nup_old_codepath2

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu justinvyu merged commit a173890 into ray-project:master Oct 5, 2023
60 of 64 checks passed
@justinvyu justinvyu deleted the cleanup_old_codepath2 branch October 5, 2023 21:25
Zandew pushed a commit to Zandew/ray that referenced this pull request Oct 10, 2023
@justinvyu
[train] New persistence mode cleanup: Remove legacy air.Checkpoint (r…
f5e5f73 
…ay-project#40061)

This PR removes `air.Checkpoint` and other legacy framework checkpoints that depend on it.

---------

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
vymao pushed a commit to vymao/ray that referenced this pull request Oct 11, 2023
@justinvyu
[train] New persistence mode cleanup: Remove legacy air.Checkpoint (r…
c540325 
…ay-project#40061)

This PR removes `air.Checkpoint` and other legacy framework checkpoints that depend on it.

---------

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Victor <vctr.y.m@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewdeng matthewdeng matthewdeng approved these changes

@krfricke krfricke Awaiting requested review from krfricke

@richardliaw richardliaw Awaiting requested review from richardliaw

@xwjiang2010 xwjiang2010 Awaiting requested review from xwjiang2010

@amogkam amogkam Awaiting requested review from amogkam

@Yard1 Yard1 Awaiting requested review from Yard1

@maxpumperla maxpumperla Awaiting requested review from maxpumperla

Assignees

@matthewdeng matthewdeng

@krfricke krfricke

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@justinvyu @matthewdeng @krfricke