Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[train] New persistence mode cleanup: Train internals #39921

Merged
merged 21 commits into from
Oct 3, 2023
Merged

[train] New persistence mode cleanup: Train internals #39921

merged 21 commits into from
Oct 3, 2023

Conversation

justinvyu
Copy link
Contributor

@justinvyu justinvyu commented Sep 27, 2023 •
edited
Loading

Why are these changes needed?

This PR cleans up some train internals to remove the old codepath:

  • Removes old train checkpoint managers
  • Removes lazy checkpointing remnants
  • Cleans up TrainingIterator, BackendExecutor, parts of _TrainSession, and BaseTrainer

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@justinvyu
remove old path in base trainer
a9f6548 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove usage in data parallel trainer
0ce34b4 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove in backend exec
3300274 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove unused code in training iterator
9de4fc9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove unused train ckpt manager usage
8fe7e87 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove train._internal.checkpoint
c13021d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
9610db9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
cleanup session
8fbb1ed 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove old TrainingResult
c6ab523 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
finish removing training result
44271be 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove ckpt config as an arg of backend exec
c4176bb 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
8583076 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
07f0154 
…nup_old_codepath
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
c1e1b0f 
…nup_old_codepath
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
45db383 
…nup_old_codepath
@justinvyu
remove deprecated arg
457297e 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
3b24621 
…nup_old_codepath
@justinvyu
fix missing arg
7b7088d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
ac8beee 
…nup_old_codepath

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu justinvyu marked this pull request as ready for review October 2, 2023 20:45
krfricke
krfricke approved these changes Oct 3, 2023
View reviewed changes
Copy link
Contributor

@krfricke krfricke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic!

@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into clea…
0c317cf 
…nup_old_codepath
@justinvyu
remove lazy ckpt constant
9cacfaa 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu justinvyu merged commit 72ed74f into ray-project:master Oct 3, 2023
58 of 65 checks passed
@justinvyu justinvyu deleted the cleanup_old_codepath branch October 3, 2023 20:45
vymao pushed a commit to vymao/ray that referenced this pull request Oct 11, 2023
@justinvyu
[train] New persistence mode cleanup: Train internals (ray-project#39921
0cd37fe 
)

This PR cleans up some train internals to remove the old codepath:
* Removes old train checkpoint managers
* Removes lazy checkpointing remnants
* Cleans up `TrainingIterator`, `BackendExecutor`, parts of `_TrainSession`, and `BaseTrainer`

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Victor <vctr.y.m@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@krfricke krfricke krfricke approved these changes

@matthewdeng matthewdeng Awaiting requested review from matthewdeng matthewdeng is a code owner

Assignees

@matthewdeng matthewdeng

@krfricke krfricke

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@justinvyu @krfricke @matthewdeng