Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[train] New persistence mode cleanup: Remove legacy air.Checkpoint #40061

Merged
merged 39 commits into from
Oct 5, 2023
Merged

[train] New persistence mode cleanup: Remove legacy air.Checkpoint #40061

merged 39 commits into from
Oct 5, 2023

Commits on Sep 27, 2023

  1. remove old path in base trainer 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    a9f6548 View commit details
    Browse the repository at this point in the history

  2. remove usage in data parallel trainer 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    0ce34b4 View commit details
    Browse the repository at this point in the history

  3. remove in backend exec 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    3300274 View commit details
    Browse the repository at this point in the history

  4. remove unused code in training iterator 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    9de4fc9 View commit details
    Browse the repository at this point in the history

  5. remove unused train ckpt manager usage 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    8fe7e87 View commit details
    Browse the repository at this point in the history

  6. remove train._internal.checkpoint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    c13021d View commit details
    Browse the repository at this point in the history

  7. fix lint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    9610db9 View commit details
    Browse the repository at this point in the history

  8. cleanup session 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    8fbb1ed View commit details
    Browse the repository at this point in the history

  9. remove old TrainingResult 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    c6ab523 View commit details
    Browse the repository at this point in the history

  10. finish removing training result 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    44271be View commit details
    Browse the repository at this point in the history

  11. remove ckpt config as an arg of backend exec 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    c4176bb View commit details
    Browse the repository at this point in the history

  12. fix lint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    8583076 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath
    @justinvyu
    justinvyu committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    07f0154 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath
    @justinvyu
    justinvyu committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    c1e1b0f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath
    @justinvyu
    justinvyu committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    45db383 View commit details
    Browse the repository at this point in the history

  3. remove deprecated arg 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    457297e View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath
    @justinvyu
    justinvyu committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    3b24621 View commit details
    Browse the repository at this point in the history

  2. fix missing arg 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    7b7088d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    ac8beee View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    0c317cf View commit details
    Browse the repository at this point in the history

  2. remove lazy ckpt constant 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    9cacfaa View commit details
    Browse the repository at this point in the history

  3. remove air.Checkpoint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    78655bb View commit details
    Browse the repository at this point in the history

  4. remove legacy air framework ckpts 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    bc53837 View commit details
    Browse the repository at this point in the history

  5. update horovod trainer docstring 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    909c330 View commit details
    Browse the repository at this point in the history

  6. remove some air.checkpoint usage 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    9a743a8 View commit details
    Browse the repository at this point in the history

  7. remove air.checkpoint usage in fn traianble + label some code to remove 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    528b024 View commit details
    Browse the repository at this point in the history

  8. remove air integrations from serve 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    0814cf4 View commit details
    Browse the repository at this point in the history

  9. remove air dict ckpt dependency from trainable 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    1d89c19 View commit details
    Browse the repository at this point in the history

  10. remove more ray.air ckpt related imports 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    6d7f391 View commit details
    Browse the repository at this point in the history

  11. fix lint 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    ea2c853 View commit details
    Browse the repository at this point in the history

  12. remove legacy checkpoint remainders 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    734824c View commit details
    Browse the repository at this point in the history

  13. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath2

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    71988db View commit details
    Browse the repository at this point in the history

  14. remove refs to legacy ckpts in notebooks 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    26cf9b9 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath2
    @justinvyu
    justinvyu committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    7151479 View commit details
    Browse the repository at this point in the history

  2. fix test_huggingface 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    c795441 View commit details
    Browse the repository at this point in the history

  3. fix hf text classification 

    Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    f4923cb View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath2
    @justinvyu
    justinvyu committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    89f56b8 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath2
    @justinvyu
    justinvyu committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    e8b8ce8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ray-project/ray into clea… 

    …nup_old_codepath2

Signed-off-by: Justin Yu <justinvyu@anyscale.com>
    @justinvyu
    justinvyu committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    5e8e058 View commit details
    Browse the repository at this point in the history