Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactoring Trainer, adds save_only_model arg and simplifying FSDP integration #27652

Merged
merged 13 commits into from
Nov 24, 2023
Merged

Refactoring Trainer, adds save_only_model arg and simplifying FSDP integration #27652

merged 13 commits into from
Nov 24, 2023

Commits on Nov 22, 2023

  1. add code changes 

    1. Refactor FSDP
2. Add `--save_only_model` option: When checkpointing, whether to only save the model, or also the optimizer, scheduler & rng state.
3. Bump up the minimum `accelerate` version to `0.21.0`
    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    eea69a6 View commit details
    Browse the repository at this point in the history

  2. quality

    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    4d2a794 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into smangrul/FSDP-refactor

    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    489a5df View commit details
    Browse the repository at this point in the history

  4. fix quality?

    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    149330a View commit details
    Browse the repository at this point in the history

  5. Revert "fix quality?" 

    This reverts commit 149330a.
    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    3dc64c9 View commit details
    Browse the repository at this point in the history

  6. fix fsdp doc strings

    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    9685857 View commit details
    Browse the repository at this point in the history

  7. fix quality

    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    203c38d View commit details
    Browse the repository at this point in the history

  8. Update src/transformers/training_args.py 

    Co-authored-by: Zach Mueller <muellerzr@gmail.com>
    @pacman100 @muellerzr
    pacman100 and muellerzr committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    7e4f99a View commit details
    Browse the repository at this point in the history

  9. please fix the quality issue 😅

    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    bebc321 View commit details
    Browse the repository at this point in the history

  10. Apply suggestions from code review 

    Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
    @pacman100 @BenjaminBossan
    pacman100 and BenjaminBossan committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    835e41b View commit details
    Browse the repository at this point in the history

  11. address comment

    @pacman100
    pacman100 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    4237570 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2023

  1. simplify conditional check as per the comment

    @pacman100
    pacman100 committed Nov 24, 2023
    Configuration menu
    Copy the full SHA
    bdef4ac View commit details
    Browse the repository at this point in the history

  2. update documentation

    @pacman100
    pacman100 committed Nov 24, 2023
    Configuration menu
    Copy the full SHA
    c966e06 View commit details
    Browse the repository at this point in the history