Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug/feature: Fixing mixed precision training #290

Open
wants to merge 69 commits into
base: main
Choose a base branch
from
Open

bug/feature: Fixing mixed precision training #290

wants to merge 69 commits into from

Commits on Feb 16, 2024

  1. Adding grad scaler

    @isamu-isozaki
    isamu-isozaki committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e8b084c View commit details
    Browse the repository at this point in the history

  2. Fixed double backward

    @isamu-isozaki
    isamu-isozaki committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    dc795ff View commit details
    Browse the repository at this point in the history

  3. zero grad

    @isamu-isozaki
    isamu-isozaki committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    895c65f View commit details
    Browse the repository at this point in the history

  4. Merge branch 'grad_scaler' of https://github.com/isamu-isozaki/refiners… 

    … into grad_scaler
    @isamu-isozaki
    isamu-isozaki committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    6049d40 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. Fixed accordign to review

    @isamu-isozaki
    isamu-isozaki committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    7ccb836 View commit details
    Browse the repository at this point in the history

  2. Fixed styles

    @isamu-isozaki
    isamu-isozaki committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    5e539c1 View commit details
    Browse the repository at this point in the history

  3. Cleaned up code

    @isamu-isozaki
    isamu-isozaki committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    35401da View commit details
    Browse the repository at this point in the history

  4. Removed diffs

    @isamu-isozaki
    isamu-isozaki committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    4ccca8d View commit details
    Browse the repository at this point in the history

  5. Disable grad scaler for bf16

    @isamu-isozaki
    isamu-isozaki committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    3c7f8f3 View commit details
    Browse the repository at this point in the history

  6. Fixed mixed precision training

    @isamu-isozaki
    isamu-isozaki committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    eebaad7 View commit details
    Browse the repository at this point in the history

  7. Added option for None as dtype for mixed precision training

    @isamu-isozaki
    isamu-isozaki committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    d362686 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. Remove dtype

    @isamu-isozaki
    isamu-isozaki committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    6e5bb46 View commit details
    Browse the repository at this point in the history

  2. Testing with no api change

    @isamu-isozaki
    isamu-isozaki committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    e2291b9 View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. Fixed typo

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    a45c9a4 View commit details
    Browse the repository at this point in the history

  2. Amp config

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    65c4195 View commit details
    Browse the repository at this point in the history

  3. Style fixes

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    ba438aa View commit details
    Browse the repository at this point in the history

  4. Attempt file fix

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    88f002c View commit details
    Browse the repository at this point in the history

  5. Remove diff

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    e9d857d View commit details
    Browse the repository at this point in the history

  6. Remove diffs

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    0803b86 View commit details
    Browse the repository at this point in the history

  7. Fixed always true condition

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    ca4ad1b View commit details
    Browse the repository at this point in the history

  8. Removed not implemented error

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    a3674e4 View commit details
    Browse the repository at this point in the history

  9. Remove comment

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    f0bc382 View commit details
    Browse the repository at this point in the history

  10. Remove overflow grad check

    @isamu-isozaki
    isamu-isozaki committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    59cf8ab View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. Removed grad scaler for non-amp training

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    73a4692 View commit details
    Browse the repository at this point in the history

  2. Remove comments

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    ab00953 View commit details
    Browse the repository at this point in the history

  3. Cleaner amp

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    903fbfe View commit details
    Browse the repository at this point in the history

  4. Fix default

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    e289d64 View commit details
    Browse the repository at this point in the history

  5. More explicit name

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    2959c30 View commit details
    Browse the repository at this point in the history

  6. Remove accelerate

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    362bd2d View commit details
    Browse the repository at this point in the history

  7. Linting

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    2d64832 View commit details
    Browse the repository at this point in the history

  8. Adding grad scaler

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    05e77b6 View commit details
    Browse the repository at this point in the history

  9. zero grad

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    be99c30 View commit details
    Browse the repository at this point in the history

  10. Fixed double backward

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    c121dfb View commit details
    Browse the repository at this point in the history

  11. Fixed accordign to review

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    b4da3ef View commit details
    Browse the repository at this point in the history

  12. Fixed styles

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    aa6f0a4 View commit details
    Browse the repository at this point in the history

  13. Cleaned up code

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    e227f0d View commit details
    Browse the repository at this point in the history

  14. Removed diffs

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    e1f309d View commit details
    Browse the repository at this point in the history

  15. Disable grad scaler for bf16

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    ae08009 View commit details
    Browse the repository at this point in the history

  16. Fixed mixed precision training

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    ce364e1 View commit details
    Browse the repository at this point in the history

  17. Added option for None as dtype for mixed precision training

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    5dc798a View commit details
    Browse the repository at this point in the history

  18. Remove dtype

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    9876caf View commit details
    Browse the repository at this point in the history

  19. Testing with no api change

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    b547ee1 View commit details
    Browse the repository at this point in the history

  20. Fixed typo

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    d78525d View commit details
    Browse the repository at this point in the history

  21. Amp config

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    a82daf4 View commit details
    Browse the repository at this point in the history

  22. Style fixes

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    1cf64e3 View commit details
    Browse the repository at this point in the history

  23. Attempt file fix

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    bc20875 View commit details
    Browse the repository at this point in the history

  24. Remove diff

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    691aeb5 View commit details
    Browse the repository at this point in the history

  25. Remove diffs

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    dce3cd4 View commit details
    Browse the repository at this point in the history

  26. Fixed always true condition

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    486de2a View commit details
    Browse the repository at this point in the history

  27. Removed not implemented error

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    9e7b65a View commit details
    Browse the repository at this point in the history

  28. Remove comment

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    5d3cd51 View commit details
    Browse the repository at this point in the history

  29. Remove overflow grad check

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    acba9cf View commit details
    Browse the repository at this point in the history

  30. Removed grad scaler for non-amp training

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    78e3a52 View commit details
    Browse the repository at this point in the history

  31. Remove comments

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    03f4352 View commit details
    Browse the repository at this point in the history

  32. Cleaner amp

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    3ed2c29 View commit details
    Browse the repository at this point in the history

  33. Fix default

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    4df3741 View commit details
    Browse the repository at this point in the history

  34. More explicit name

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    123b4d4 View commit details
    Browse the repository at this point in the history

  35. Remove accelerate

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    72e8216 View commit details
    Browse the repository at this point in the history

  36. Linting

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    e206b7a View commit details
    Browse the repository at this point in the history

  37. Merge branch 'grad_scaler' of https://github.com/isamu-isozaki/refiners… 

    … into grad_scaler
    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    07185be View commit details
    Browse the repository at this point in the history

  38. Merge branch 'main' into grad_scaler

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    7c7c3ff View commit details
    Browse the repository at this point in the history

  39. Merge branch 'grad_scaler' of https://github.com/isamu-isozaki/refiners… 

    … into grad_scaler
    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    8a9e034 View commit details
    Browse the repository at this point in the history

  40. Remove diffs

    @isamu-isozaki
    isamu-isozaki committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    a5ff578 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. Did fixes

    @isamu-isozaki
    isamu-isozaki committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    da188f7 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. Fixed import

    @isamu-isozaki
    isamu-isozaki committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    49ebf48 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. Resolve merge conflicts

    @isamu-isozaki
    isamu-isozaki committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    58fcb15 View commit details
    Browse the repository at this point in the history

  2. Delete double optimizer step

    @isamu-isozaki
    isamu-isozaki committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    1d648a2 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Fixed dtype

    @isamu-isozaki
    isamu-isozaki committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    4efdffb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into grad_scaler

    @isamu-isozaki
    isamu-isozaki committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    27cbdba View commit details
    Browse the repository at this point in the history