Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bring back global gradient clipping and improve speed of collecting metrics #326

Merged
merged 11 commits into from
Oct 12, 2023
Merged

Bring back global gradient clipping and improve speed of collecting metrics #326

merged 11 commits into from
Oct 12, 2023

Commits on Oct 11, 2023

  1. Bring back global gradient clipping

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    6578709 View commit details
    Browse the repository at this point in the history

  2. fix device type

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    4f5d3b3 View commit details
    Browse the repository at this point in the history

  3. collect metrics on GPU

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    e342d88 View commit details
    Browse the repository at this point in the history

  4. fix device type of numel tensor

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    d8031f5 View commit details
    Browse the repository at this point in the history

  5. separate clipping strategies to methods

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    3327591 View commit details
    Browse the repository at this point in the history

  6. clean up

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    07a9cab View commit details
    Browse the repository at this point in the history

  7. wait longer for host-device sync of loss

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    e66f8d7 View commit details
    Browse the repository at this point in the history

  8. overlap z-batch-loss reduction too

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    938c3c8 View commit details
    Browse the repository at this point in the history

  9. fix typo in comment

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    b476200 View commit details
    Browse the repository at this point in the history

  10. add more no_grads for future proofing

    @epwalsh
    epwalsh committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    ec8db60 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. clean up

    @epwalsh
    epwalsh committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    1b1ee91 View commit details
    Browse the repository at this point in the history