Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updates warmup_cosine_decay_schedule to allow 0 as peak_value. Currently it errors out as divide by 0. #770

Merged
merged 1 commit into from
Feb 5, 2024
Merged

Updates warmup_cosine_decay_schedule to allow 0 as peak_value. Currently it errors out as divide by 0. #770

merged 1 commit into from
Feb 5, 2024

Commits on Feb 5, 2024

  1. Updates warmup_cosine_decay_schedule to allow 0 as peak_value. Curren… 

    …tly it errors out as divide by 0.

The need for peak_value to be 0 is when we want to turn off training for certain portions of the network by setting the learning rate to be 0.

PiperOrigin-RevId: 604272302
    OptaxDev committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    c902205 View commit details
    Browse the repository at this point in the history