Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Additional Features #11

Closed
SteveImmanuel opened this issue Oct 26, 2022 · 1 comment
Closed

Additional Features #11

SteveImmanuel opened this issue Oct 26, 2022 · 1 comment

Comments

@SteveImmanuel
Copy link

SteveImmanuel commented Oct 26, 2022 •
edited
Loading

Hi, I have some suggestions for features:

  • Can you add initial warmup variant so that the warmup steps only apply on the first cycle like the one shown here?
  • Can you infer the max_lr parameter value from the optimizer base learning rate for each group? Currently, if you have an optimizer with multiple groups with different learning rates, all of their learning rate values will get overridden by the max_lr.

BTW very awesome implementation!
@SteveImmanuel SteveImmanuel changed the title Initial Warmup Variant Additional Features Oct 26, 2022
@katsura-jp
Copy link
Owner

Hi, @SteveImmanuel

Thank you for your suggestions.
It looks so good and I am going to work on your suggestions.

Once I close the issue, I will add a new issue for each suggestion.

Thank you.

This was referenced Oct 28, 2022
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@katsura-jp @SteveImmanuel