-
-
Notifications
You must be signed in to change notification settings - Fork 605
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
WarmRestarts seems not working with create_lr_scheduler_with_warmup function #2910
Comments
import torch
import torchvision
import ignite
from ignite.handlers.param_scheduler import create_lr_scheduler_with_warmup
import matplotlib.pyplot as plt
import numpy as np
model = torchvision.models.resnet18()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
scheduler = create_lr_scheduler_with_warmup(torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer,
T_0=10,
T_mult=1,
verbose=False),
warmup_start_value=1e-10,
warmup_end_value=1e-3,
warmup_duration=10)
lr = []
for epoch in range(100):
scheduler(None)
lr.append(optimizer.param_groups[0]["lr"])
plt.plot(np.arange(len(lr)), lr, label="lr") |
@developer0hye, thank you for reporting this bug. This occurs because we do not call |
This is related to #813 (comment) |
Hey @developer0hye we just fixed this issue and after midnight this fix will be available on pytorch-ignite nightly:
if you would like to give it a try. It works as reported in the PR: #2938 (comment) |
@vfdev-5 Thanks! |
馃悰 Bug description
Environment
5.10.147+#1
SMP Sat Dec 10 16:00:40 UTC 2022 x86_64 x86_64 x86_64 GNU/Linuxconda
,pip
, source):pip
The text was updated successfully, but these errors were encountered: