Warning with PyTorch 1.4

[djstrong]

UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
"https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)

[prajjwal1]

You can send a PR for this, I guess

[djstrong]

I changed it in

adaptive-span/trainer.py

Lines 65 to 67 in d882404

	if scheduler is not None:
	scheduler.step()
	optimizer.step()

but scheduler.step is also called in _load_checkpoint.

[tesatory]

Yes, I've seen this warning, but don't think it matters that much since our warming up takes thousands of steps and shift by one step should be negligible.

[djstrong]

Great, thanks for explanation.