Pytorch SequentialLR cannot be used

[blefaudeux]

🐛 Bug

Using Pytorch's SequentialLR with Lightning leads to AttributeError: 'SequentialLR' object has no attribute 'optimizer' in _validate_scheduler_optimizer (pytorch_lightning/trainer/optimizers.py), which makes sense since SequentialLR does not have a self.optimizer attribute indeed (though on the user side this should not be visible and things should work)

cc @SeanNaren @tchaton, this would be a nice way to repro MinGPTs schedule (warmup then exp down)

To Reproduce

Instantiate torch.optim.lr_scheduler.SequentialLRas your scheduler in Lightning

Expected behavior

Works flawlessly

Environment

Pytorch 1.10

[rohitgr7]

hey @blefaudeux thanks for raising this. seems like we need to support SequentialLR and ChainedScheduler explictly :)

[rohitgr7]

ok seems like it's fixed on PyTorch master for SequentialLR here: pytorch/pytorch#67406
for ChainedScheduler added an issue here: pytorch/pytorch#67601

closing this. feel free to reopen if required :)