Access and change models optimizer after setup

[Haydnspass]

Hey,
what is the canonical way to access/change the optimizer of a pl.LightningModule after model.setup('fit') was called?

E.g. if I follow https://pytorch-lightning.readthedocs.io/en/latest/lr_finder.html 4 (the last part where I explicitly plot the found lr and then change it) I don’t see a way how to change the optimizers lr.
If I call model.configure_optimizers() it’ll just return the optimizer and not set them.
Is there no official way to access and change them after construction?

Cheers

[SkafteNicki]

What you probably want to do, is to have an learning_rate attribute that you can change after inspecting the plot:
So your model should look something like this:

class MyModel(pl.LightningModule):
    def __init__(self, ...)
        self.learning_rate = 1e-2

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=self.learning_rate)

then you should be able to do:

model = MyModel(...)
trainer = Trainer(...)
lrfinder = trainer.tuner.lr_finder(...)
lffinder.plot() # find the learning rate you want
model.learning_rate = 1234 # set what you want

[Haydnspass]

Thanks for your suggestion.

Though I find it striking that explicit access to the optimizer is gone once it has been configured. Any reasons for that?

[SkafteNicki]

That is special for the learning rate finder, that the optimizers is destroyed afterwards.
During a standard call to fit you will always be able to access the optimizers in trainer.optimizers