are predecessor modules weights frozen or not

[TobiasLee]

Hi, thanks for your great work.
According to the paper description, the predecessor module weights are frozen after fine-tuned on the task data( including embedding & output classifier).
The code, however, if my understanding is correct, the fine-tuned predecessor weights are not frozen, instead, the loss can BP to the corresponding parameters.
So, which pattern is supposed to be correct? Thanks in advance.

[JetRunner]

They are frozen. Please check the optimizer. Thanks!

[TobiasLee]

Thanks for the notification.

[Ca0L]

Sorry, I don't see how the optimizer freeze predecessor modules weights. Could you please explain it? Thanks.

[JetRunner]

Sorry, I don't see how the optimizer freeze predecessor modules weights. Could you please explain it? Thanks.

They are not in the optimized parameters of the optimized. PyTorch optimizers can only change the parameters passed to them.