Adaptive gradient clipping shouldn't be used in the final classifier layer

[sidml]

Describe the bug
In the paper it is mentioned that AGC is NOT used in the final layer.

Expected behavior
It will be great AGC can be disabled for final layer. Adding a check here may be sufficient to get the desired behavior.
Please let me know if i am misunderstanding something. Thanks.

[vballoli]

Yeah, I did notice that in the paper, but I forgot to take note of it. I'll definitely add it. Thank you!

[vballoli]

Hi, the SGD_AGC implementation kind of restricts the implementation since checking for linear layers makes it unusable for Transformer like architectures, but I could generalize this function in the generic AGC optimizer where I ignore the names of the modules. You can find the code in this commit eaa4b87 and should be availble in the latest release. Do let me know if this works out !