pytorch 1.12.1 Adam Optimizer Malfunction!!!

[DeepFocuser]

If you have a question or would like help and support, please ask at our
forums.

If you are submitting a feature request, please preface the title with [feature request].
If you are submitting a bug report, please fill in the following details.

Issue description

Provide a short description.
In pytorch 1.12.1, Adam optimization doesn't work well.
I think, It seems that the internal behavior has changed as the version is upgraded, please check

Code example

Please try to provide a minimal example to repro the bug.
Error messages and stack traces are also helpful.

System Info

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

• PyTorch
• How you installed PyTorch Conda
• Build command you used (if compiling from source):
• OS: window10, Ubuntu 18.04
• PyTorch version: 1.12.1
• Python version: 3.8.0
• CUDA/cuDNN version: Cuda11.3
• GPU models and configuration: gtx1080ti, gtx3070
• GCC version (if compiling from source):
• CMake version:
• Versions of any other relevant libraries:

cc @vincentqb @jbschlosser @albanD

[mikaylagawarecki]

Hello, do you have a minimal script that we could use to reproduce the behavior you are running into? Thank you!