Error with PyTorch 2.3.0: Missing '_refresh_per_optimizer_state' in 'torch.cuda.amp.grad_scaler' #576
Comments
Priyanshupareek
added a commit
to Priyanshupareek/petals
that referenced
this issue
Apr 25, 2024
Addressing the import error encountered with PyTorch 2.3.0 as detailed in issue bigscience-workshop#576. fixes bigscience-workshop#576
This was referenced Apr 27, 2024
mryab
pushed a commit
that referenced
this issue
Apr 27, 2024
* Pin PyTorch version to 2.2.2 to resolve import error Addressing the import error encountered with PyTorch 2.3.0 as detailed in issue #576. fixes #576 * Update setup.cfg Modified the version constraint for PyTorch in setup.cfg to torch>=1.12,<2.3.0 to avoid the import errors introduced in version 2.3.0 while still supporting earlier compatible versions. This change follows feedback from @mryab to allow flexibility for users on different versions.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem Description
After the recent update to PyTorch 2.3.0, petals encounters an import error when using the
torch.cuda.amp.grad_scalermodule. The specific error message is:ImportError: cannot import name '_refresh_per_optimizer_state' from 'torch.cuda.amp.grad_scaler' site-packages/torch/cuda/amp/grad_scaler.py)The issue is due to changes in the new PyTorch version that are currently incompatible with current codebase.
Quick Workaround
Resolve this issue, by simply reverting the PyTorch version specified in
setup.cfgfrom 'torch>=1.12' to 'torch==2.2.2', which is the last version known to work without this problem. Just so it works stably while I investigate changes in new pytorch and make the codebase to be compatible with PyTorch 2.3.0 or later.Steps to Reproduce
Information
Action
Submitting a pull request to modifying the
install_requiresin oursetup.cfgas described above, pending team feedback on this issue.The text was updated successfully, but these errors were encountered: