This repository was originally created to set up the environment for reproducing the issue discussed in Hugging Face Transformers Issue #32312. The issue has since been resolved, and you should now be able to run everything successfully.
- GPU: RTX 3090
- Python Version: 3.10.14
- PyTorch Version: 2.4.1+cu121
- Transformers Version: 4.45.0.dev0
- DeepSpeed Version: 0.9.3
After running setting up the environment, run
deepseed main.py