-
-
Notifications
You must be signed in to change notification settings - Fork 729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recent RunPod Axolotl error #1596
Comments
Tried it again today on a 8x H100 SXM
|
A different error for 1xH100 SXM
|
@drummerv I don't know that the " RunPod's Axolotl Jupyter template" is the "official" correct template. This direct link should get you the correct image (https://runpod.io/gsc?template=v2ickqhz9s&ref=6i7fkpdz), although there is a bug w runpod using that link, so for now you can use this link: https://www.runpod.io/console/explore/v2ickqhz9s?ref=6i7fkpdz |
@winglian Fixed it. I just had to rollback to an older commit:
|
thanks, there are quite a few changes since then! Just going to drop this here so I can remember to look through the changeset later: 132eb74...main |
@drummerv If it helps, I also just ran into this This issue on alpaca-lora seems related: tloen/alpaca-lora#174 |
Please check that this issue hasn't been reported before.
Expected Behavior
I ran Axolotl around two days ago and it worked fine. 8xH100 SXM using RunPod's Axolotl Jupyter template.
Current behaviour
When I ran the same config today, it gave me this error:
Steps to reproduce
Config yaml
Possible solution
Does the runpod / docker template use the latest commit? We can narrow it down to the last 1 to 2 days.
Which Operating Systems are you using?
Python Version
main-latest
axolotl branch-commit
main-latest
Acknowledgements
The text was updated successfully, but these errors were encountered: