Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System Requirements? Is it possible to run the ToRA-Code-34B on a Windows 10 PC with a 3090ti? #6

Closed
HumanGamer opened this issue Oct 22, 2023 · 1 comment

Comments

@HumanGamer
Copy link

HumanGamer commented Oct 22, 2023

as the title says, Is it possible to run the ToRA-Code-34B on a Windows 10 PC with a 3090ti?

If not, what are the system requirements?

@HumanGamer HumanGamer changed the title Is it possible to run the ToRA-Code-34B on a Windows 10 PC with a 3090ti? System Requirements? Is it possible to run the ToRA-Code-34B on a Windows 10 PC with a 3090ti? Oct 22, 2023
@ZubinGou
Copy link
Contributor

You can perform inference with the 34B model with reduced accuracy using GPTQ (https://huggingface.co/TheBloke/tora-code-34b-v1.0-GPTQ) or GGUF (https://huggingface.co/TheBloke/tora-code-34b-v1.0-GGUF).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants