-
Notifications
You must be signed in to change notification settings - Fork 271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How much GPU memory to run inference.py? #17
Comments
24GB vram was suggested Edit: |
So there is no difference between inference.py and demo.py for GPU memory requirement? How to resize batch_size so it can run on the small amount of GPU memory, the tradeoff is a little bit longer to process? |
Hi,
in demo.py, the BTW, I am trying FP16 inference and it seems to have little difference in performance. It is possible that the GPU memory requirement can be reduced to below 10GB. I will push the updated code later. |
Cool I cannot wait to see the update 😉 |
Thank you @tyxsspa , I fetch the FP16 commit, and now I success to run demo.py on my 8GB GPU. 🥰🙏 |
I tried to run inference, but failed 'CUDA out of memory', I'm using 8GB RTX 3080 TI, does it work?
The text was updated successfully, but these errors were encountered: