New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

最小显存是否有要求 #5

Open

tocotaco opened this issue May 16, 2023 · 1 comment

tocotaco commented May 16, 2023 •

edited

Loading

运行 chatpdf.py 时报错：

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 12.00 MiB (GPU 0; 5.79 GiB total capacity; 5.05 GiB already allocated; 13.88 MiB free; 5.11 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

GPU :2060 显存6GB
LLM :chatglm-6b-int4
Embedding : text2vec-base

尝试修改batch_size，无果~

Owner

shibing624 commented May 16, 2023

显存12GB,内存16G

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment