Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

在跑github下载已经转好的qwen-vl-chat-combine.bmodel模型时,会提示内存不足 #37

Open
xuyang1102 opened this issue Aug 1, 2024 · 3 comments

Comments

@xuyang1102
Copy link

用bmrt_test --bmodel 测试模型时发现的这个问题
IMG_20240801_222640
IMG_20240801_222701
IMG_20240801_222721
IMG_20240801_230835

@chuxiaoyi2023
Copy link
Collaborator

可以试试在bmrt_test的时候,观察bm-smi的变动情况,如果内存逐渐增加,并且到11711MB左右时快爆了,那说明可能需要调大内存

参考这个
https://github.com/sophgo/LLM-TPU/blob/main/docs/FAQ.md

Q10这个问题

./memory_edit.sh -c -npu 7168 -vpu 3072 -vpp 4096

@xuyang1102
Copy link
Author

但目前看到的是,显存到了8.6G左右时,就报这个错误了,还没有达到我设置的11G

@xuyang1102
Copy link
Author

如果设置的11G,是NPU可用的总共11G吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants