-
Notifications
You must be signed in to change notification settings - Fork 365
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG:多GPU环境,docker v0.9.2 modelscope源启动bge-reranker-base和bge-reranker-large模型失败 #1132
Comments
调用报错有错误栈吗? |
xinference 日志的错误有没有 |
感觉 dify 的请求没有打过来?dify 的版本是最新的吗 |
dify v0.5.8,在单卡上可以调用rerank模型。测试将qwen 1.5 运行在GPU1上,也可以被正常调用。只有bge-reranker存在这个问题 |
找到原因了,启动rerank模型失败的机器,cuda版本12.2;启动rerank模型成功的机器cuda版本12.1 |
我们镜像应该是基于12.1 的。 |
先关闭,碰到问题可以再开。 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Web UI 显示bge-reranker-base或bge-reranker-large运行在GPU1上
![image](https://private-user-images.githubusercontent.com/45476210/311970062-6eb1eed9-09c5-4f71-98fd-a79df4c810d1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU0NjA2NzAsIm5iZiI6MTcyNTQ2MDM3MCwicGF0aCI6Ii80NTQ3NjIxMC8zMTE5NzAwNjItNmViMWVlZDktMDljNS00ZjcxLTk4ZmQtYTc5ZGY0YzgxMGQxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA0VDE0MzI1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTQ5MWMwYWEyOTY4ZTg2OGMxZjVmYzAzNGQ3ZmYzYTJjZDkzMjZkN2ZhMzkyMGIzMTczZWFiNGU0ZGFjZmU3NDgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.kUXEChbHmuMn3twFgYqB1l-NfpTZFObR5XzSmDptbHU)
但GPU1上无任何模型运行
使用dify验证,模型不能被调用。
但在单GPU环境,rerank模型运行在GPU0上,模型可以正常启动
The text was updated successfully, but these errors were encountered: