[Bug] lmdeploy 部署internvl-38这种，张量并行使用3或者6这种，是否建议？效果和2卡部署有差别吗（比如并发，显存占用，调用速度）？是不是卡数却多性能越强（比如并发，显存占用，调用速度）？

[lmingze]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

lmdeploy 部署internvl-38这种，张量并行使用3或者6这种，是否建议？效果和2卡部署有差别吗（比如并发，显存占用，调用速度）？是不是卡数却多性能越强（比如并发，显存占用，调用速度）？

Reproduction

lmdeploy 部署internvl-38这种，张量并行使用3或者6这种，是否建议？效果和2卡部署有差别吗（比如并发，显存占用，调用速度）？是不是卡数却多性能越强（比如并发，显存占用，调用速度）？

Environment

无

Error traceback

[lvhan028]

使用 tp 时，需要保证 kv_head_num, attention_head_num 是 tp 的整数倍。

不一定 tp 越大越好，因为卡间通信也有开销。最好是自己根据实际场景做些消融实验看看

[lmingze]

使用 tp 时，需要保证 kv_head_num, attention_head_num 是 tp 的整数倍。

不一定 tp 越大越好，因为卡间通信也有开销。最好是自己根据实际场景做些消融实验看看

谢谢！请问这个两个参数应该去哪里找到？

[lvhan028]

模型的 config.json 文件

[github-actions]

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

[github-actions]

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.