-
Notifications
You must be signed in to change notification settings - Fork 92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TensorRT version is not compatible, expecting library version 9.2.0.4 got 9.2.0.5 #194
Comments
Can you use Triton server's 23.10 container with v0.6.1 as documented (https://github.com/NVIDIA/TensorRT-LLM/tree/v0.6.1#release-notes) and let us know? |
Sure. I converted the engine file using Triton server's 23.10 container (nvcr.io/nvidia/tritonserver:23.10-py3) with v0.6.1, build.py works fine. However, when I tried to run the model using the newest image nvcr.io/nvidia/tritonserver:23.11-trtllm-python-py3, I encountered the above error. I check the tensorrt version inside nvcr.io/nvidia/tritonserver:23.11-trtllm-python-py3, it uses 9.2.0.4. |
@zTaoplus @wjueyao If you want to use Tritonserver container (nvcr.io/nvidia/tritonserver:23.11-trtllm-python-py3), could you please use the same container to build the engine? The container has TRT 9.2.0.4 installed, which will not be working with TRT 9.2.0.5 used in v0.6.1. If you want to use the engine built by v0.6.1, please try deploy the model using the container built by https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/dockerfile/Dockerfile.trt_llm_backend Thanks. |
nvcr.io/nvidia/tritonserver:23.11-trtllm-python-py3 use trt 9.2.0.4 run tritonserver error reports:
when I build the TensorRT-LLM how can I get 9.2.0.4version, I Pass 9.2.0.4 as trt_ver but it cant download.
|
@kaiyux help,hao can I get the right url of special version:9.2.0.4, or could you support 9.2.0.5 in tritonserver |
Building a new container using https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/dockerfile/Dockerfile.trt_llm_backend works for me. But still it would be nice to have nvidia pre-built container for 23.11 with 9.2.5 |
@FightingMan Please use If you want to use TensorRT 9.2.0.5 in tritonserver container, please use Dockerfile.trt_llm_backend to bulid the container and do the deployment. Thanks. |
I encountered the same issue. However ,when using the nvcr.io/nvidia/tritonserver:23.12-trtllm-python-py3 container, the issue does not occur. |
Do we need to re-build TensorRT-LLM in |
I ended up using Dockerfile.trt_llm_backend as suggested and it worked. Seems to be TensorRT 9.2.0.5 and the release of TensorRT-LLM also lists this version number install_tensorrt.sh. Will a future version of |
@shannonphu You are right about the versions for trtllm_backend vs trtllm, thanks for the patience on that. The latest image |
I'm closing this issue, please feel free to leave comments or open new issues if you're still seeing problems. Thanks. |
I try run example with Is there an option to not build a package, but use ready pre-build package like |
I encountered the following error when using the newest image nvcr.io/nvidia/tritonserver:23.11-trtllm-python-py3
The engine plan file is not compatible with this version of TensorRT, expecting library version 9.2.0.4 got 9.2.0.5, please rebuild
I followed the exact steps listed on this issue to build model files. Instead of using main branch, I used v0.6.1.#128 (comment)
And I checked the tensorrt version in the install_tensorrt.sh, it did use 9.2.0.5
https://github.com/NVIDIA/TensorRT-LLM/blob/v0.6.1/docker/common/install_tensorrt.sh
Therefore, I was wondering which one should be the correct version to use
The text was updated successfully, but these errors were encountered: