the tensorrtllm backends and onnxruntime backends

[tricky61]

I have install this image: tritonserver-23.10-trtllm-python-py3
but the backends only has tensorrtllm and python, Can I use previous backends like: onnxruntime、pytorch、tensorrt...? how to add them

[byshiue]

For specific backend docker image, it only installs the corresponding backend to save the memory. If you want to install several backends in one docker image, you could refer the document of triton backend about building docker image. Hope this document https://github.com/triton-inference-server/backend?tab=readme-ov-file#can-i-add-or-remove-a-backend-to-an-existing-triton-installation is helpful.

[tricky61]

For specific backend docker image, it only installs the corresponding backend to save the memory. If you want to install several backends in one docker image, you could refer the document of triton backend about building docker image. Hope this document https://github.com/triton-inference-server/backend?tab=readme-ov-file#can-i-add-or-remove-a-backend-to-an-existing-triton-installation is helpful.

Thanks for your reply