How to deploy Triton using Docker on Jetson Orin NX？

[liujiachang]

CUDA：11.4.315
cuDNN：8.6.0.166
TensorRT：8.5.2.2
Jetpack：5.1.2
I tried deploying the 23.06 version image, but it failed.

Is it not possible to deploy using containers?
I tried using another method for deployment, but encountered a lot of issues when installing environment dependencies according to the documentation. I believe that container deployment is a simpler and faster method.
https://github.com/triton-inference-server/server/blob/r23.06/docs/user_guide/jetson.md

[indrajit96]

Hello @liujiachang thanks a lot for reaching out
CC @mc-nv @nv-kmcgill53

[nv-kmcgill53]

Hi @liujiachang, Triton publishes containers for Orin and above on NGC. You can try the latest here:

• nvcr.io/nvidia/tritonserver:24.02-py3-igpu
• nvcr.io/nvidia/tritonserver:24.02-py3-igpu-sdk

[liujiachang]

Thanks!

[liujiachang]

When I use the above environment (Jetson 5.1.2), this image is normal.

Now,
CUDA：10.2.300
cuDNN：8.2.1.32
TensorRT：8.0.1.6
Jetpack：4.6
I try use nvcr.io/nvidia/tritonserver:24.02-py3-igpu, but failed，reason：unable to load shared library libnvcudla.so.
I want to know if he has a version correspondence.