Docker image for Jetson Nano

[shahadarsh]

I'm trying to find a docker image that has TensorRT installed for Jetson Nano. Is that available somewhere to be used?

[rmccorm4]

Hi @shahadarsh,

I don't believe we currently have Docker images for ARM/Jetson. I think Jetson's software is typically installed through the Jetpack SDK: https://developer.nvidia.com/embedded/jetpack

[hermanius]

There are a few docker images for jetson based on nvcr.io/nvidia/l4t-base:r32.3.1

I don't know yet about a docker with tensorrt preinstalled.

[make-suffer]

I'm trying to find a docker image that has TensorRT installed for Jetson Nano. Is that available somewhere to be used?

Nvidia is behaving as usual, giving no explanations or coherent documentation.

My solution for using TensorRT-enabled container was just to use nvidia-docker.
I created file /etc/nvidia-container-runtime/host-files-for-container.d/tensorrt.csv
With contents:

lib, /usr/lib/aarch64-linux-gnu/libnvinfer.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvonnxparser_runtime.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvparsers.so.6.0.1
sym, /usr/lib/aarch64-linux-gnu/libnvcaffe_parser.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvcaffe_parser.so.6.0.1
sym, /usr/lib/aarch64-linux-gnu/libnvinfer.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvonnxparser_runtime.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvparsers.so.6
dir, /usr/src/tensorrt
dir, /usr/lib/python3.6/dist-packages/tensorrt/

Assuming, you're using l4t-base container and python 3.6, you're good to go.
You can check it by importing tensorrt in python inside the container. if python isn't your case, you can obviously drop the last line.

[hermanius]

There is some information to install Tensorflow 1.5 or 2.0 on a Nano on Nvidia's forum:
https://devtalk.nvidia.com/default/topic/1048776/jetson-nano/official-tensorflow-for-jetson-nano-1
This can be installed in an l4t docker.

It's not TensorRT, but the Tensoflow distributions Nvidia provides do contain TF-TRT which allows you to do conversions from Tensorflow networks to TensorRT networks (or a mix of the two), and run them almost like TensorRT. It takes some effort to understand TF-TRT, but it works.

@make-suffer I don't know the technique you are using with the host-files-for-container csv. I'm interested to know more how exactly this configurations works and how to run the container.

[make-suffer]

There is some information to install Tensorflow 1.5 or 2.0 on a Nano on Nvidia's forum:
https://devtalk.nvidia.com/default/topic/1048776/jetson-nano/official-tensorflow-for-jetson-nano-1
This can be installed in an l4t docker.

It's not TensorRT, but the Tensoflow distributions Nvidia provides do contain TF-TRT which allows you to do conversions from Tensorflow networks to TensorRT networks (or a mix of the two), and run them almost like TensorRT. It takes some effort to understand TF-TRT, but it works.

@make-suffer I don't know the technique you are using with the host-files-for-container csv. I'm interested to know more how exactly this configurations works and how to run the container.

It's nvidia-docker thing.
There's csv files in /etc/nvidia-container-runtime/host-files-for-container.d/ used to mount some stuff like cuda from the host system. It's installed in the latest Jetpack by default.

[hermanius]

Thanks @make-suffer for the information.
I have replaced the nvidia container with the latest normal docker container (19.03.6) which also support cuda through the --gpus option, but I don't know wether it supports the host-files-for-container technique in some way.
Maybe I need to go back to nvidia-container to use your technique if it is only supported in the nvidia-container.

[Zhou1993napolun]

I'm trying to find a docker image that has TensorRT installed for Jetson Nano. Is that available somewhere to be used?

Nvidia is behaving as usual, giving no explanations or coherent documentation.

My solution for using TensorRT-enabled container was just to use nvidia-docker.
I created file /etc/nvidia-container-runtime/host-files-for-container.d/tensorrt.csv
With contents:

lib, /usr/lib/aarch64-linux-gnu/libnvinfer.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvonnxparser_runtime.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvparsers.so.6.0.1
sym, /usr/lib/aarch64-linux-gnu/libnvcaffe_parser.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvcaffe_parser.so.6.0.1
sym, /usr/lib/aarch64-linux-gnu/libnvinfer.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvonnxparser_runtime.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvparsers.so.6
dir, /usr/src/tensorrt
dir, /usr/lib/python3.6/dist-packages/tensorrt/

Assuming, you're using l4t-base container and python 3.6, you're good to go.
You can check it by importing tensorrt in python inside the container. if python isn't your case, you can obviously drop the last line.

I'm trying to find a docker image that has TensorRT installed for Jetson Nano. Is that available somewhere to be used?

Nvidia is behaving as usual, giving no explanations or coherent documentation.

My solution for using TensorRT-enabled container was just to use nvidia-docker.
I created file /etc/nvidia-container-runtime/host-files-for-container.d/tensorrt.csv
With contents:

lib, /usr/lib/aarch64-linux-gnu/libnvinfer.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvonnxparser_runtime.so.6.0.1
lib, /usr/lib/aarch64-linux-gnu/libnvparsers.so.6.0.1
sym, /usr/lib/aarch64-linux-gnu/libnvcaffe_parser.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvcaffe_parser.so.6.0.1
sym, /usr/lib/aarch64-linux-gnu/libnvinfer.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvonnxparser_runtime.so.6
sym, /usr/lib/aarch64-linux-gnu/libnvparsers.so.6
dir, /usr/src/tensorrt
dir, /usr/lib/python3.6/dist-packages/tensorrt/

Assuming, you're using l4t-base container and python 3.6, you're good to go.
You can check it by importing tensorrt in python inside the container. if python isn't your case, you can obviously drop the last line.

Hello @make-suffer could you give me more explication on that?
you mean i just need add those file on the TX2 and then build a docker image with:
FROM nvcr.io/nvidia/l4t-base:r32.4.2
....
with python3 installed
and then i can run my AI model docker image correct without some issues: such as no tenssorRT libs...
Thanks!

[nibbleshift]

On both my Jetson Nano and Jetson Xavier

I can look in apt to find the available container csv files:

$ apt-cache search nvidia-container-csv
nvidia-container-csv-cuda - Jetpack CUDA CSV file
nvidia-container-csv-cudnn - Jetpack CUDNN CSV file
nvidia-container-csv-tensorrt - Jetpack TensorRT CSV file
nvidia-container-csv-visionworks - Jetpack VisionWorks CSV file

To enable tensorrt in my containers I ran on the host:

apt install nvidia-container-csv-tensorrt

After that installs, all of the libraries are available in my container when i run: docker run --runtime nvidia ....

Hope this helps.