A clean starter project for running image classification through CNN inference on CUDA, using TensorRT.
Compile one of the examples for your platform after cloning this repo:
git clone https://github.com/limitz/cuda-infer-template/
cd cuda-infer-template/examples/basic
make
Follow the readme in the models directory to get the required files.
Run the end result to see the classification in action:
./program
TensorRT is included with the latest JetPack releases.
In order to create some models it might be needed to create a BIG swapfile.
sudo fallocate -l 12G /mnt/swapfile
sudo chmod 600 /mnt/swapfile
sudo mkswap /mnt/swapfile
sudo swapon /mnt/swapfile
To make the swapfile persistent across reboots add the following line to /etc/fstab
/mnt/swapfile swap swap defaults 0 0
Invalid output on Jetson after deserialization of the engine
There is an issue with the prebuilt tensorrt package shipped in JetPack it seems. The solution is to build TensorRT yourself from tags/7.1.3 like so:
cd TensorRT
rm -rf build
rm -rf lib
rm -rf bin
git checkout tags/7.1.3
git reset --hard
git submodule update --init --recursive
mkdir build
cd build
cmake .. -DCUDA_VERSION=10.2 -DGPU_ARCHS="53 62 72"
make -j2
make install
sudo cp -P ../lib/libnv* /usr/lib/aarch64-linux-gnu/
Then make sure to remove any serialized engine in the models directory (ssd.engine) and rebuild the application
cd cuda-infer-template
rm models/ssd.engine
cd examples/basic
make clean
make
Nvidia Jetson and NVJPEG
nvjpeg is not supported (yet) on the Jetson platform, partly because of it's Ubuntu 18.04 limit and GPU limits tied to that. See this forum post for more information on this topic. For jetson the libjpeg library is used, which can be substituted by jpeg-turbo if more performance is needed.
Install TensorRT through Nvidia's instructions.
Make sure to export CUDA_BIN_PATH
for easy access to nvcc
and other tools.
TensorRT is only compatible with dependency CUDA 11.0
and 11.1
. The default CUDA version in Pacman is 11.2
. Make sure this version is not installed to prevent conflicts and install the previous version through AUR:
pamac install cuda11.1
Export bin path to CUDA tools (preferably from .profile
or similar):
export CUDA_BIN_PATH=/opt/cuda/bin
Download the correct tar.gz
archive from the TensorRT download page. You will need an Nvidia Developers account.
At the time of writing you need the 7.2.3.4
version with CUDA 11.1
.
The exact filename is mentioned in the TensorRT
AUR package's PKGBUILD
file.
Create build dir for TensorRT AUR package:
cd /tmp
git clone https://aur.archlinux.org/tensorrt.git
cd tensorrt
Now move the downloaded TensorRT .tar.gz
into this folder:
mv ~/my-downloads/TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.1.cudnn8.1.tar.gz .
Compile the AUR package:
makepkg -si