Generate .so shared library for Cuda 10.1

[mtduman]

Hi,
I am using Jetson Nano and able to generate .so shared library for pre-trained and custom yolov4 models and they works perfect. The Cuda version of my Jetson Nano is 10.2.

I am using Nvidia-Deepstream docker with yoloV4/yoloV4-tiny models at Jetson Nano an it works just fine.

Also, I use docker on AWS VM for better performance. The yoloV3/yoloV3-tiny models works without issue, but when I tried yoloV4 model with your solution, it didn't work. I checked my AWS VM Cuda version and it is 10.1. I do not have deepstream installed on VM, so couldn't generated .so shared library for Cuda 10.1

I believe the problem comes from different Cuda version, because docker is running over there without any issue for yoloV3.

If it is a version issue, how could I generate .so lib file for Cuda 10.1 or is there any other solution to get around the issue.

Your help would be appreciated!

[marcoslucianops]

Can you send the output log when you try to run YOLOv4? Are you generating a new engine from model in CUDA 10.1?

[mtduman]

debug1.txt

The file is exist and path are correct.

Thanks!

[marcoslucianops]

Hi, you need to compile nvdsinfer_custom_impl_Yolo for CUDA 10.1 to generate a new libnvdsinfer_custom_impl_Yolo.so

Do make with this command

CUDA_VER=10.1 make -C nvdsinfer_custom_impl_Yolo

You need to generate model.engine for yor CUDA and TensorRT version too.

[mtduman]

Thank you for checking. I tried to generate .so for 10.1, bit didn't work

nano@nano:/opt/nvidia/deepstream/deepstream-5.0/sources/yolo$ CUDA_VER=10.1 make -C nvdsinfer_custom_impl_Yolo
make: Entering directory '/opt/nvidia/deepstream/deepstream-5.0/sources/yolo/nvdsinfer_custom_impl_Yolo'
g++ -o libnvdsinfer_custom_impl_Yolo.so nvdsinfer_yolo_engine.o nvdsparsebbox_Yolo.o yoloPlugins.o layers/convolutional_layer.o layers/dropout_layer.o layers/shortcut_layer.o layers/route_layer.o layers/upsample_layer.o layers/maxpool_layer.o layers/activation_layer.o utils.o yolo.o yoloForward.o -shared -Wl,--start-group -lnvinfer_plugin -lnvinfer -lnvparsers -L/usr/local/cuda-10.1/lib64 -lcudart -lcublas -lstdc++fs -Wl,--end-group
/usr/bin/ld: cannot find -lcudart
collect2: error: ld returned 1 exit status
Makefile:67: recipe for target 'libnvdsinfer_custom_impl_Yolo.so' failed
make: *** [libnvdsinfer_custom_impl_Yolo.so] Error 1
make: Leaving directory '/opt/nvidia/deepstream/deepstream-5.0/sources/yolo/nvdsinfer_custom_impl_Yolo'
nano@nano:/opt/nvidia/deepstream/deepstream-5.0/sources/yolo$

[marcoslucianops]

Add file cuda.conf to the directory /etc/ld.so.conf.d/, with the contents

/usr/local/cuda/lib

Then you run sudo ldconfig.

[mtduman]

nano@nano:/etc/ld.so.conf.d$ sudo vi cuda.conf
[sudo] password for nano:
nano@nano:/etc/ld.so.conf.d$ sudo ldconfig
nano@nano:/etc/ld.so.conf.d$

cuda.conf:
/usr/local/cuda/lib

nano@nano:/opt/nvidia/deepstream/deepstream-5.0/sources/yolo$ CUDA_VER=10.1 make -C nvdsinfer_custom_impl_Yolomake: Entering directory '/opt/nvidia/deepstream/deepstream-5.0/sources/yolo/nvdsinfer_custom_impl_Yolo'
g++ -o libnvdsinfer_custom_impl_Yolo.so nvdsinfer_yolo_engine.o nvdsparsebbox_Yolo.o yoloPlugins.o layers/convolutional_layer.o layers/dropout_layer.o layers/shortcut_layer.o layers/route_layer.o layers/upsample_layer.o layers/maxpool_layer.o layers/activation_layer.o utils.o yolo.o yoloForward.o -shared -Wl,--start-group -lnvinfer_plugin -lnvinfer -lnvparsers -L/usr/local/cuda-10.1/lib64 -lcudart -lcublas -lstdc++fs -Wl,--end-group
/usr/bin/ld: cannot find -lcudart
collect2: error: ld returned 1 exit status
Makefile:67: recipe for target 'libnvdsinfer_custom_impl_Yolo.so' failed
make: *** [libnvdsinfer_custom_impl_Yolo.so] Error 1
make: Leaving directory '/opt/nvidia/deepstream/deepstream-5.0/sources/yolo/nvdsinfer_custom_impl_Yolo'

[marcoslucianops]

I think you have a problem with your CUDA installation, try to reinstall CUDA

[mtduman]

Thanks!

Summary:
I have Cuda 10.2 on my Nano and able run your yoloV4 solution on it including Deepstream docker.

The AWS VM has Cuda 10.1 and Deepstream DSK is not installed there, so I couldn't run your solution at docker.

Tried your cuda.conf solution on my Nano to generate .so file for Cuda 10.1 and it didn't work.

Do you think that I have Cuda installation problem on Jetson Nano (Cuda 10.2) or AWS (Cuda 10.1)?

[marcoslucianops]

You can't generate .so lib for CUDA 10.1 in Nano, because it uses CUDA 10.2. I don't know how AWS VM works, but you need to install CUDA 10.1/10.2 (if not installed yet) and install DeepStream SDK (local/docker) to run the model, compiling .so lib and model engine directly into AWS based in installed CUDA version.