Installing the torch2trt with plugins. Nvinfer.h is not existed.

[vujadeyoon]

Dear all,

I manage to install TensorRT using a tar file by referring to the official NVIDIA site and try to install the torch2trt with plugins by referring to the official torch2trt github.
I installed the torch2trt installation without plugins without any problems.
However, I fail to install the torch2trt with plugins. To be more specific, I'm having a problem with a Nvinfer.h.
I think the Nvinfer.h is not existed in my machine.

I installed below packages such as GPU driver, CUDA toolkit, cuDNN and TensorRT by referring to the DL installation and TensorRT-Torch2TRT.
My machine environments are as follows:

• Operating System (OS): Ubuntu MATE 18.04.3 LTS (Bionic)
• Graphics Processing Unit (GPU): NVIDIA TITAN RTX, 4ea-
• GPU driver: Nvidia-440.100
• CUDA toolkit: 10.1 (default), 10.2
• cuDNN: cuDNN v7.6.5
• PyTorch: 1.3.0
• TensorRT: 7.0.0.11
• torch2trt: 0.1.0

The debug information is as below when installing the torch2trt with plugins. In other words I run the command, sudo python setup.py install --plugins.

running install
running bdist_egg
running egg_info
writing torch2trt.egg-info/PKG-INFO
writing dependency_links to torch2trt.egg-info/dependency_links.txt
writing top-level names to torch2trt.egg-info/top_level.txt
reading manifest file 'torch2trt.egg-info/SOURCES.txt'
writing manifest file 'torch2trt.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
building 'plugins' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/aarch64-linux-gnu -I/home/vujadeyoon/.local/lib/python3.6/site-packages/torch/include -I/home/vujadeyoon/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/vujadeyoon/.local/lib/python3.6/site-packages/torch/include/TH -I/home/vujadeyoon/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c torch2trt/plugins/interpolate.cpp -o build/temp.linux-x86_64-3.6/torch2trt/plugins/interpolate.o -DUSE_DEPRECATED_INTLIST -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=plugins -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
torch2trt/plugins/interpolate.cpp:6:10: fatal error: NvInfer.h: No such file or directory
 #include <NvInfer.h>
          ^~~~~~~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

My questions are below:

1. How to install the torch2trt with plugins.
2. Could not the Nvinfer.h be installed when installing TensorRT using a tar file?
3. Please give me any other advice. If you have succeeded in installing the torch2trt with plugins, please give your way for the installation.

Best regards,
Vujadeyoon

[vujadeyoon]

Dear all,

I found a solution related to the problem, non-NvInfer.h.
The installation method related to TensorRT plugin seems to be missing from the official NVIDIA site.
I wrote the installation method and experiments between PyTorch and TensorRT-Torch2TRT are summarized for the customized ResNet-18 including interpolation function in the below github.
Please refer to Sections 3-F and 4-C in the github in order to install the TensorRT correctly.

• Github: https://github.com/vujadeyoon/TensorRT-Torch2TRT

Best,
Vujadeyoon

[ZhangGe6]

I solved this problem by adding include directories and link directories in setup.py as this issue suggests. Specifically, in setup.py

plugins_ext_module = CUDAExtension(
        ...
        include_dirs=[
            trt_inc_dir(),
            'path/to/your/TensorRT/include'       # add include directories
        ],
        library_dirs=[
            trt_lib_dir(),
            'path/to/your/TensorRT/lib'              # add link directories
        ],
        ...
)

As an alternative solution 😄

[AntyRia]

I solved this problem by adding include directories and link directories in setup.py as this issue suggests. Specifically, in setup.py

plugins_ext_module = CUDAExtension(
        ...
        include_dirs=[
            trt_inc_dir(),
            'path/to/your/TensorRT/include'       # add include directories
        ],
        library_dirs=[
            trt_lib_dir(),
            'path/to/your/TensorRT/lib'              # add link directories
        ],
        ...
)

As an alternative solution 😄

thanks ~ it's useful！！！