❓ [Question] Can torch_tensorRT be used in C++ with multiprocessing using fork?

[peduajo]

❓ Question

Can torch_tensorRT be used in C++ with multiprocessing using fork?

What you have already tried

I have doubts if this library can be used in C++ multiprocessing (using fork()) where each process loads a TorchScript model compiled for Torch-TensorRT. I have the pipeline that works with no Torch-TensorRT but it fails when I try to load models from it with torch::jit::load (with Torch-TensorRT installed). Related issue: #758. I have not put this as a bug because I have seen in forums that NVIDIA does not recommend using TensorRT with multiprocessing. Mi error is the following on torch::jit::load:

terminate called after throwing an instance of 'torch_tensorrt::Error'
  what():  [Error thrown at /home/eduardo/project/TensorRT/core/runtime/runtime.cpp:99] Expected (cudaGetDevice(reinterpret_cast<int*>(&device)) == cudaSuccess) to be true but got false
Unable to get current device (runtime.get_current_device)

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• PyTorch Version (e.g., 1.0): 2.2.0
• CPU Architecture: amd64
• OS (e.g., Linux): Ubuntu 22.04
• How you installed PyTorch (conda, pip, libtorch, source): libtorch + Torch-TensorRT (source) compiled on tag v2.2.0
• Build command you used (if compiling from source): on tag v2.2.0: cmake -S. -Bbuild -DcuDNN_ROOT_DIR=~/Documents/project/deps/cudnn -DCMAKE_MODULE_PATH=cmake/Modules -DTorch_DIR=/usr/local/libtorch/share/cmake/Torch -DTensorRT_ROOT=~/Documents/TensorRT-8.6.1.6/ -DCMAKE_BUILD_TYPE=Debug
• Are you using local sources or building from archives:
• G++ version: 11.4.0
• CUDA version:12.1
• GPU models and configuration: rtx 4090
• Any other relevant information:

Additional context

Sorry but I am new to C++ and I may have made a mistake somewhere in the compilation or in linking the libraries.