-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Description
I have custom build the onnxruntime 1.20.0 with TensorRT, CUDA and OpenVINO. Build is done successfully without error. I have two sessions to initialize. I have tried the following:
Option 1: Initialize both sessions with OpenVINO -> WORKS!
Option 2: Initialize both sessions with TensorRT -> WORKS!
Option 3: Initialize one sessions with CUDA, another with OpenVINO-> WORKS!
Option 4: Initialize one session with OpenVINO, another with TensorRT without including cuda_runtime.h -> WORKS!
Option 5: Initialize one session with OpenVINO, another with TensorRT with including cuda_runtime.h -> Not Work.
I need to include cuda_runtime.h because I have other kernel functions, I need to use.
The error message at option 5 is: OpenVINO "No available backend". Why there are conflict between cuda_runtime.h and OpenVINO provider? It seems, once I load the cuda dlls, there are conflict, and OpenVINO recognize the CPU as NPU (But I don't have NPU on my PC)
My CUDA Version is 12.4, CuDNN Version 9.4, TensorRT 10.5, OpenVINO 2024.4.0.
My operating system is Windows 11. Visual Studio 2022.