Skip to content

Error when I use cuda_runtime.h and OpenVINO EP at the same time #23941

@ddrepkk89

Description

@ddrepkk89

I have custom build the onnxruntime 1.20.0 with TensorRT, CUDA and OpenVINO. Build is done successfully without error. I have two sessions to initialize. I have tried the following:
Option 1: Initialize both sessions with OpenVINO -> WORKS!
Option 2: Initialize both sessions with TensorRT -> WORKS!
Option 3: Initialize one sessions with CUDA, another with OpenVINO-> WORKS!
Option 4: Initialize one session with OpenVINO, another with TensorRT without including cuda_runtime.h -> WORKS!
Option 5: Initialize one session with OpenVINO, another with TensorRT with including cuda_runtime.h -> Not Work.

I need to include cuda_runtime.h because I have other kernel functions, I need to use.

The error message at option 5 is: OpenVINO "No available backend". Why there are conflict between cuda_runtime.h and OpenVINO provider? It seems, once I load the cuda dlls, there are conflict, and OpenVINO recognize the CPU as NPU (But I don't have NPU on my PC)

My CUDA Version is 12.4, CuDNN Version 9.4, TensorRT 10.5, OpenVINO 2024.4.0.

My operating system is Windows 11. Visual Studio 2022.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ep:OpenVINOissues related to OpenVINO execution providerstaleissues that have not been addressed in a while; categorized by a bot

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions