onnxruntime no longer /delayload CUDA dlls when compiled for Windows

[nietras]

Previously, the onnxruntime.dll would delayload all cuda dlls when building with CUDA, this behavior appears to unfortunately have changed, can I change this to actually delayload or could there be a separate onnxruntime_providers_cuda.dll like for tensorrt?

Based on the latest commit 4df356d

[pranavsharma]

We're working on a feature to enable this on all platforms (Windows, Linux and Mac). Stay tuned! cc @RyanUnderhill

[snnn]

For reference: https://devblogs.microsoft.com/oldnewthing/20190718-00/?p=102719

We encountered the same issue.

[nietras]

@pranavsharma thanks, do you have an approximate time frame for this? We have to update onnx runtime to get support for Ampere, without it we can run on that.

If there is anything that can be done to help let us know. :)

[pranavsharma]

We don't have an ETA yet, but rest assured that this item is very much on our roadmap and we're actively working on it as we speak.

[nietras]

@pranavsharma any issue(s) tracking this?

[pranavsharma]

We can this issue to track. As such we're tracking it internally.

[tjjack1226]

any updates on this issue? we'd like to run the same version of onnxruntime on machines both with and without CUDA installed.
I've just tried building onnxruntime version 1.7.2 from source and it unfortunately attempts to link to CUDA libraries right away. I've even tried editing the cmake file to add /DELAYLOAD commands for each of the CUDA libraries, while it works for cudnn, cudfft, and cudart, and it doesn't seem to "stick" for cublas

[pranavsharma]

Starting 1.8.1 you can use onnxruntime's GPU pkg on CPU-only machines as well (i.e. where CUDA is not installed or no GPU is present).