[Build] Support CUDA 12 onnxruntime-gpu pypi package

[circuluspibo]

Describe the issue

It seems to support cuda 12 at py3.9 but 3.10 and 3.11 seems to be cuda 11.8

Urgency

No response

Target platform

ubuntu2204

Build script

.

Error / output

.

Visual Studio Version

No response

GCC / Compiler Version

No response

[tianleiwu]

@circuluspibo, could you try install cuda 12 package according to https://onnxruntime.ai/docs/install/

[aashaypm]

Jumping in, @tianleiwu I don't see version 1.18 on here. Anyway, I did follow the method mentioned on the install page, and I get the following error when trying to use the library:

2024-05-22 16:59:06.850388755 [E:onnxruntime:Default, provider_bridge_ort.cc:1744 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/cor
e/session/provider_bridge_ort.cc:1426 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to 
load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory  

Which makes me think the CUDA 11 compatible version was installed even after running pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/

[snnn]

@yihonglyu , please help publish CUDA 12 python & nuget packages there.

[circuluspibo]

@tianleiwu as aashaypm mentioned, only installed 1.18 with cuda 11.8 only....

https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12

If you check this site, only support 1.17.1, can't find 1.18 for cuda 12 (seems to supoort only onnxruntime-training at cuda 12)

[NarutoUA]

#20409

[dbuades]

Same issue here. 1.18 package is missing from https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12

[typon]

Running into the same issue.

For people Googling a fix:

pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple

Will install onnxruntime-gpu==1.18.0 which has libraries built for CUDA 11!

2. To fix this, please install like this:

 pip install --no-cache-dir onnxruntime-gpu==1.17.1 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple

This will allow use with CUDA 12

[snnn]

Should be "-i" instead of "--extra-index-url". You should not have the "extra" there.

python -m pip install  -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/ onnxruntime-gpu

[dbuades]

Update

1.18.0 is now available. Thank you!