Closed as duplicate of#23643
Description
Cuda execution provider is not available
root@ubuntu:~# mkdir proj
root@ubuntu:~# cd proj
root@ubuntu:~/proj# uv init
Initialized project `proj`
root@ubuntu:~/proj# uv add onnxruntime-gpu onnxruntime
Using CPython 3.12.9
Creating virtual environment at: .venv
Resolved 12 packages in 293ms
Prepared 4 packages in 2.97s
Installed 10 packages in 34ms
+ coloredlogs==15.0.1
+ flatbuffers==25.2.10
+ humanfriendly==10.0
+ mpmath==1.3.0
+ numpy==2.2.3
+ onnxruntime==1.20.1
+ onnxruntime-gpu==1.20.1
+ packaging==24.2
+ protobuf==5.29.3
+ sympy==1.13.3
root@ubuntu:~/proj# uv run python3
Python 3.12.9 (main, Feb 12 2025, 14:50:50) [Clang 19.1.6 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime
>>> onnxruntime.get_available_providers()
['AzureExecutionProvider', 'CPUExecutionProvider']
>>>
root@ubuntu:~/proj# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.5 LTS"
root@ubuntu:~/proj# lspci | grep -i nvidia
00:07.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
00:08.0 Audio device: NVIDIA Corporation Device 22ba (rev a1)
root@ubuntu:~/proj# nvidia-smi
Thu Feb 27 05:44:54 2025
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4090 Off | 00000000:00:07.0 Off | Off |
| 31% 39C P0 78W / 480W | 0MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
root@ubuntu:~/proj# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0
Also, I would add warning log if onnxruntime-gpu
installed but there's no any GPU found!
Maybe even GPU vendor detection with instructions for the most common distro (eg. Ubuntu) for how to fix the issue.