Skip to content

Cuda execution provider is not available #23833

Closed as duplicate of#23643
Closed as duplicate of#23643
@thewh1teagle

Description

@thewh1teagle

Cuda execution provider is not available

root@ubuntu:~# mkdir proj
root@ubuntu:~# cd proj
root@ubuntu:~/proj# uv init
Initialized project `proj`
root@ubuntu:~/proj# uv add onnxruntime-gpu onnxruntime
Using CPython 3.12.9
Creating virtual environment at: .venv
Resolved 12 packages in 293ms
Prepared 4 packages in 2.97s
Installed 10 packages in 34ms
 + coloredlogs==15.0.1
 + flatbuffers==25.2.10
 + humanfriendly==10.0
 + mpmath==1.3.0
 + numpy==2.2.3
 + onnxruntime==1.20.1
 + onnxruntime-gpu==1.20.1
 + packaging==24.2
 + protobuf==5.29.3
 + sympy==1.13.3
root@ubuntu:~/proj# uv run python3
Python 3.12.9 (main, Feb 12 2025, 14:50:50) [Clang 19.1.6 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime
>>> onnxruntime.get_available_providers()
['AzureExecutionProvider', 'CPUExecutionProvider']
>>> 
root@ubuntu:~/proj# cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.5 LTS"
root@ubuntu:~/proj# lspci | grep -i nvidia
00:07.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
00:08.0 Audio device: NVIDIA Corporation Device 22ba (rev a1)
root@ubuntu:~/proj# nvidia-smi
Thu Feb 27 05:44:54 2025       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01             Driver Version: 535.183.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4090        Off | 00000000:00:07.0 Off |                  Off |
| 31%   39C    P0              78W / 480W |      0MiB / 24564MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+
root@ubuntu:~/proj# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

Also, I would add warning log if onnxruntime-gpu installed but there's no any GPU found!
Maybe even GPU vendor detection with instructions for the most common distro (eg. Ubuntu) for how to fix the issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ep:CUDAissues related to the CUDA execution provider

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions