What is the issue?
Environment
System 1 (Working):
GPU: NVIDIA GeForce RTX 4060 Laptop GPU
CPU: Intel i7-14650HX (with Intel integrated GPU)
Ollama Version: 0.6.5
OS: Windows
System 2 (Issue):
GPU: NVIDIA GeForce RTX 5080
CPU: AMD 9900X3D (with AMD integrated GPU)
Ollama Version: 0.9.6
OS: Windows
Problem Description
I’m running the same model (Qwen2.5 3B Instruct) and query on both systems, but observing inconsistent GPU utilization:
On System 1 (RTX 4060 + Intel iGPU), the model loads and runs on the NVIDIA GPU successfully (CUDA acceleration works).
On System 2 (RTX 5080 + AMD iGPU), the model falls back to CPU, even though the NVIDIA GPU is detected.
Key Observations from Logs
System 1 (RTX 4060 + Intel iGPU):
Successfully detects and initializes CUDA:
plaintext
ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Laptop GPU, compute capability 8.9
load_backend: loaded CUDA backend from [path]/ggml-cuda.dll
Model layers are offloaded to GPU:
plaintext
offload library=cuda layers.offload=37
load_tensors: offloaded 37/37 layers to GPU
System 2 (RTX 5080 + AMD iGPU):
Detects AMD iGPU but fails to find ROCm (expected, as it’s an NVIDIA system):
plaintext
amdgpu detected, but no compatible rocm library found. Please install ROCm
unable to verify rocm library: no suitable rocm found, falling back to CPU
No CUDA initialization/backend loading logs (missing ggml_cuda_init or loaded CUDA backend entries).
Model loads entirely on CPU:
plaintext
load_tensors: CPU model buffer size = 1834.82 MiB
(Hypothesis)
The presence of an AMD integrated GPU triggers Ollama’s ROCm detection logic. When ROCm is missing (which is expected on an NVIDIA-focused system), Ollama incorrectly falls back to CPU globally—ignoring the compatible NVIDIA GPU and its CUDA support.
In contrast, Intel integrated GPUs do not trigger this behavior; Ollama skips them and correctly uses the NVIDIA GPU with CUDA.
Request
Could this be a compatibility issue with AMD iGPUs causing Ollama to bypass NVIDIA CUDA acceleration? Any fixes or workarounds to ensure NVIDIA GPUs are prioritized even when AMD iGPUs are present would be appreciated.
Logs for both systems are available for further debugging. Let me know if additional details are needed!
Relevant log output
OS
Windows
GPU
No response
CPU
No response
Ollama version
No response
What is the issue?
Environment
System 1 (Working):
GPU: NVIDIA GeForce RTX 4060 Laptop GPU
CPU: Intel i7-14650HX (with Intel integrated GPU)
Ollama Version: 0.6.5
OS: Windows
System 2 (Issue):
GPU: NVIDIA GeForce RTX 5080
CPU: AMD 9900X3D (with AMD integrated GPU)
Ollama Version: 0.9.6
OS: Windows
Problem Description
I’m running the same model (Qwen2.5 3B Instruct) and query on both systems, but observing inconsistent GPU utilization:
On System 1 (RTX 4060 + Intel iGPU), the model loads and runs on the NVIDIA GPU successfully (CUDA acceleration works).
On System 2 (RTX 5080 + AMD iGPU), the model falls back to CPU, even though the NVIDIA GPU is detected.
Key Observations from Logs
System 1 (RTX 4060 + Intel iGPU):
Successfully detects and initializes CUDA:
plaintext
ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Laptop GPU, compute capability 8.9
load_backend: loaded CUDA backend from [path]/ggml-cuda.dll
Model layers are offloaded to GPU:
plaintext
offload library=cuda layers.offload=37
load_tensors: offloaded 37/37 layers to GPU
System 2 (RTX 5080 + AMD iGPU):
Detects AMD iGPU but fails to find ROCm (expected, as it’s an NVIDIA system):
plaintext
amdgpu detected, but no compatible rocm library found. Please install ROCm
unable to verify rocm library: no suitable rocm found, falling back to CPU
No CUDA initialization/backend loading logs (missing ggml_cuda_init or loaded CUDA backend entries).
Model loads entirely on CPU:
plaintext
load_tensors: CPU model buffer size = 1834.82 MiB
(Hypothesis)
The presence of an AMD integrated GPU triggers Ollama’s ROCm detection logic. When ROCm is missing (which is expected on an NVIDIA-focused system), Ollama incorrectly falls back to CPU globally—ignoring the compatible NVIDIA GPU and its CUDA support.
In contrast, Intel integrated GPUs do not trigger this behavior; Ollama skips them and correctly uses the NVIDIA GPU with CUDA.
Request
Could this be a compatibility issue with AMD iGPUs causing Ollama to bypass NVIDIA CUDA acceleration? Any fixes or workarounds to ensure NVIDIA GPUs are prioritized even when AMD iGPUs are present would be appreciated.
Logs for both systems are available for further debugging. Let me know if additional details are needed!
Relevant log output
OS
Windows
GPU
No response
CPU
No response
Ollama version
No response