You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Detect CUDA/HIP GPUs (were recorded as "CPU")
llama.cpp's CUDA/HIP backend prints its device to stderr as
" Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes",
which parse_device didn't match (it only handled the Vulkan/Metal
"ggml_xxx: 0 = <name> | ..." form). With no device parsed the runner
fell back to "CPU", so real GPU runs (e.g. an RTX 3090 doing 85 tok/s on
gemma-4-12b Q4) were recorded as CPU — which also made the site derive
"GPU layers: 0".
- parse_device now also reads the "Device N: <name>, ..." banner.
- Added an nvidia-smi fallback: when the banner yields no device but the
run used the GPU (ngl != 0), query nvidia-smi for the card name, so a
GPU run can't be mislabeled CPU even on a build whose banner we don't
recognize.
- Tests for the CUDA layout + both banner shapes.