Skip to content

v0.3.2

Choose a tag to compare

@github-actions github-actions released this 28 Jun 09:05
· 56 commits to main since this release
77cade2
Detect CUDA/HIP GPUs (were recorded as "CPU")

llama.cpp's CUDA/HIP backend prints its device to stderr as
"  Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes",
which parse_device didn't match (it only handled the Vulkan/Metal
"ggml_xxx: 0 = <name> | ..." form). With no device parsed the runner
fell back to "CPU", so real GPU runs (e.g. an RTX 3090 doing 85 tok/s on
gemma-4-12b Q4) were recorded as CPU — which also made the site derive
"GPU layers: 0".

- parse_device now also reads the "Device N: <name>, ..." banner.
- Added an nvidia-smi fallback: when the banner yields no device but the
  run used the GPU (ngl != 0), query nvidia-smi for the card name, so a
  GPU run can't be mislabeled CPU even on a build whose banner we don't
  recognize.
- Tests for the CUDA layout + both banner shapes.