v0.6.4

whichllm 0.6.4

A patch for 0.6.3. Once the index was rebuilt with 0.6.3's lookup-table fix, I ran whichllm on my own PC (RTX 5080 16 GB, 39 GB of RAM under WSL), and Qwen3.8-Flash-Next still came out wrong. It sat 11th through an MLX repo ranked as FP16 safetensors, which nothing on that machine can run, and none of its GGUFs showed up. This release fixes why.

Qwen3.8-Flash-Next ranks with the file that fits

On that PC the one build that fits is ISTA-DASLab's IQ3_XXS, which Strata's measurement puts on par with IQ4_XS. Two things kept it out. whichllm leaves out types below 3 bits unless you ask for one with --quant, and it checked the file's own type, not the IQ4_XS it's rated as. And the official repo's row only offers Q3_K_M to Q8_0, so even as a candidate the file could only rank as ISTA-DASLab's own repo, whose benchmark evidence is inherited and counts for less.

Now the rated type decides, and the official row can take an upload's measured file as that exact file, with its rating and its recipe. This is that PC now.

│   1 │ Qwen/Qwen3.8-27B          │ Q3_K_M │  12.5 GB │ Full GPU │     59.0 tok/s │      256k │ 2026-08-05 │  86.2 │
│     │ 27.8B                     │        │          │ 14.2 GB  │                │      →18k │            │       │
├─────┼───────────────────────────┼────────┼──────────┼──────────┼────────────────┼───────────┼────────────┼───────┤
│   2 │ Qwen/Qwen3.8-Flash-Next   │ IQ3_X… │  70.6 GB │ GPU+RAM  │   30.0 tok/s ~ │      256k │ 2026-08-24 │  84.1 │
│     │ 180.0B (10.4Ba)           │        │          │ 46.6 GB  │                │      →43k │            │       │
├─────┼───────────────────────────┼────────┼──────────┼──────────┼────────────────┼───────────┼────────────┼───────┤
│   3 │ google/gemma-4-26B-A4B-it │ Q3_K_M │  11.9 GB │ Full GPU │  201.1 tok/s ~ │      256k │ 2026-03-11 │  72.5 │
│     │ 25.8B (3.8Ba)             │        │          │ 13.0 GB  │                │     →101k │            │       │

With --profile coding it's first, at 85.4 to Qwen3.8-27B's 84.5. The --json row and the recipe name ISTA-DASLab's file.

llama serve -hf ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF:IQ3_XXS -c 4096 -ngl 99 --n-cpu-moe 37 -fa on

Rows that could not run are gone

• MLX weights no longer pass for FP16. A repo the Hub tags mlx is MLX whatever it's called, and MLX isn't a candidate anywhere, since it needs Apple Silicon and a Mac's ranking takes GGUF files.
• A Strata pack whose GGUFs are Strata's own pieces (the n-gram table, the dense weights and the token embeddings) no longer passes for a model. Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata counted as a 51.2B model that fits a GPU in 4.7 GB. This one takes effect with the next index build.

Also

• The weekly check of engine cards against their upstream releases, which 0.6.3's notes said had started, actually runs from this week. GitHub had never registered the workflow.

Still open

On Linux with an NVIDIA GPU, a safetensors checkpoint too big for the GPU is still timed as if llama.cpp kept its experts in RAM, which nothing that loads it does. That gets fixed with the GPU servers (vLLM, SGLang), which come next.

How it was checked

All 1,956 tests pass, and the wheel carries all 13 engine cards. The ranking above is this release on my PC against the current index. In the golden set, Qwen3.8-Flash-Next now also ranks second on an RTX 5090 with 32 GB, with the same file at about 15 tok/s.

Upgrade

uv tool upgrade whichllm
pip install -U whichllm
brew upgrade whichllm

Everything else is in CHANGELOG.md.