Skip to content

TurboQuant b10018-1.1.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 22 Jul 08:46
b10018-1.1.0
67861a9

TurboQuant KV Cache — b10018-1.1.0

Built from master at commit 67861a917, based on upstream llama.cpp b10018.

What's new in 1.1.0

  • NVFP4 quantization target: llama-quantize model-f16.gguf out.gguf NVFP4 (ggml already had the kernels; the ftype is now wired through). llama-quantize ships in all archives.
  • First Linux CUDA backend (linux-x64-cuda-13.3): native CUDA kernels on Linux incl. native FP4 on Blackwell; cudart/cublas bundled; archs 80/86/89/90/120. Verified on an RTX 5090: 60.5k t/s pp512 / 1349 t/s tg128 on an NVFP4 model (2.2-2.4x the Vulkan path).
  • GPU smoke CI: released archives are now smoke-tested on rented GPUs (NVFP4 quantize -> serve -> coherence + GPU-actually-used asserts), on demand and nightly.
  • Fixes: test-llama-archs laguna failure (MoE-only arch + fixture keys + gate-meta fallback); llama_model_saver wrote n_ff_chexp under the shared-expert key, zeroing n_ff_shexp on save/reload roundtrip.

What's included

  • llama-server with --cache-type-k turbo3 / turbo4 support
  • llama-cli, llama-bench, llama-perplexity, llama-quantize
  • All supported backends in one release:
Backend Asset
Linux x64 (Vulkan + portable CPU) llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU) llama-turboquant-linux-x64-cuda-13.3.tar.gz
Windows x64 CPU llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4 llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3 llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized) llama-turboquant-macos-arm64.zip

Versioning

<upstream-base>-<fork-semver>: b10018 is the upstream llama.cpp build this fork is based on, 1.1.0 is the TurboQuant fork version. llama-server --version reports version: b10018-1.1.0.

Assets 12
Loading