TurboQuant b10269-1.5.1

TurboQuant b10269-1.5.1

Fixed

• Ling-3.0-flash (BailingMoeV3) no longer emits garbage token bursts. The
  model is trained with clamped SwiGLU activations in its late layers, and the
  per-layer limits live in config.json under expert_swiglu_limit_list and
  share_expert_swiglu_limit_list. The public HF modeling code ignores those
  keys and so did this port, which caused deterministic transient logit
  collapse - output like count += 1eville dropped into otherwise fine
  generations. Measured at roughly -20 pass@1 on HumanEval (72.6% -> 93%+ with
  the fix); the garbage-token repro is eliminated.

Notes

• Re-convert your Ling-3.0-flash GGUF to get the fix. The clamp limits are
  written by the converter into two new KVs ({arch}.swiglu_clamp_exp and
  {arch}.swiglu_clamp_shexp); a GGUF produced before this release does not
  carry them, and the runtime then defaults to no clamping. Re-download the
  quant or re-run conversion/bailingmoe.py.
• Both KVs are optional and default to zero, so existing GGUFs and every other
  architecture are unaffected. The graph needed no change - the SwiGLU clamp
  branches in build_ffn / build_moe_ffn already trigger on a nonzero
  per-layer limit, matching the vLLM SwigluStepAndMul semantics.

Downloads

Backend	Asset
Linux x64 CPU only	llama-turboquant-linux-x64-cpu.tar.gz
Linux x64 Vulkan (+ portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 12.4, older drivers (+ portable CPU)	llama-turboquant-linux-x64-cuda-12.4.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Linux arm64 CUDA 13.3, DGX Spark / GB10 (+ portable CPU)	llama-turboquant-linux-arm64-cuda-13.3.tar.gz
Linux x64 AMD ROCm, RDNA2-RDNA4 (+ portable CPU)	llama-turboquant-linux-x64-rocm.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

The AMD ROCm archive targets RDNA2 through RDNA4 (gfx1030/1100/1101/1102/1151/1200/1201) and CDNA (gfx90a, gfx942 -- Instinct MI200/MI300) in a single archive, and needs the ROCm runtime installed on the system. Older GCN cards: use the Vulkan build.

The Linux arm64 CUDA archive is built for NVIDIA DGX Spark (GB10, sm_121); other arm64 NVIDIA machines (GH200, GB200, Thor) run it too but JIT the kernels from PTX on first launch.

All commits since b10269-1.5.0

• 📝 docs(changelog): add the b10269-1.5.1 section
• release: b10269-1.5.1
• bailingmoe3: apply trained SwiGLU clamps (expert_swiglu_limit_list)

What every archive contains, and how versions work

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize

Built from master at commit cd5609390, based on upstream llama.cpp b10269.

<upstream-base>-<fork-semver>: b10269 is the upstream llama.cpp build this fork is based on, 1.5.1 is the TurboQuant fork version. llama-server --version reports version: b10269-1.5.1.