Skip to content

TurboQuant b10269-1.4.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 04 Aug 20:59
b10269-1.4.0
074bf82

TurboQuant KV Cache b10269-1.4.0

Built from master at commit 074bf826e, based on upstream llama.cpp b10269.

Feaures

  • DeepSeek V4 Flash 0731 support & speedup (+custom ops CUDA kernels)
  • Kimi K3 + vision full support
  • RDNA2-4 + CDNA support for AMD GPUs in linux x64 release
  • commits sync from upstream

What's included

  • llama-server with --cache-type-k turbo3 / turbo4 support
  • llama-cli, llama-bench, llama-perplexity, llama-quantize
  • All supported backends in one release:
Backend Asset
Linux x64 CPU only llama-turboquant-linux-x64-cpu.tar.gz
Linux x64 Vulkan (+ portable CPU) llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 12.4, older drivers (+ portable CPU) llama-turboquant-linux-x64-cuda-12.4.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU) llama-turboquant-linux-x64-cuda-13.3.tar.gz
Linux x64 AMD ROCm (+ portable CPU) llama-turboquant-linux-x64-rocm.tar.gz
Windows x64 CPU llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4 llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3 llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized) llama-turboquant-macos-arm64.zip

The AMD ROCm archive targets RDNA2 through RDNA4 (gfx1030/1100/1101/1102/1151/1200/1201) and CDNA (gfx90a, gfx942 -- Instinct MI200/MI300) in a single archive, and needs the ROCm runtime installed on the system. Older GCN cards: use the Vulkan build.

Versioning

<upstream-base>-<fork-semver>: b10269 is the upstream llama.cpp build this fork is based on, 1.4.0 is the TurboQuant fork version. llama-server --version reports version: b10269-1.4.0.

Assets 18
Loading