v0.4.0

Changelog

• Updated the llama.cpp base through upstream commit 32e789fdf. This includes
  EAGLE3 and expanded MTP support; new multimodal models and video input;
  LFM2/LFM2.5 tool parsing; router, API, and Web UI updates; the ET-SOC backend;
  and updates across the existing compute backends.
• Added KVarN target-context KV-cache compression through --cache-type-k and
  --cache-type-v values kvarn2, kvarn3, kvarn4, kvarn5, kvarn6,
  and kvarn8. It supports independent K/V widths, Qwen3.6 and Gemma 4,
  unified and non-unified caches, separate SWA cache types, CPU/CUDA stores,
  and native CUDA FlashAttention. Draft and auxiliary contexts continue to use
  standard cache types; unsupported placements fail closed or use an explicit
  bit-width-matched fallback.
• The standard KV-cache surface now includes q2_0, q2_1, q3_0, q3_1,
  and q6_1 alongside q6_0, with CPU/CUDA quantization,
  SET_ROWS/GET_ROWS, CUDA MMQ/vec-dot, and FlashAttention vector coverage.
  Cache-facing q2_0 uses GGML_TYPE_Q2_0S, keeping it distinct from
  upstream's serialized Q2_0 weight format.
• New KV cache precision tail (KVCPT) controls --kv-tail-tokens and --kv-tail-type keep the newest
  attention-visible entries exact in F16 or BF16 for standard quantized and
  KVarN target caches. Partial tails use compact overlays and fully covered
  groups can use native exact storage. Tail routing respects attention bias and
  cache placement, while state save/restore, prompt-cache reuse, sequence
  operations, and server handoff preserve the same model. Restores are
  transactional, and incompatible v0.3.1 sessions fail closed.
• Scoped --cache-ram to prompt-cache storage instead of active context
  checkpoint policy. Prompt-cache saves are byte-counted against the host
  budget before commit, KV rollback aligns to multimodal chunk boundaries,
  text-only slot save/restore works when an mtmd projector is loaded, and MTP
  prompt-cache reuse no longer requires regular checkpoints.
• Replaced the fork DFlash implementation with upstream draft-dflash.
  The speculative type uses upstream's draft-dflash name, and draft GGUFs
  must use upstream's dflash architecture, metadata keys, tensor names, and
  tokenizer contract. Bee retains a default-on profit-based draft-depth
  controller and reasoning-loop guard while using upstream token, sampler, and
  checkpoint behavior. Without --spec-draft-n-max, the draft limit comes
  from dflash.block_size - 1; an explicit value takes precedence.
• Removed TurboQuant/TCQ cache formats, TQ3/TQ4 weight formats, DDTree,
  CopySpec, the fork DFlash ring/capture/tape and reduced-verifier paths, the
  fringe controller, and their private arguments and environment variables.
  The turbo2, turbo3, and turbo4 cache names, including TCQ variants,
  warn and redirect by width to KVarN for target caches or a standard low-bit
  type for draft caches. Legacy TQ GGUF type IDs fail with a re-quantization
  error; copyspec, suffix, and recycle fail with migration guidance to
  draft-dflash or upstream n-gram modes.
• Replaced the CUDA FlashAttention quant build policy with 50 standard vector
  pairs and 15 balanced KVarN fast-decode pairs by default. Standard quant
  pairs follow the KVarN bit-pair rules; homogeneous F16 and BF16 pairs retain
  native precision-tail support.
  GGML_CUDA_FA_ALL_QUANTS=ON expands these to 169 standard pairs and all 36
  ordered KVarN pairs. The default-on GGML_CUDA_KVARN option is the single
  KVarN compilation gate for CUDA and HIP; disabling it omits the dedicated
  kernels and template instances.
• Hardened server control and router APIs. /v1/chat/completions/control
  reports success only when an active reasoning sampler accepts the transition.
  GET /models is read-only and sanitized; authenticated refresh uses
  POST /models/reload. Hugging Face tokens reach child processes only through
  HF_TOKEN and are excluded from arguments, presets, logs, and responses.
• Updated release packaging with CUDA 12.4/13.1 assets, upstream-DFlash and
  KVarN container metadata, ROCm/HIP shuffle compatibility, Windows CPU OpenMP
  runtime packaging, and updated Intel SYCL images. Every platform build,
  package, container, and publication step uses the same resolved source SHA;
  publication stops if the selected branch or tag moves.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.1
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.1 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.0
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.0
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.0
• CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.0
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.0
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.0
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.0

Browse all container images