v0.4.1 Preview

Warning

Rolling preview build from v0.4.1 at 3be484a6974c. Assets and moving Docker tags are replaced by newer successful branch builds. Last updated: 25/07/2026 18:06.

Changelog

• Fixed the static/test source build after the memory interface gained cell-level sequence operations, including the abstract mock_memory regression in test-batch-alloc.
• Extended native KVarN attention beyond the optimized NVIDIA CUDA path. CPU and Vulkan now consume compressed records directly, and CUDA/HIP has a portable direct-record path for devices that cannot use the CUDA MMA implementation. Runtime capability checks select the direct path per layer and retain an explicit materialization fallback for unsupported shapes/features; CPU placement works with KV offload disabled, while Vulkan requires shader Int64 and buffer-device-address support. CUDA and ROCm release builds explicitly enable the shared KVarN kernels.
• Reduced KVarN host checkpoint stalls for large exact tails by serializing component-major contiguous slot runs instead of issuing one backend transfer per payload row. A normal 4096-token tail now takes 32-64 host tensor operations instead of 131,072. KVarN state format v13 preserves all exact-tail bytes, transactional restores, host/on-device sequence state, and v12 host-state compatibility while exposing debug-level transfer-count diagnostics.
• Kept small speculative verification batches on CUDA's native KVarN attention path instead of materializing the full active K/V window. CUDA now reports its rotated-query capacity to the model graph through 16 tokens. Single-token generation retains the split/vector decode kernels, while multi-token verification uses tiled descriptor-native MMA so each decoded K/V tile is reused across query rows and does not create the split kernel's context-sized per-query partial output. Rotated-record loads now distribute reconstruction across the full CTA, cache K/V axes per record, decode packed value and key pairs cooperatively, and use an adaptive 128-column tile for Q9-Q16/GQA>4 when the device's opt-in shared-memory limit and measured kernel occupancy allow it; unsupported devices retain the 64-column fallback. Larger batches retain the native prefill route. This applies uniformly to DFlash, MTP, EAGLE3, and n-gram speculation. CUDA builds with dedicated KVarN kernels disabled also retain complete false capability stubs and link cleanly.
• Fixed portable-backend KVarN prompt processing and Vulkan long-context decode. CPU, HIP, and Vulkan now materialize large query batches into their tiled standard FlashAttention path instead of running prompt prefill through the scalar direct-record shader; one-row decode and Vulkan batches through 16 rows remain native, while CUDA retains its descriptor-native prefill route. Vulkan direct decode now reuses each K/V reconstruction across groups of up to four GQA heads, applies split-K when the query/head grid under-occupies the device, reuses shared K/V indices, and uses subgroup reductions.
• Reduced KV precision-tail memory and made tail routing per-layer. Persistent exact storage is now N + rollback rows, or W + rollback rows with no compressed body when a tail fully covers its SWA window, instead of a ubatch-sized arena rounded to the FlashAttention tile; current K/V and execution alignment live in graph-local workspaces. Each layer carries one execution descriptor that drives allocation, graph construction, graph identity, and backend validation, so a planned native route is checked on the final fused attention node and fails closed instead of being scheduled silently on CPU, and llama-bench --kv-memory additionally reports native, device-fallback, and CPU layer counts.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.1
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.1 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-preview-v0.4.1
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-preview-v0.4.1
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-preview-v0.4.1
• CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-preview-v0.4.1
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-preview-v0.4.1
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-preview-v0.4.1
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-preview-v0.4.1

Browse all container images