Repository navigation
llama-modchip v3.2.0 — Upstream b12604 Rebase, Quantized Split-KV Decode (+15–61%) & Native C++ HARP-Cache v2
Latestv3.2.0 — Upstream llama.cpp @ 46eb174 (b12604) Rebase, Quantized & Heterogeneous Split-KV Decode (+15.1%–+61.4% at 32K), & Native C++ Chained-Prefix HARP-Cache v2 (16.40 ms /dev/shm Post-Restart Restore)
Release Date: November 2026
Highlights:
- Upstream
llama.cpp @ 46eb1742d(b12604) Rebase & NativeInkling-2Support (patches/llama-modchip-v3.2.patch):- Rebased the unified engine patch cleanly onto
ggml-org/llama.cpp @ 46eb1742db6d2caec1bfa848c5702b1bdcc815e5(b12604,SHA256 = 43a090791e3d062be28106e65101418f644bb21c13feab681f4a471ce84419ab; previousv3.1patch archived underpatches/archive/llama-modchip-v3.1.patch). - Picks up native upstream
src/models/inkling2.cpp(Inkling-2) and allb12416..b12604runtime improvements while preserving the zero-collisionggmlABI (41 = Q1_0,42 = Q2_0,43 = F8_E4M3untouched;44 = FRACTAL_64,45 = VQ3_64) — verified across all 38 tensor types intest-quantize-fns. - Pinned
46eb1742d/LLAMA_BUILD_NUMBER=12604/HF_UI_VERSION=b12604across all four platform build scripts (scripts/build_zen5_llama.sh,fedora/nemotron.sh,apple-silicon/build-dsv4-engine.sh,windows/build_patched_engine.ps1).
- Rebased the unified engine patch cleanly onto
- Quantized & Heterogeneous Split-KV Decode (
ggml/src/ggml-cpu/ops.cpp,ggml/src/ggml-cpu/ggml-cpu.c):- Unlocks CPU multi-thread Split-KV decode (
use_split_kv_path) for quantized and heterogeneous KV caches (Q8_0/Q4_0,Q4_0/Q4_0, etc.) wheneverneq2 < nth || (nek1 >= 2048 && neq2 % nth != 0)(runtime env overrideLLAMA_MODCHIP_SPLIT_KV_QUANT=0|1|2, default1). - Fixes two latent bugs in the CPU FlashAttention decode path: adds
+ CACHE_LINE_SIZE_F32scratch padding inggml-cpu.cdecode workspace sizing, and correctscur_pos/n_kv_colsSWA & NTA boundary tracking insideggml_compute_forward_flash_attn_ext_f16_one_chunk(ops.cpp). - Verified Persistent-Threadpool
32KDecode Attention Speedup (16C Zen 5,1.77e-8..2.54e-8max abs error vsnth=1ref,0%regression on even-head32/16):Q=12 / KV=4 / D=256(q8_0/q4_0):4.356 ms ->2.699 ms(+61.4%)Q=24 / KV=8 / D=128(q8_0/q4_0):4.152 ms ->2.972 ms(+39.7%)Q=24 / KV=8 / D=128(q4_0/q4_0):+21.6%Q=32 / KV=4 / 12T(q8_0/q4_0):+15.1%
- Unlocks CPU multi-thread Split-KV decode (
- Native C++ Chained-Prefix HARP-Cache v2 & Python Causal Fix (
tools/server/rc_harp_radix.h,speculative/cpp/rc_harp_radix.h,speculative/cpp/test_harp_radix.cpp,kv_cache/harp_cache.py):- Replaces unchained block hashing in
kv_cache/harp_cache.pywith chained causal prefix hashingH_k = FNV64(H_{k-1} || B_k)+model_fingerprint(eliminating false-positive prefix matches across diverged prompts; verified by divergence regression tests inkv_cache/test_harp_cache.py). - Adds standalone zero-dependency C++17
rc_harp_radix.h(verified under ASan/UBSan viaspeculative/cpp/test_harp_radix.cppand synced byte-for-byte in Preflight Gate 3 & Gate 9), wired directly intollama-server(server-slot-sched.{h,cpp}&server-context.cpp):- O(L/256) Chained Radix Slot Lookup + True LRU Eviction + Non-Destructive Shared-Prefix Preservation (fixes the upstream
f_keep < 0.25cliff for long shared prefixeslcp >= 512). - Opt-In
/dev/shmPersistent Slot Mirror (LLAMA_MODCHIP_HARP_SHM=1+--slot-save-path):0600permissions, anti-symlinkO_NOFOLLOW, atomic.tmp-> rename writes, and 24-byteharp_<fp>.idxancestor index.
- O(L/256) Chained Radix Slot Lookup + True LRU Eviction + Non-Destructive Shared-Prefix Preservation (fixes the upstream
- Verified Live
llama-serverMulti-Turn + Cold-Restart Benchmark (8B Q4_K_M,-ctk q8_0 -ctv q4_0,LLAMA_MODCHIP_HARP_SHM=1):- Cold Turn 1:
14,468 ms-> Multi-Branch Turn 2 (cache_n = 1303):1,285 ms(11.3×) -> Causal Divergence Turn 4:cache_n = 0(exact safety) -> Cold Server Restart Turn 5 restored1,302 / 1,319tokens (105.20 MiB) from/dev/shmin16.40 ms(6.41 GiB/s,1,351 mstotal TTFT,10.7×speedup).
- Cold Turn 1:
- Zero GGUF checkpoint changes — all
v3.0/v3.1.1GGUF SHA-256s remain 100% identical.
- Replaces unchained block hashing in
- 12 Rebuilt Static-
libgompPortableGLIBC_2.30Linuxx86_64Binaries, Tarball & Patch (v3.2.0SHA256SUMS):- Built on
ubuntu:20.04(gcc-10/g++-10,GLIBC <= 2.30, staticlibgomp.a+libstdc++.a+libgcc.a,0%zmmin-avx2,0vdpbf16psin-avx512,0dynamiclibgomp/libstdc++/libssldependencies):43a090791e3d062be28106e65101418f644bb21c13feab681f4a471ce84419ab llama-modchip-v3.2.patch0cf2d3bcd31349d98bcef1d23af3545912b70930049437e8aa7f6cf5e2ff2082 llama-server-modchip-linux-x86_64-avx512d96745036cb44f76bce70237fc1e474058958e4d2eff739b85415eac4c78131d llama-server-modchip-linux-x86_64-avx2(andllama-server-modchip-linux-x86_64)dc4c9da5016a8bf4407e14691407b0471625bb775cf76ad2a71e7bc25e0a98af llama-cli-modchip-linux-x86_64-avx51208c6236eec64d645b0528d1752f8423bfa028753c8f974c46b6556b1a9022709 llama-cli-modchip-linux-x86_64-avx2(andllama-cli-modchip-linux-x86_64)9e40525f859214dc36aab36cb99bc81b68762816e1dbe07b31dc8d92b2a037e6 llama-bench-modchip-linux-x86_64-avx5125d6f9c7a8fd63d3ba6243f3c96b64fe7ae1fd762d814e8b509b9589465a7cff6 llama-bench-modchip-linux-x86_64-avx2b52c9b537ec9dc6f635b2d45e20b6d6b48123eacb7218584be5e1056a51e5913 llama-perplexity-modchip-linux-x86_64-avx512edae007a49e88e14d2d9a79f0d07fb2b32ee003c6016f55610806bf2ddbdfcc0 llama-perplexity-modchip-linux-x86_64-avx2f24985fa7f595aded7bc9cb523bbf4153362126e80f7bf161b339d5c07352cbe llama-quantize-modchip-linux-x86_64-avx512d8a00d6853b9c345a48c66ffa8948ac52d08b48db65c949831bfd2e890d49424 llama-quantize-modchip-linux-x86_64-avx25f3cd4d73e7407817d6da2680150f27202d85291275d56221502709782d5ed1c llama-modchip-v3.2.0-linux-x86_64.tar.gz
- Built on