Skip to content

v2.5.5 — 100% Donor-Free DeepSeek-V4 MaxQ-v4 (57.99 GiB ConeCollapse), Metal/CUDA/Vulkan/CPU Loader, & 51-Chunk ref5632.kld Benchmarks

Choose a tag to compare

@synchopate synchopate released this 15 Nov 09:08

v2.5.5 — 100% Donor-Free DeepSeek-V4-Flash-0731-MaxQ-v4 (57.99 GiB ConeCollapse + DualGram, Universal Metal/CUDA/Vulkan/CPU Compatibility), Hardened Cross-Platform Loader, & Full 51-Chunk / 49-Chunk Blind ref5632.kld Evaluation

Highlights

  1. 100% Donor-Free DeepSeek-V4-Flash-0731-MaxQ-v4 (57.99 GiB / 62,270,037,248 B < 58.000 GiB, 1,362 Tensors, Zero Custom Tensor Types) (Synchopate/DeepSeek-V4-Flash-0731-MaxQ-v4-64GB-CPU-GGUF, scripts/build_dsv4_maxq_v4.py, scripts/maxq_v4_collapse.c, scripts/make_calib_v4_clean_utf8.py):

    • Built hermetically from deepseek-ai/DeepSeek-V4-Flash-0731 (@ 7872f01b1d1fe23eabc4c98b48bffcef5a386062) with zero donor GGUF weights:
      • Universal Upstream Backend Compatibility (0 GGML_TYPE_Q2_0 = 42 / 0 GGML_TYPE_VQ3_0 = 43 tensors): Replaces all 317 custom Fractal-64 tensors of MaxQ-Fractal with 0731-imatrix-weighted standard ggml formats (43 × Q8_0 attn_kv, 145 × Q6_K KV/Indexer compressors & indexer.attn_q_b, 3 × Q5_K L00 MLA, 3 × Q6_K L42 MLA, 123 × Q4_K L01..L41 MLA, plus 172 × Q6_K shared experts, attn_q_a, and output.weight), making MaxQ-v4 100% natively compatible with Apple Metal GPU (M1–M5), NVIDIA CUDA MMQ (with automatic MUL_MAT_ID + ADD_ID kernel fusion), Vulkan, ARM64 NEON, and x86-64 AVX2/AVX-512.
      • ConeCollapse (34 Layers 2048 -> 1792) + FP64 Woodbury DualGram (NC = 384): Retains 9 full-width K = 2048 anchor layers ({0, 30, 34, 36, 37, 39, 40, 41, 42}, uncentered ridge) and collapses 34 interior layers to K = 1792 (7 × 256, -12.5% MoE intermediate channels) via Deterministic Routed-Stride + Router sqrt(softplus) Top-Score Fill in an unconditionally well-conditioned ($\kappa_2 \le 86.33$) FP64 Cholesky dual system, absorbing residual drift into 34 × F32 blk.*.ffn_down_exps.bias (0.60 * mu), with MSE-optimal MXFP4 (4.25 bpw) on L41–L42 ffn_down_exps, IQ2_XXS (2.06 bpw) on L42 gate/up and L00..L40 down, and IQ1_M (1.75 bpw) across 17 sensitive layers (L20, L26..L41).
  2. Full 51-Chunk ref5632.kld & Gate-Free 49-Chunk Blind Split (3..51) Certification (benchmarks/results/atomicchat_wikialt_ctx5632_51c_dsv4_maxq_v4.log, benchmarks/QUALITY_GATE_VQ3_64.md):

    • Full 51-Chunk AtomicChat ref5632.kld (1..51, 143,565 scored positions vs. Mean PPL(base) = 4.331017 ± 0.027402):
      • MaxQ-v4 (57.99 GiB, 100% Donor-Free): Mean PPL(Q) = 6.8592 ± 0.0482 (-6.01% vs MaxQ-Fractal 7.2977 ± 0.0504), Mean KLD = 0.648514 ± 0.003535 nats (-12.36% vs MaxQ-Fractal 0.739998), Median KLD = 0.183817 nats (-32.02% vs MaxQ-Fractal 0.270386), Max KLD = 20.542444, RMS Δp = 27.562 ± 0.095%, and Same top p = 73.283 ± 0.117% (+2.886 pp vs MaxQ-Fractal 70.397%).
    • Strictly Unseen 49-Chunk Blind Partition (3..51, 137,935 scored positions — excluding diagnostic gate chunks 1..2):
      • PPL(Q) = 6.8471 (-6.12% vs MaxQ-Fractal 7.2938), Mean KLD = 0.625743 nats (-12.84% vs MaxQ-Fractal 0.717907), RMS Δp = 26.726%, and Same top p = 73.595% (+2.894 pp vs 70.701%).
  3. Cross-Platform ConeCollapse Loader, Header Bounds Validation, & deepseek4.cpp IPC Cleanup (patches/deepseek-v4-flash-fractal-256k.patch, patches/dsv4-64gb-cpu.patch, scripts/build_zen5_llama.sh, scripts/llama_modchip.py):

    • Unified per-layer n_ff_exp_i (1792 / 2048), o_lora_rank_i, and optional blk.*.ffn_down_exps.bias (GGML_OP_ADD_ID) loading across Linux, Windows, and macOS Apple Silicon Metal (dsv4-64gb-cpu.patch), and removed all legacy /tmp/dsv4_moe_cfg.bin / /opt/models paths from src/models/deepseek4.cpp.
    • Added strict GGUF dimension & superblock alignment checks in llama_model_deepseek4::load_arch_tensors (n_ff_exp_i in [256, 8*n_ff_exp], % 256 == 0, % ggml_blck_size == 0; o_lora_rank_i in [64, 4*o_lora_rank], % 64 == 0), raising a clean std::runtime_error on malformed GGUFs instead of triggering GGML_ASSERT abort().
    • Updated install.sh and modchip with commit-pinned dsv4-maxq (MaxQ-v4) default and --preset dsv4-maxq-fractal (56.93 GiB single-pass Fractal-64).

DeepSeek-V4-Flash-0731-MaxQ-v4 Shard Manifest (57.99 GiB, Pinned HF Rev 48356a120e78c01e82f425a74380f46f3925fb9c)

6112e887b21d1fdf7b7196232b528698a6ec2d5a59760a040cb4231f8e8d691b  DeepSeek-V4-Flash-0731-MaxQ-v4-00001-of-00003.gguf  (5,258,336 B)
5d85c740992b70395da5c3e4a856b3fad6ef725271722c8d837b35befe28330d  DeepSeek-V4-Flash-0731-MaxQ-v4-00002-of-00003.gguf  (35,543,586,816 B)
a8654d4d5fedf748d6f2b6404a8b7f9e19435a8aa8e2bb97ce9a66bc012c0f3a  DeepSeek-V4-Flash-0731-MaxQ-v4-00003-of-00003.gguf  (26,721,192,096 B)

Portable GLIBC_2.30 Linux x86_64 Binaries (SHA256SUMS)

Built inside Ubuntu 20.04 (GLIBC_2.30 max symbol, -static-libstdc++, zero libssl/libcurl dependency; requires OpenMP libgomp1 / libgomp):

  • *-avx512: AMD Zen 4/5 (Ryzen 7000/8000/9000, EPYC Genoa/Turin, Threadripper 7000) & Intel Ice Lake / Sapphire Rapids (avx512f + avx512vbmi; -DGGML_AVX512_BF16=OFF).
  • *-avx2 (and unsuffixed *-linux-x86_64): Pre-AVX512 x86-64-v3 CPUs (Haswell, Skylake, Alder/Raptor Lake, Zen 1/2/3 — 0 %zmm instructions).
4d6722a92fd93da7eab75cf4dd82727a3ae788f40cbafb6f415d3cb96ef3bcdf  llama-cli-modchip-linux-x86_64
4d6722a92fd93da7eab75cf4dd82727a3ae788f40cbafb6f415d3cb96ef3bcdf  llama-cli-modchip-linux-x86_64-avx2
376f73ef93a13ba33164a1f1995f1a0e6c74e19af0d47b05fc72c975cc4c4253  llama-cli-modchip-linux-x86_64-avx512
ee819d255ca67c5b01cdc8d52bc5c39cad25ee7f08c248f4ab7f46e1189bbd75  llama-perplexity-modchip-linux-x86_64-avx2
0a778eb1d5a046f8801d5812b3382162415fe4f513327f8a4d1bb8eae6d741b6  llama-perplexity-modchip-linux-x86_64-avx512
a02b27be012edf365f51a1a94fe28640ed8f31cb4b2ed1b793e77eb67473f5e5  llama-server-modchip-linux-x86_64
a02b27be012edf365f51a1a94fe28640ed8f31cb4b2ed1b793e77eb67473f5e5  llama-server-modchip-linux-x86_64-avx2
631a9a9b8da286ef0827b779c863d2b93347bf69186245c86c1c90a7d463f579  llama-server-modchip-linux-x86_64-avx512