Repository navigation
v2.5.5 — 100% Donor-Free DeepSeek-V4 MaxQ-v4 (57.99 GiB ConeCollapse), Metal/CUDA/Vulkan/CPU Loader, & 51-Chunk ref5632.kld Benchmarks
v2.5.5 — 100% Donor-Free DeepSeek-V4-Flash-0731-MaxQ-v4 (57.99 GiB ConeCollapse + DualGram, Universal Metal/CUDA/Vulkan/CPU Compatibility), Hardened Cross-Platform Loader, & Full 51-Chunk / 49-Chunk Blind ref5632.kld Evaluation
Highlights
-
100% Donor-Free
DeepSeek-V4-Flash-0731-MaxQ-v4(57.99 GiB/62,270,037,248B< 58.000 GiB,1,362Tensors, Zero Custom Tensor Types) (Synchopate/DeepSeek-V4-Flash-0731-MaxQ-v4-64GB-CPU-GGUF,scripts/build_dsv4_maxq_v4.py,scripts/maxq_v4_collapse.c,scripts/make_calib_v4_clean_utf8.py):- Built hermetically from
deepseek-ai/DeepSeek-V4-Flash-0731(@ 7872f01b1d1fe23eabc4c98b48bffcef5a386062) with zero donor GGUF weights:-
Universal Upstream Backend Compatibility (
0GGML_TYPE_Q2_0 = 42/0GGML_TYPE_VQ3_0 = 43tensors): Replaces all317customFractal-64tensors ofMaxQ-Fractalwith0731-imatrix-weighted standardggmlformats (43 × Q8_0attn_kv,145 × Q6_KKV/Indexer compressors &indexer.attn_q_b,3 × Q5_KL00MLA,3 × Q6_KL42MLA,123 × Q4_KL01..L41MLA, plus172 × Q6_Kshared experts,attn_q_a, andoutput.weight), makingMaxQ-v4100% natively compatible with Apple Metal GPU (M1–M5), NVIDIA CUDA MMQ (with automaticMUL_MAT_ID + ADD_IDkernel fusion), Vulkan, ARM64 NEON, and x86-64 AVX2/AVX-512. -
ConeCollapse (
34Layers2048 -> 1792) + FP64 Woodbury DualGram (NC = 384): Retains9full-widthK = 2048anchor layers ({0, 30, 34, 36, 37, 39, 40, 41, 42}, uncentered ridge) and collapses34interior layers toK = 1792(7 × 256,-12.5%MoE intermediate channels) via Deterministic Routed-Stride + Routersqrt(softplus)Top-Score Fill in an unconditionally well-conditioned ($\kappa_2 \le 86.33$ ) FP64 Cholesky dual system, absorbing residual drift into34 × F32blk.*.ffn_down_exps.bias(0.60 * mu), with MSE-optimalMXFP4(4.25 bpw) onL41–L42ffn_down_exps,IQ2_XXS(2.06 bpw) onL42gate/upandL00..L40down, andIQ1_M(1.75 bpw) across17sensitive layers (L20, L26..L41).
-
Universal Upstream Backend Compatibility (
- Built hermetically from
-
Full 51-Chunk
ref5632.kld& Gate-Free 49-Chunk Blind Split (3..51) Certification (benchmarks/results/atomicchat_wikialt_ctx5632_51c_dsv4_maxq_v4.log,benchmarks/QUALITY_GATE_VQ3_64.md):-
Full 51-Chunk
AtomicChat ref5632.kld(1..51,143,565scored positions vs.Mean PPL(base) = 4.331017 ± 0.027402):-
MaxQ-v4(57.99 GiB, 100% Donor-Free):Mean PPL(Q) = 6.8592 ± 0.0482(-6.01%vsMaxQ-Fractal7.2977 ± 0.0504),Mean KLD = 0.648514 ± 0.003535 nats(-12.36%vsMaxQ-Fractal0.739998),Median KLD = 0.183817 nats(-32.02%vsMaxQ-Fractal0.270386),Max KLD = 20.542444,RMS Δp = 27.562 ± 0.095%, andSame top p = 73.283 ± 0.117%(+2.886pp vsMaxQ-Fractal70.397%).
-
-
Strictly Unseen 49-Chunk Blind Partition (
3..51,137,935scored positions — excluding diagnostic gate chunks1..2):-
PPL(Q) = 6.8471(-6.12%vsMaxQ-Fractal7.2938),Mean KLD = 0.625743 nats(-12.84%vsMaxQ-Fractal0.717907),RMS Δp = 26.726%, andSame top p = 73.595%(+2.894pp vs70.701%).
-
-
Full 51-Chunk
-
Cross-Platform ConeCollapse Loader, Header Bounds Validation, &
deepseek4.cppIPC Cleanup (patches/deepseek-v4-flash-fractal-256k.patch,patches/dsv4-64gb-cpu.patch,scripts/build_zen5_llama.sh,scripts/llama_modchip.py):- Unified per-layer
n_ff_exp_i(1792/2048),o_lora_rank_i, and optionalblk.*.ffn_down_exps.bias(GGML_OP_ADD_ID) loading across Linux, Windows, and macOS Apple Silicon Metal (dsv4-64gb-cpu.patch), and removed all legacy/tmp/dsv4_moe_cfg.bin//opt/modelspaths fromsrc/models/deepseek4.cpp. - Added strict GGUF dimension & superblock alignment checks in
llama_model_deepseek4::load_arch_tensors(n_ff_exp_i in [256, 8*n_ff_exp],% 256 == 0,% ggml_blck_size == 0;o_lora_rank_i in [64, 4*o_lora_rank],% 64 == 0), raising a cleanstd::runtime_erroron malformed GGUFs instead of triggeringGGML_ASSERTabort(). - Updated
install.shandmodchipwith commit-pinneddsv4-maxq(MaxQ-v4) default and--preset dsv4-maxq-fractal(56.93 GiBsingle-passFractal-64).
- Unified per-layer
DeepSeek-V4-Flash-0731-MaxQ-v4 Shard Manifest (57.99 GiB, Pinned HF Rev 48356a120e78c01e82f425a74380f46f3925fb9c)
6112e887b21d1fdf7b7196232b528698a6ec2d5a59760a040cb4231f8e8d691b DeepSeek-V4-Flash-0731-MaxQ-v4-00001-of-00003.gguf (5,258,336 B)
5d85c740992b70395da5c3e4a856b3fad6ef725271722c8d837b35befe28330d DeepSeek-V4-Flash-0731-MaxQ-v4-00002-of-00003.gguf (35,543,586,816 B)
a8654d4d5fedf748d6f2b6404a8b7f9e19435a8aa8e2bb97ce9a66bc012c0f3a DeepSeek-V4-Flash-0731-MaxQ-v4-00003-of-00003.gguf (26,721,192,096 B)
Portable GLIBC_2.30 Linux x86_64 Binaries (SHA256SUMS)
Built inside Ubuntu 20.04 (GLIBC_2.30 max symbol, -static-libstdc++, zero libssl/libcurl dependency; requires OpenMP libgomp1 / libgomp):
*-avx512: AMD Zen 4/5 (Ryzen 7000/8000/9000,EPYC Genoa/Turin,Threadripper 7000) & Intel Ice Lake / Sapphire Rapids (avx512f+avx512vbmi;-DGGML_AVX512_BF16=OFF).*-avx2(and unsuffixed*-linux-x86_64): Pre-AVX512x86-64-v3CPUs (Haswell,Skylake,Alder/Raptor Lake,Zen 1/2/3—0%zmminstructions).
4d6722a92fd93da7eab75cf4dd82727a3ae788f40cbafb6f415d3cb96ef3bcdf llama-cli-modchip-linux-x86_64
4d6722a92fd93da7eab75cf4dd82727a3ae788f40cbafb6f415d3cb96ef3bcdf llama-cli-modchip-linux-x86_64-avx2
376f73ef93a13ba33164a1f1995f1a0e6c74e19af0d47b05fc72c975cc4c4253 llama-cli-modchip-linux-x86_64-avx512
ee819d255ca67c5b01cdc8d52bc5c39cad25ee7f08c248f4ab7f46e1189bbd75 llama-perplexity-modchip-linux-x86_64-avx2
0a778eb1d5a046f8801d5812b3382162415fe4f513327f8a4d1bb8eae6d741b6 llama-perplexity-modchip-linux-x86_64-avx512
a02b27be012edf365f51a1a94fe28640ed8f31cb4b2ed1b793e77eb67473f5e5 llama-server-modchip-linux-x86_64
a02b27be012edf365f51a1a94fe28640ed8f31cb4b2ed1b793e77eb67473f5e5 llama-server-modchip-linux-x86_64-avx2
631a9a9b8da286ef0827b779c863d2b93347bf69186245c86c1c90a7d463f579 llama-server-modchip-linux-x86_64-avx512