ds4 cannot load MXFP4Experts-0731 GGUF: unsupported GGUF type 39 (MXFP4) for routed experts

[aaa2015]

Summary

ds4-server crashes when loading DeepSeek-V4-Flash-MXFP4Experts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-mxfp4-0731.gguf (from this repo, added 2026-08-01): routed expert tensors use GGUF type 39 (MXFP4), which the runtime does not recognize — it expects a routed-expert quant type and aborts.

Error output

ds4: warning: tensor blk.41.ffn_gate_exps.weight has unsupported GGUF type 39
ds4: warning: tensor blk.41.ffn_down_exps.weight has unsupported GGUF type 39
ds4: warning: tensor blk.41.ffn_up_exps.weight has unsupported GGUF type 39
ds4: warning: tensor blk.42.ffn_gate_exps.weight has unsupported GGUF type 39
ds4: warning: tensor blk.42.ffn_down_exps.weight has unsupported GGUF type 39
ds4: warning: tensor blk.42.ffn_up_exps.weight has unsupported GGUF type 39
ds4: tensor blk.0.ffn_gate_exps.weight has type 39 (unknown), expected a routed expert quant type

(ds4-server then exits/crashes — same failure also when loading via ds4 CLI.)

Repro

./ds4-server -m DeepSeek-V4-Flash-MXFP4Experts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-mxfp4-0731.gguf

Environment: macOS (M2 Ultra, 192 GB), ds4 built 2026-07-27, branch laguna-s2.1 (also checked origin/main — same).

Analysis

• gguf-tools/quants.h defines DS4Q_TYPE_MXFP4 = 39, and the quantizer can emit MXFP4 expert tensors — so the tooling side knows the type.
• The runtime expert-loading path (ds4.c / ds4.h) does not handle type 39, so it treats it as unknown and aborts. All branches (main, laguna-s2.1, glm5.2) currently have the same gap.

Why this matters

This file is the highest-fidelity 0731 option in the repo: routed experts stay bit-exact in the official MXFP4 format (like the native DeepSeek checkpoint), with HC/compressor/indexer kept in F16. It would be great to actually run it — the quality is meaningfully better than the Q4_K-experts re-quant.

Suggestions

1. Implement MXFP4 expert decoding in the runtime (dequant path for type 39), or
2. If support is planned but not ready, mark this file as "not yet loadable" in the README so users don't download 156 GB and hit a crash.

Thanks for the great work on the 0731 quants — Q4KExperts-imatrix-0731 runs perfectly via ds4 + DSpark here.

[OmarB97]

Confirming ds4f-mxfp4 (@4893e0c4) resolves this on our setup: the MXFP4Experts-0731 GGUF that previously failed with "unsupported GGUF type 39" now serves cleanly on a two-Mac pipelined pair (2× M4 Max 128GB over Thunderbolt 5, coordinator layers 0:20 + worker 21:output, ctx 262144, ds4-server API). Sustained 26.2 tok/s decode over 2,000 tokens with the DSpark MTP draft, up from 22.7 tok/s on the Q4K-experts imatrix quant of the same checkpoint — faster and lossless. No SSD streaming needed at this split, so PR #578's mlock fix wasn't required for the in-RAM two-rank case.