DeepSeek V4 Flash ships FP4 experts natively, and ds4f-mxfp4 serves them --
but only from GGUFs produced by its own quantizer from the original
safetensors. Community GGUF exports in the llama.cpp dialect, e.g.
bullerwins/DeepSeek-V4-Flash-0731-MXFP4_MOE-Q8_0 (156 GB), do not load at
all, even though their MXFP4 routed-expert payload is exactly what this
branch was built to serve.
Two independent gaps on the metadata/naming side (part 1 of 2; the dense
tensor type gap is filed separately):
-
Hard-required metadata keys. The config loader uses
required_u32(m, "deepseek4.vocab_size")-style hard requirements
(ds4.c ~5613 on this branch). The community dialect omits
deepseek4.vocab_size plus eight deepseek4.* hc/lora/hash-layer keys,
so the load dies at the first missing key. All of them are derivable:
vocab_size from the length of tokenizer.ggml.tokens, and the hc/lora/
hash-layer values from keys the dialect does carry.
-
Tensor-name mismatches. About twenty tensor lookups fail because the
community naming differs from ds4 canonical names (missing suffixes,
renamed attention and expert tensors).
Both are addressable without touching the native path: derivation only runs
when a required key is absent, and the alias table is consulted only when
the canonical lookup misses, so GGUFs from the branch's own quantizer load
exactly as before.
Evidence this closes a real gap: with the compat layer applied (plus the
dense type conversion in the companion issue), the bullerwins GGUF has been
serving in production on a GB10 CUDA node with --ssd-streaming for several
weeks, passing a 12/12 eval battery.
DeepSeek V4 Flash ships FP4 experts natively, and
ds4f-mxfp4serves them --but only from GGUFs produced by its own quantizer from the original
safetensors. Community GGUF exports in the llama.cpp dialect, e.g.
bullerwins/DeepSeek-V4-Flash-0731-MXFP4_MOE-Q8_0(156 GB), do not load atall, even though their MXFP4 routed-expert payload is exactly what this
branch was built to serve.
Two independent gaps on the metadata/naming side (part 1 of 2; the dense
tensor type gap is filed separately):
Hard-required metadata keys. The config loader uses
required_u32(m, "deepseek4.vocab_size")-style hard requirements(ds4.c ~5613 on this branch). The community dialect omits
deepseek4.vocab_sizeplus eightdeepseek4.*hc/lora/hash-layer keys,so the load dies at the first missing key. All of them are derivable:
vocab_size from the length of
tokenizer.ggml.tokens, and the hc/lora/hash-layer values from keys the dialect does carry.
Tensor-name mismatches. About twenty tensor lookups fail because the
community naming differs from ds4 canonical names (missing suffixes,
renamed attention and expert tensors).
Both are addressable without touching the native path: derivation only runs
when a required key is absent, and the alias table is consulted only when
the canonical lookup misses, so GGUFs from the branch's own quantizer load
exactly as before.
Evidence this closes a real gap: with the compat layer applied (plus the
dense type conversion in the companion issue), the bullerwins GGUF has been
serving in production on a GB10 CUDA node with
--ssd-streamingfor severalweeks, passing a 12/12 eval battery.