You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
DeepSeek V4 (model_type=deepseek_v4) as a development-convertible MoE
family: inspect, plan-manual, and convert on mixed FP4+FP8 Flash sources via
dequant → affine re-pack under the plan predicate.
Experimental 2-bit / 3-bit manual recipes for Flash
(examples/deepseek-v4-experimental-{2,3}bit-v0.1.yaml).
Development model-card product classes for 2bit / 3bit (and *-experimental variants).
Inventory reconstructs logical parameters for FP4 expert bodies packed in
I8/U8 (expert_dtype: fp4); HyperConnection learnable .scale is not
treated as quant metadata.
Integrated MTP is byte-preserved into mtp.safetensors without MLX expert
fusion; protected-MTP verify allows native source packing (including FP4).
Convert dequants leftover QuantizedMultiLinear modules that public mlx_lm.utils.dequantize_model skips (DeepSeek attn.wo_a).
Notes / operators
DeepSeek V4 conversion needs an mlx-lm build that ships mlx_lm.models.deepseek_v4 (not all PyPI 0.31.x wheels). See docs/compatibility.md.
Published Flash packs are development evidence only (not certified).
2-bit / 3-bit remain experimental; measured BPW is higher than the product
class because of protected tensors and MTP.
No migration step for existing non-DeepSeek artifacts or plans.