Skip to content

v1.6.0

Choose a tag to compare

@github-actions github-actions released this 08 Aug 15:42
· 11 commits to main since this release

Added

  • DeepSeek V4 (model_type=deepseek_v4) as a development-convertible MoE
    family: inspect, plan-manual, and convert on mixed FP4+FP8 Flash sources via
    dequant → affine re-pack under the plan predicate.
  • Experimental 2-bit / 3-bit manual recipes for Flash
    (examples/deepseek-v4-experimental-{2,3}bit-v0.1.yaml).
  • Development model-card product classes for 2bit / 3bit (and
    *-experimental variants).

Changed

  • Convert path handles DeepSeek V4 sanitizer renames: HC tensors, shared-expert
    w1/w2/w3 → proj names, router ffn.gate.biase_score_correction_bias,
    FusedSwitchGLU gate+up fuse, and MultiLinear wo_a reshape.
  • Inventory reconstructs logical parameters for FP4 expert bodies packed in
    I8/U8 (expert_dtype: fp4); HyperConnection learnable .scale is not
    treated as quant metadata.
  • Integrated MTP is byte-preserved into mtp.safetensors without MLX expert
    fusion; protected-MTP verify allows native source packing (including FP4).
  • Convert dequants leftover QuantizedMultiLinear modules that public
    mlx_lm.utils.dequantize_model skips (DeepSeek attn.wo_a).

Notes / operators

  • DeepSeek V4 conversion needs an mlx-lm build that ships
    mlx_lm.models.deepseek_v4 (not all PyPI 0.31.x wheels). See
    docs/compatibility.md.
  • Published Flash packs are development evidence only (not certified).
    2-bit / 3-bit remain experimental; measured BPW is higher than the product
    class because of protected tensors and MTP.
  • No migration step for existing non-DeepSeek artifacts or plans.