[Request] MLX support for Kimi K3 architecture (Apple Silicon)

[zhwangsir]

Request: MLX support for Kimi K3 architecture (Apple Silicon)

Model

• Name: Kimi K3
• HuggingFace: https://huggingface.co/moonshotai/Kimi-K3
• Size: 32B active parameters, 1T total parameters (MoE)
• Novel architecture:
  • Kimi Delta Attention (KDA)
  • Attention Residuals
  • Stable LatentMoE
  • MoonViT-V2 (vision encoder for multimodal inputs)
• Weight format: MXFP4 / MXFP8 safetensors

Use case

I am trying to run Kimi K3 on a 4× Mac Studio M3 Ultra cluster (512 GB unified memory each, ~2 TB total) connected via RDMA over Thunderbolt. This hardware configuration is uniquely suited to large MoE models because of the massive unified memory pool and fast interconnect, but it depends on an MLX-based inference path.

Blockers for an MLX port

1. MXFP4 / MXFP8 dequantization — Kimi K3 weights are published in these low-precision formats; MLX needs conversion/dequantization support (or a conversion script similar to the existing llms/mlx_lm flows).
2. Kimi Delta Attention (KDA) — a new attention variant that is not a standard MHA/GQA/MQA layer.
3. Attention Residuals — additional residual connections around attention blocks.
4. Stable LatentMoE — a new MoE routing/expert design that differs from standard Switch-MoE.
5. MoonViT-V2 — vision encoder needed for the multimodal variant.

Ask

• Is there any plan to add Kimi K3 support to mlx-lm or mlx-community?
• Could the team publish a conversion guide or script once the architecture is understood?
• Are there specific MLX primitives that are currently missing for MXFP4/MXFP8 that the community should prioritize?

I have the hardware ready and can run validation, benchmarks, and memory profiling for any experimental MLX branch.

References

• Model card: https://huggingface.co/moonshotai/Kimi-K3
• Technical report / code: https://github.com/MoonshotAI/Kimi-K3

Thanks for all the work on MLX — it is the best path to run frontier models locally on Apple Silicon.

[zcbenz]

We do want to support Kimi K3 but at the moment we don't have resources to add support in a timely manner, it is suggested to use community projects like mlx-vlm before we add support in mlx-lm.