Request: Apple Silicon / MLX / EXO support roadmap for Kimi K3
First of all, thank you for open-sourcing Kimi K3 — the 32B/1T MoE design and the reported benchmarks are very impressive.
Hardware context
I operate a 4× Mac Studio M3 Ultra cluster, each with 512 GB unified memory, connected via RDMA over Thunderbolt. This gives ~2 TB of shared addressable memory and very high inter-node bandwidth, which seems ideal for running Kimi K3 in a distributed setting.
Current situation
The current first-party inference integrations focus on NVIDIA/AMD stacks (vLLM, SGLang, TokenSpeed). For Apple Silicon, the viable paths appear to be:
- MLX (via
mlx-lm or mlx-community conversion)
- EXO (distributed Apple Silicon inference with RDMA support)
Both would need support for the new architecture components introduced in Kimi K3:
- Kimi Delta Attention (KDA)
- Attention Residuals
- Stable LatentMoE
- MoonViT-V2 (for multimodal)
- MXFP4 / MXFP8 weight handling
Questions
- Does Moonshot AI plan to release reference implementations or conversion scripts for MLX / Apple Silicon?
- Is there a target timeline for community ports (e.g., llama.cpp, EXO, mlx-community)?
- Would the team consider releasing BF16/FP16 weights in addition to MXFP4/8 to lower the barrier for frameworks that do not yet support these new FP4/FP8 layouts?
- Are there detailed architecture docs or reference CUDA kernels coming that would help third-party implementations?
I am happy to contribute validation resources (remote access to the 4-node M3 Ultra RDMA cluster, benchmarks, profiling) if that helps an Apple Silicon port move faster.
References
Thank you again for the open-source release.
Request: Apple Silicon / MLX / EXO support roadmap for Kimi K3
First of all, thank you for open-sourcing Kimi K3 — the 32B/1T MoE design and the reported benchmarks are very impressive.
Hardware context
I operate a 4× Mac Studio M3 Ultra cluster, each with 512 GB unified memory, connected via RDMA over Thunderbolt. This gives ~2 TB of shared addressable memory and very high inter-node bandwidth, which seems ideal for running Kimi K3 in a distributed setting.
Current situation
The current first-party inference integrations focus on NVIDIA/AMD stacks (vLLM, SGLang, TokenSpeed). For Apple Silicon, the viable paths appear to be:
mlx-lmormlx-communityconversion)Both would need support for the new architecture components introduced in Kimi K3:
Questions
I am happy to contribute validation resources (remote access to the 4-node M3 Ultra RDMA cluster, benchmarks, profiling) if that helps an Apple Silicon port move faster.
References
Thank you again for the open-source release.