Skip to content

v0.6.7

@pjordanandrsn pjordanandrsn tagged this 30 Jul 09:57
Four blockers cleared in #41, each only visible once the previous was gone:
trust_remote_code (opt-in, default OFF), kimi_k3 registered as a supported
architecture, a per-family multimodal checkpoint prefix, and a per-expert MXFP4
branch.

The MXFP4 branch works because 0.6.6 made dequantize_mxfp4 rank-general -- K3
ships [rows, K//2] + [rows, K//32] per expert, the rank-3 case transpose(1,2)
rejected.

This path dequantizes to bf16 for a resident forward. It does not shrink K3;
unpacking 1.446 TB of packed experts makes them larger. Computing on the packed
bytes is a separate route in grouped-nf4-gemm.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assets 2
Loading