feat: Qwen3-Coder-Next support — 8-bit gate dequant + fused projection deinterleave

[userFRM]

Summary

Adds support for running Qwen3-Coder-Next (78B MoE, 48 layers, 512 experts) on flash-moe. Also fixes a class of bugs affecting any MLX model with mixed-precision quantization.

Key Findings

1. Mixed-Precision Gate Quantization (affects all MLX models)

MLX 4-bit quantized models use 8-bit quantization for routing gates (mlp.gate, mlp.shared_expert_gate), specified per-tensor in config.json:

"quantization": {
    "bits": 4, "group_size": 64,
    "model.layers.0.mlp.gate": {"bits": 8},
    "model.layers.0.mlp.shared_expert_gate": {"bits": 8}
}

The 4-bit dequant kernel reads 8 nibbles per uint32 but these tensors pack 4 bytes per uint32. This corrupts routing scores, selecting wrong experts. Fix: new dequant_matvec_8bit Metal kernel + CPU fallback, with bits field in BatchMatvecSpec for per-tensor dispatch.

2. Fused Linear Attention Projections (Coder-Next specific)

Coder-Next uses fused projections (in_proj_qkvz, in_proj_ba) instead of Qwen3.5-397B's separate projections (in_proj_qkv, in_proj_z, in_proj_b, in_proj_a). The fused output is interleaved per k-head:

[h0_q(128), h0_k(128), h0_v(256), h0_z(256), h1_q(128), ...]

Not flat [all_Q, all_K, all_V, all_Z]. Fix: per-k-head deinterleave after projection matvec, with automatic detection of fused vs separate naming.

3. Verified Against MLX Reference

• CPU-only path produces correct output verified against mlx-lm (cosine similarity 0.999999 on embedding, matching gate scores)
• GPU pipeline has a remaining buffer dimension issue in CMD2 (tracked separately)

Changes

• shaders.metal: Added dequant_matvec_8bit kernel
• infer.m: 8-bit pipeline, fused projection detection + deinterleave, chat template wrapping, Coder-Next model constants
• repack_coder_next.py: Expert repacker for switch_mlp stacked tensor format

Model Constants

Parameter	Qwen3.5-397B	Qwen3-Coder-Next
hidden_size	4096	2048
num_layers	60	48
num_experts	512	512
expert_intermediate	1024	512
expert_size (4-bit)	7.08 MB	1.77 MB
total on disk	209 GB	44 GB

Status

• 8-bit gate dequant (Metal + CPU)
• Fused projection detection and deinterleave
• Chat template wrapping
• Expert repacking from switch_mlp format
• CPU-only path verified correct
• GPU pipeline CMD2 buffer dimension fix (WIP)

Test

# CPU-only path (correct output, ~0.3 tok/s)
./infer --model <path> --prompt "What is 2+2?" --tokens 20 --k 4
# With g_metal=NULL forced in code

[userFRM]

Closing — splitting into separate PRs. The 8-bit gate dequant fix will be submitted standalone.