KTransformers v0.6.3

KTransformers: MiniMax-M3 & GLM-5.2 Day0 Support

We're excited to announce Day0 support for two new large MoE models: MiniMax-M3 and GLM-5.2 in KTransformers + SGLang (sglang-kt). Both models are now available for CPU-GPU heterogeneous inference via kt-kernel. This update also ships end-to-end KT LoRA serving for Qwen3.5 MoE.

🚀 Core Highlights

• MiniMax-M3 Day0 Support: Full inference support for MiniMax's 128-routed-expert sparse MoE architecture (MiniMaxAI/MiniMax-M3-MXFP8). Runs on a single 96 GB GPU or scales to 8×GPU with kt-kernel CPU expert offload.
• GLM-5.2 Day0 Support: Full inference support for Zhipu AI's GLM-5.2 (zai-org/GLM-5.2, zai-org/GLM-5.2-FP8), with FP8 and BF16 precision, NSA attention backend, and dynamic expert updates via kt-kernel.
• KT LoRA Serving: End-to-end Qwen3.5 MoE KT LoRA serving workflow — train with KT SFT, convert once, serve through SGLang with a single merged adapter path.

📌 KT LoRA Serving for Qwen3.5 MoE

A new end-to-end fine-tuning-to-serving workflow for Qwen3.5 MoE with KT expert LoRA:

• Unified adapter pipeline: KT SFT raw output → convert_kt_to_sglang_adapter.py → merged adapter directory → SGLang --lora-paths serving.
• Expert + non-expert LoRA: the server auto-splits expert and non-expert LoRA weights internally; users serve a single merged adapter path.
• SFT kernel improvements: refactored AMX SFT MoE kernels, AVX2 MoE base updates, and tensor-parallel MoE SFT helpers.
• Docs: KT-FT Fine-Tuning and Inference Loop (EN) | 中文

📌 GLM-5.2

GLM-5.2 is Zhipu AI's latest MoE model with NSA (Native Sparse Attention). KTransformers supports it with:

• FP8 and BF16 precision: Both zai-org/GLM-5.2-FP8 and zai-org/GLM-5.2 are supported.
• NSA attention backend: Native sparse attention with skip_topk-gated indexer for DSA layers.
• Dynamic expert updates: --kt-enable-dynamic-expert-update with uniform placement strategy.
• Tool calling & reasoning: glm47 tool-call parser and glm45 reasoning parser.

👉 Full Tutorial

📌 MiniMax-M3

MiniMax-M3 is a 128-expert sparse MoE model shipped natively in MXFP8 (fp8 e4m3 + uint8 ue8m0 1×32 scale). KTransformers supports M3 end-to-end with:

• KT-Kernel hybrid: CPU expert offload with MXFP8 byte-copy and GPU shadow FP8 view; validated on 8×H20 and single H20.
• MXFP8: Direct MXFP8 inference path on Hopper GPUs via Triton MoE backend.
• Tool calling: minimax-m3 function-call parser (native <minimax:tool_call> XML → OpenAI tool_calls array).
• Thinking mode: Request-level control via chat_template_kwargs.thinking_mode — enabled / disabled / adaptive.

👉 Full Tutorial

📥 Installation

# Option A: One-click install (from ktransformers root)
git clone https://github.com/kvcache-ai/ktransformers.git
cd ktransformers
git submodule update --init --recursive
cd kt-kernel && ./install.sh
cd .. && ./install.sh

# Option B: pip install
pip install sglang-kt
pip install kt-kernel

Requires CUDA 12.0+; CUDA 12.8+ recommended for FP8/MXFP8 deployments.

📝 Changelog

ktransformers (kvcache-ai/ktransformers)

• feat: add end-to-end Qwen3.5 MoE KT LoRA serving workflow (#2031) — @jdai0
• [feat] MXFP8 MoE support (#2041) — MXFP8 MoE kernel for MiniMax-M3 Day0.
• fix: CudaGraph replay fix and Add MoE startup log (#2037).
• update glm52 tutorial (#2046) — @ovowei
• [build]: sync sglang submodule to 51032b71 (#2032).
• docs: add MiniMax-M3 SGLang + KT-Kernel tutorial (#2051) — @yyj6666667

sglang-kt (kvcache-ai/sglang)

• feat: support end-to-end KT LoRA serving for Qwen3.5 MoE (#53) — @jdai0
• support glm5.2 — @ovowei
• fix(dsa): wire skip_topk-gated indexer for GlmMoeDsa to unblock GLM-5.2 — @yyj6666667
• fix: kt_ep_wrapper silently fails to import after a2f451315 (#57).
• Feat/minimax m3 (#56): full MiniMax-M3 model port — @yyj6666667

🌟 Contributors

Thanks to all contributors who helped ship this release.

CC: @jdai0 @ovowei @ouqingliang @ErvinXie @chenht2022 @KMSorSMS @SkqLiao @JimmyPeilinLi @mrhaoxx @yyj6666667