2026.4.16

Optimize MoE expert caching and fix AutoRound/TurboQuant quantization handling

What Changed

• Added _prepare_cached_experts to Qwen3SparseMoeBlock to prepare required experts in batch and avoid evicting experts that are still needed in the current routing pass.
• Updated the routing flow to preselect hot experts based on usage and reuse the prepared GPU expert cache instead of materializing experts one at a time.
• Fixed AutoRoundGPTQLinear qzeros unpacking logic to match the (zero_point - 1) storage format used by AutoRound/AutoGPTQ exports, and aligned the XPU int4 conversion path with the same behavior.
• Added a decode cache to TurboQuantKVRow so the quantized KV prefix is materialized once and reused across materialize and decode_attention.
• Properly invalidated the decode cache after appending new tokens, and carried the cached state through clone and to() operations.

Impact

• Reduces expert cache churn and staging overhead in MoE offload execution.
• Cuts repeated dequantization work during single-token decode.
• Fixes zero-point interpretation for AutoRound quantized weights and improves consistency across quantized execution paths.

---

优化 MoE 专家缓存，并修复 AutoRound/TurboQuant 量化处理

背景

当前 Qwen3SparseMoeBlock 在 expert offload 场景下，命中专家仍以逐个方式进行缓存拉取和淘汰，容易带来额外的 staging 开销。同时，TurboQuant 的 decode 路径会重复反量化量化前缀，AutoRoundGPTQLinear 对 AutoRound 导出的 qzeros 处理也存在兼容性偏差。

主要改动

• 为 Qwen3SparseMoeBlock 新增 _prepare_cached_experts，根据当前路由命中的 expert 批量准备缓存，并尽量避免淘汰本轮仍需使用的 expert。
• 调整路由执行流程，先基于 usage 选出热点 expert，再复用已准备好的 GPU expert 缓存，减少逐个 materialize 的开销。
• 修正 AutoRoundGPTQLinear 对 qzeros 的解包逻辑，兼容 AutoRound/AutoGPTQ 导出的 (zero_point - 1) 存储格式，并统一 XPU int4 转换时的零点处理。
• 为 TurboQuantKVRow 增加 decode cache，对量化后的 KV 前缀只 materialize 一次，并在 materialize 和 decode_attention 中复用。
• 在追加新 token 后正确失效 decode cache，并补齐 clone 和 to() 场景下的缓存复制与迁移逻辑。

影响

• 降低 MoE offload 场景下的 expert 缓存抖动和 staging 开销。
• 减少单 token decode 过程中重复反量化带来的额外计算。
• 修复 AutoRound 权重量化 zero-point 的解析偏差，提升量化执行结果的一致性。