Is MegaMoE currently limited to DeepSeek models? Any plans to support other architectures like GLM?

[zcfh]

Hi team,
I'm trying to use MegaMoE with GLM-5.2 but running into issues. It seems the quantization scale differs from what MegaMoE expects, so it doesn't work out of the box.
A few questions:

1. Is MegaMoE currently only supported on DeepSeek-series models?
2. Would re-quantizing GLM-5.2 with a matching scale potentially fix the issue, or are there other incompatibilities I should be aware of?
3. Is there any plan on the roadmap to officially support GLM?

Thanks!

[Shyboy0499]

From the current source, MegaMoE is wired up and validated for the DeepSeek family — there's no explicit "DeepSeek-only" gate, but everything around it is DeepSeek-oriented, so GLM (and other architectures) don't work out of the box.

What MegaMoE actually is (from python/sglang/srt/arg_groups/mega_moe_hook.py):

• A MoE runner / A2A (all-to-all) backend, selected via --moe-runner-backend megamoe (an alias that sets --moe-a2a-backend megamoe).
• A W4A4 MXFP4 quantization path, gated by --enable-w4a4-mxfp4-megamoe, which sets DG_USE_FP4_ACTS=1 and DG_USE_MXF4_KIND=1 (DeepGEMM FP4 activations + MXF4 format).

Why GLM-5.2 fails and what it takes to make it run:

1. The weight layout must match. The FP4 path expects weights in the MXFP4 block format (MXF4 kind). GLM ships with a different quantization scale/format, so the layers don't load as-is — exactly the mismatch you're hitting.
2. Re-quantizing GLM-5.2 to the matching MXFP4 layout is the necessary precondition. With weights converted to the format MegaMoE expects, the A2A + FP4 kernel path can run. But it may not be sufficient — the path is tuned and validated on DeepSeek-V4-style MoE configs (see the e2e test test/registered/models_e2e/test_deepseek_v4_flash_fp4_megamoe_b200.py), so other config assumptions (MoE group sizes, heads, etc.) may still bite.
3. Roadmap: there's no GLM support wired up today — mega_moe_hook.py and deepseek_v4_hook.py are the entire current scope. The right move is to file an issue/feature request on the sglang repo; maintainers are the only source of truth on timing.

Practical next step: convert GLM-5.2 to the MXFP4 block layout, launch with --moe-a2a-backend megamoe --enable-w4a4-mxfp4-megamoe, and see how far it gets. If it fails on config assumptions rather than weights, that's a clean signal to file the feature request with the exact error.