W8A8 INT8 support for Qwen3.6-35B-A3B (Qwen3.5 MoE architecture)

[Priyjain-amd]

We need to quantize Qwen/Qwen3.6-35B-A3B to W8A8 (INT8 weights with static per-channel scales, INT8 activations with dynamic per-token scales) for deployment on INT8-capable GPUs (Turing and newer), without using NVFP4 or FP8.

Today, LLM Compressor does not provide a clear, tested path for this:

1. No W8A8 MoE example for Qwen3.6 — NVFP4 and FP8 examples exist; W8A8 does not.
2. Missing architecture mappings — Qwen3_5MoeForConditionalGeneration and related classes are not registered in SmoothQuant and static AWQ mapping tables, so calibration-based flows may use incorrect default mappings.
3. MoE and hybrid attention — Qwen3.6 expert weights are stored as fused 3D tensors in Hugging Face. Quantization must unfuse them into per-expert nn.Linear modules. The model also mixes full self-attention and linear attention (Gated DeltaNet); we need W8A8 on MoE expert Linears and linear-attention Linears, not only dense attention layers.

Qwen3.6 uses the same stack as Qwen3.5 (Qwen3_5MoeForConditionalGeneration, Qwen3_5MoeSparseMoeBlock). MoE calibration code already exists in qwen3_5_moe.py, but W8A8 support still needs registry entries, documentation, and a runnable example.

[brian-dellabetta]

Hi @Priyjain-amd , the PR to add Qwen 3.5 mappings to smoothquant is

• build SmoothQuant mapping dynamically for Qwen3.5 MoE hybrid attention #2718

It should be fairly trivial to create a script from the examples by combining the Qwen 3.6 model docs with an example using the W8A8 scheme like this one. You can try from #2718 if you'd like to use SmoothQuant, or you can just skip that in recipe until it's in main

[dsikka]

@Priyjain-amd We don't typically add examples for every quantization scheme but as @brian-dellabetta pointed out, W8A8 is supported, you'll just need to update the examples to swap schemes and potentially use a different modifier, if that is what you're looking to apply. This model, like Qwen3.5, has had calibration support for a few weeks now.