Summary
Investigate and, if feasible, extend Olive's AWQ pass (olive/passes/pytorch/autoawq.py) to support MoE fused-expert weights, matching the RTN pass's moe=True support landed in microsoft/Olive#2584.
Why this is a different shape of problem than GPTQ/RTN
Unlike rtn.py and gptq.py (both native Olive implementations built on olive/common/quant/), autoawq.py is a thin wrapper around the external autoawq PyPI package (AutoAWQForCausalLM.from_pretrained(...).quantize(...)). Olive does not control the internal weight-replacement/calibration logic — it only configures and invokes the library.
What needs investigating (before any design work)
- Does the currently-pinned
autoawq version already support quantizing MoE architectures we care about (Mixtral, gpt-oss, Qwen3-MoE, DeepSeek, etc.)? If yes for some subset, what output format does it produce for expert weights, and does it match (or can it be mapped to) Olive's QuantTensor/on-disk buffer convention (olive/common/quant/state_dict.py), or does it need its own separate output path?
- If
autoawq does not support MoE at all, is upstream support planned/in-progress, or would this require vendoring/patching behavior — and is that worth pursuing given autoawq is a third-party dependency we don't control?
- Scope check: does "AWQ support MoE" mean literally the
autoawq wrapper, or would a native olive-side AWQ implementation (mirroring the gptq.py vs autogptq.py split) be more appropriate long-term? Worth a quick decision before committing to either wrapper-config-passthrough or a bigger native reimplementation.
References
Summary
Investigate and, if feasible, extend Olive's AWQ pass (
olive/passes/pytorch/autoawq.py) to support MoE fused-expert weights, matching the RTN pass'smoe=Truesupport landed inmicrosoft/Olive#2584.Why this is a different shape of problem than GPTQ/RTN
Unlike
rtn.pyandgptq.py(both native Olive implementations built onolive/common/quant/),autoawq.pyis a thin wrapper around the externalautoawqPyPI package (AutoAWQForCausalLM.from_pretrained(...).quantize(...)). Olive does not control the internal weight-replacement/calibration logic — it only configures and invokes the library.What needs investigating (before any design work)
autoawqversion already support quantizing MoE architectures we care about (Mixtral, gpt-oss, Qwen3-MoE, DeepSeek, etc.)? If yes for some subset, what output format does it produce for expert weights, and does it match (or can it be mapped to) Olive'sQuantTensor/on-disk buffer convention (olive/common/quant/state_dict.py), or does it need its own separate output path?autoawqdoes not support MoE at all, is upstream support planned/in-progress, or would this require vendoring/patching behavior — and is that worth pursuing givenautoawqis a third-party dependency we don't control?autoawqwrapper, or would a native olive-side AWQ implementation (mirroring thegptq.pyvsautogptq.pysplit) be more appropriate long-term? Worth a quick decision before committing to either wrapper-config-passthrough or a bigger native reimplementation.References
jambayk/moe-quant: extend RTN weight quantization to MoE experts #2583