[RFC]: Intel Quantization Support Roadmap (H1 2026)

[yiliu30]

Status

Scheme Status

• 🙋 help wanted

Scheme	Module	Intel GPU	CPU
wNa16 INT (W4A16)	Linear	✅ Merged #37986 @yiliu30 auto_round:auto_gptq; ✅ Merged #43404 @Liangliang-Ma auto_round:auto_awq	✅ Merged #38192 @Zhenzhong1
wNa16 INT (W4A16, ARK)	Linear	✅ Merged #39778 @Zhenzhong1	✅ Merged #39778 @Zhenzhong1
wNa16 INT (W4A16)	MoE	✅ Merged #47124 @lkk12014402
w4a4 MXFP4	Linear	✅ Merged #47124 @lkk12014402
w8a8 MXFP8	Linear	✅ Merged #47514 @Zhenzhong1
w8a8 FP8	Linear	WIP #47434 @Zhenzhong1
w4a4 MXFP4	MoE	✅ Merged #47124 @lkk12014402
w8a8 MXFP8	MoE	Planed @jl9876
w8a8 FP8	MoE	🙋
wNa16 INT (W2A16)	Linear	✅ Merged #47521 @Zhenzhong1

Architectural Cleanup (Done)

#40601 @yiliu30

---

Original RFC

Related RFCs

• Consolidate Intel Quantization Toolkit Integration in vLLM
• XPU Kernel Migration to vllm-xpu-kernels
• Deprecate Legacy Quantization Formats

Motivation

Previously, we merged auto_round.py into inc.py as the unified Intel quantization backend. The vllm-xpu-kernels replacement for IPEX is a work in progress. This H1 2026 roadmap covers completing the consolidation, migrating from IPEX to vllm-xpu-kernels, and extending quantization scheme coverage on Intel CPU and XPU platforms.

Goals

1. Broad scheme coverage — support the quantization formats that matter for Intel hardware (wNa16 INT, w8a16 FP8), for both Linear and MoE layers, on both XPU and CPU.
2. Architectural cleanup — decouple the quant_method dispatch logic from INCConfig so INC acts purely as a config translator, not a kernel router.

1. Extend Quantization Scheme Coverage

Expand Intel platform support for the quantization schemes needed by production workloads.

1a. wNa16 (INT) — Weight-Only Integer Quantization

• XPU: Linear (W4A16), MoE (W4A16)
• CPU: Linear (W4A16), MoE (W4A16)

1b. w8a16 (FP8) — FP8 Weight-Only Quantization

• XPU: Linear (W8A16 FP8), MoE (W8A16 FP8)
• CPU: Linear (W8A16 FP8), MoE (W8A16 FP8)

Note: Some schemes may depend on kernel readiness.

2. Architectural Cleanup

Decouple the quant_method dispatch logic from INCConfig. Today, INCConfig.get_quant_method() contains per-backend routing that duplicates logic already in GPTQ/AWQ/Marlin configs:

get_quant_method()
├── is_cpu/is_xpu/ipex?  → apply_ipex_quant_layer()   # dead code
├── gptq format?         → apply_gptq_quant_layer()    # duplicates GPTQConfig
└── awq format?          → apply_awq_quant_layer()     # duplicates AWQConfig

Proposed Refactoring

Replace the monolithic get_quant_method() with a two-level dispatch architecture, inspired by the compressed-tensors design:

Level 1 — Module-type method (AutoRoundQuantLinearMethod, AutoRoundMoEMethod):
Implements the vLLM method interface (LinearMethodBase / FusedMoEMethodBase). A static factory (get_method() / get_moe_method()) resolves the per-layer quantization scheme and delegates to the correct Level 2 impl.

Level 2 — Scheme-specific impl (e.g. AutoRoundWNA16LinearImpl, AutoRoundFP8LinearImpl):
Implements an abstract AutoRoundQuantImpl base class that defines create_weights(), process_weights_after_loading(), and apply_weights(). Each impl owns a single quantization scheme and its kernel calls.

INCConfig.get_quant_method(layer, prefix)
│
├── LinearBase  → AutoRoundQuantLinearMethod.get_method(config, layer, prefix)
│                   ├── wNa16 INT  → AutoRoundWNA16LinearImpl
│                   └── w8a16 FP8  → AutoRoundFP8LinearImpl
│
└── FusedMoE   → AutoRoundMoEMethod.get_moe_method(config, layer, prefix)
                    ├── wNa16 INT   → AutoRoundWNA16MoEImpl
                    └── w8a16 FP8   → AutoRoundFP8MoEImpl

Feedback Period.

Please comment on the proposal or suggest alternatives. If there are no strong objections, we will proceed with the timeline above and submit implementation PRs. Thanks!

CC List.

cc @hshen14 @thuang6 @wenhuach21 @Zhenzhong1 @jikunshang @xinyu-intel @xuechendi
cc @robertgshaw2-redhat

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Zhenzhong1]

intel/auto-round#1446 Develop Status:

[update ARK API] (yiliu30#118)

WNA16 Linear:

• [Merged] ARK XPU & CPU -INT4-WOQ-Linear [Quantization][Autoround][Toolkit] Add W4A16 Support #39778
• [Under Review] [Quantization][INC] Support INT2 XPU Linear yiliu30/vllm-fork#114

FP8 Linear:

• [Under Review][Quantization][INC] Support AR FP8 Block Linear on CUDA yiliu30/vllm-fork#112
• [Refactoring 90% WIP] ARK XPU FP8 WOQ Linear - branch zhenzhong/w8a16_ark

MXFP8 Linear:

• [Under Review] [Quantization][INC]Add MXFP8 Linear Support yiliu30/vllm-fork#117

[wenhuach21]

intel/auto-round#1446 Status:

• ARK XPU-INT4-WOQ-Linear [Quantization][Autoround][ARK] Add W4A16 Support #39778

how about cpu int2?

[Zhenzhong1]

intel/auto-round#1446 Status:

• ARK XPU-INT4-WOQ-Linear [Quantization][Autoround][ARK] Add W4A16 Support #39778

how about cpu int2?

INT2 XPU we don't support it now.

[wenhuach21]

intel/auto-round#1446 Status:

• ARK XPU-INT4-WOQ-Linear [Quantization][Autoround][ARK] Add W4A16 Support #39778

how about cpu int2?

INT2 XPU we don't support it now.

how about cpu?

[Zhenzhong1]

@wenhuach21

oh....We have the CPU INT2 pytest case. I think we can support it.

I will check this scheme inference in vLLM later.

[Zhenzhong1]

intel/auto-round#1446 Develop Status:

• Develop Done. ARK XPU-INT4-WOQ-Linear [Quantization][Autoround][ARK] Add W4A16 Support #39778
• Develop Done. ARK CPU-INT4-WOQ-Linear yiliu30@02501d6
• Develop Done. ARK XPU FP8 WOQ Linear - branch zhenzhong/w8a16_ark
• Not Support. CPU: Linear (W8A16 FP8), MoE (W8A16 FP8)

I haven't seen any ARK test cases covering CPU FP8 linear and MoE. Could you please confirm CPU FP8 ARK is not supported? @luoyu-intel

It means there is no CPU FP8 ARK version. @yiliu30

[luoyu-intel]

@Zhenzhong1 ARK has implemented CPU fp8 linear kernel. But we didn't add a test for CPUs.

[ixcans]

Hi team, Is a CPU-backend path for TurboQuant KV-cache quantization on the roadmap?

The merged implementation (#38280) and the ROCm/Metal ports are all GPU-kernel based (Triton/CUDA/HIP/Metal). For AMX-class Xeon CPU-only deployments serving long-context MoE models, KV-cache memory/bandwidth is the binding constraint, and TurboQuant's dequantize-then-BF16-attend design looks like it could map onto the torch_sdpa / IPEX CPU attention path without GPU kernels. Could the encode/decode + pre-dequant step be exposed for the CPU backend?

Thanks.