[Feature]: Support serving ModelOpt W4A8 MXFP4+FP8 checkpoints

[zeryx]

🚀 The feature, motivation and pitch

Some users have interest in w4a8 quantization configurations, in mxfp4-mxfp8 weight and activation storage. And are not interested in the GPTOSS model family.
modelopt allows for this configuration and provides support to quantize to this format, but vllm is unable to use the quantized weights via vllmconfig.

Models tested: (qwen3-30B-A3B)

vLLM v0.16.0 has FlashInfer MXFP4+MXFP8 MoE kernels for Blackwell (SM100), but the only tools that can produce compatible checkpoints are:

Quantizer	Checkpoint Format	vLLM Path	Native MXFP4 MoE on B200?
OpenAI (native training)	gpt-oss OCP MX format	quantization="mxfp4"	Yes — designed for gpt-oss
AMD Quark	OCP MX (HF export)	quantization="quark"	Yes — Quark MXFP4 docs
llm-compressor	compressed-tensors MXFP4	auto-detected → Marlin	No — CompressedTensorsW4A4Mxfp4MoEMethod hardcodes NvFp4MoeBackend.MARLIN
NVIDIA ModelOpt	W4A8_MXFP4_FP8_CFG	Rejected by QUANT_ALGOS	No — blocked at config parsing

The mxfp4.py quantization module (with its FlashInfer MXFP4+MXFP8 backends) was built for OpenAI's gpt-oss models through a [collaboration between NVIDIA, OpenAI, and AMD]

This means there is currently no NVIDIA PTQ tool (ModelOpt or llm-compressor) that can produce checkpoints which reach the native FlashInfer MXFP4 MoE kernels on Blackwell.

Current Behavior

Quantization (works):

import modelopt.torch.quantization as mtq
mtq.quantize(model, mtq.W4A8_MXFP4_FP8_CFG, forward_loop)
export_hf_checkpoint(model, export_dir="./checkpoint")

Serving (fails):

from vllm import LLM
llm = LLM(model="./checkpoint", quantization="modelopt")

Error:

1 validation error for VllmConfig
  Value error, ModelOpt currently only supports: ['FP8', 'FP8_PER_CHANNEL_PER_TOKEN',
  'FP8_PB_WO', 'NVFP4'] quantizations in vLLM. Please check the `hf_quant_config.json`
  file for your model's quant configuration.
  [type=value_error, input_value=ArgsKwargs((...)), input_type=ArgsKwargs]

In v0.16.0, MXFP8 was added to the whitelist (PR #33786), but the quant_algo produced by W4A8_MXFP4_FP8_CFG still doesn't match any entry

Additional context

Existing kernel support (unreachable from ModelOpt)

vLLM v0.16.0 has FlashInfer MoE kernels for MXFP4 weights with MXFP8 activations in the standalone mxfp4 quantization method (vllm/model_executor/layers/quantization/mxfp4.py):

class Mxfp4Backend(Enum):
    SM100_FI_MXFP4_MXFP8_TRTLLM = 1    # FlashInfer TRT-LLM kernel
    SM100_FI_MXFP4_MXFP8_CUTLASS = 2   # FlashInfer CUTLASS kernel
    SM100_FI_MXFP4_BF16 = 3            # BF16 fallback
    MARLIN = 5                          # Marlin decompression fallback

These kernels dynamically quantize activations to MXFP8 at runtime via flashinfer.mxfp8_quantize(). They're gated behind VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8=1.

However, the mxfp4 method was built for gpt-oss and expects the OCP MX checkpoint format (PR #16943). The weight layout differs

	mxfp4 method (gpt-oss/Quark)	ModelOpt export_hf_checkpoint()
MoE weights	w13_weight, w2_weight as raw nn.Parameter	hf_quant_config.json + safetensors
Scales	w13_weight_scale, w2_weight_scale (uint8 E8M0)	Per-block scales in ModelOpt format
Biases	w13_bias, w2_bias (required)	Not exported
Linear layers	Unquantized (BF16)	Quantized
Config detection	get_config_filenames() returns [] — must pass quantization="mxfp4" explicitly	Auto-detected via hf_quant_config.json

However, the mxfp4 method expects Quark/OCP checkpoint format (different weight layout from ModelOpt's export_hf_checkpoint output), so ModelOpt checkpoints can't use this path either.

Summary of the gap

Component	ModelOpt MXFP4+FP8	Status
Quantization (ModelOpt)	W4A8_MXFP4_FP8_CFG	Works
Export (ModelOpt)	export_hf_checkpoint()	Works
Config parsing (vLLM)	QUANT_ALGOS whitelist	Rejected
Linear kernels (vLLM)	Not implemented for this format	Missing
MoE kernels (vLLM)	FlashInfer MXFP4+MXFP8 exists but in mxfp4 path	Wrong path

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!