[Feature]: support serving nvfp4 W4A16 moe models uisng Marlin

[EdalatiAli]

Purpose

This PR enables serving nvfp4 W4A16 compressed-tensors quantized MoE models by adding CompressedTensorsW4A16Nvfp4MoeMethod.
Weight-only nvfp4 quantization improves the quality at the expense of higher latency at large concurrencies.
As the provided results show, nvfp4 W4A16 quantized version of Qwen/Qwen3-30B-A3B singificantly outperforms the nvfp4 W4A4 variant.

Test Plan

Running e2e quality test on Qwen3-30B-A3B

1. BF16: Qwen/Qwen3-30B-A3B
2. nvfp4 W4A4: RedHatAI/Qwen3-30B-A3B-NVFP4
3. nvfp4 W4A16: AliEdalati97/Qwen3-30B-A3B-NVFP4-W4A16

To run evalaution:

1. Serve the target model using

vllm serve  <MODEL_HF_ID> --port 8000

2. Evaluate on mmlu_pro using lm-eval[api]==0.4.9.1

lm_eval \
  --model local-completions \
  --tasks mmlu_pro \
  --model_args \
base_url=http://0.0.0.0:8000/v1/completions,\
model=<MODEL_HF_ID>,\
tokenized_requests=False,tokenizer_backend=None,\
num_concurrent=128,timeout=120,max_retries=5

Test Result

Here are the results for

1. BF16: Qwen/Qwen3-30B-A3B

Groups	Version	Filter	n-shot	Metric		Value		Stderr
mmlu_pro	2	custom-extract		exact_match	↑	0.692	±	0.004

2. nvfp4 W4A4: RedHatAI/Qwen3-30B-A3B-NVFP4

Groups	Version	Filter	n-shot	Metric		Value		Stderr
mmlu_pro	2	custom-extract		exact_match	↑	0.6499	±	0.0042

3. nvfp4 W4A16: AliEdalati97/Qwen3-30B-A3B-NVFP4-W4A16

Groups	Version	Filter	n-shot	Metric		Value		Stderr
mmlu_pro	2	custom-extract		exact_match	↑	0.6815	±	0.0041

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

Hi @EdalatiAli, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[gemini-code-assist]

Code Review

This pull request introduces support for serving nvfp4 W4A16 quantized Mixture-of-Experts (MoE) models using the Marlin kernel by adding the CompressedTensorsW4A16Nvfp4MoeMethod. The changes look good overall, but I've identified a potential issue in how device capabilities are checked. The check is hardcoded to device 0, which could be problematic in multi-GPU environments. My review includes suggestions to make this more robust by using the layer's device for the capability check.

[gemini-code-assist]

The device capability check is hardcoded to device 0. This might be incorrect in a multi-GPU setup where the layer is on a different device. It's better to get the device from the layer object to ensure the check is performed for the correct GPU.

Suggested change

	if current_platform.get_device_capability(0).major < 10:
	if current_platform.get_device_capability(layer.weight.device.index).major < 10:

[gemini-code-assist]

Similar to the issue in prepare_fp4_layer_for_marlin, the device capability check is hardcoded to device 0. This should be based on the layer's actual device to handle multi-GPU scenarios correctly.

Suggested change

	if current_platform.get_device_capability(0).major < 10:
	if current_platform.get_device_capability(layer.w13_weight.device.index).major < 10:

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Add post-load handling for nvfp4 W4A16 MoE weights

The new CompressedTensorsW4A16Nvfp4MoeMethod calls fused_marlin_moe with layer.w13_weight/layer.w2_weight here, but create_weights only registers w13_weight_packed/w2_weight_packed and this class never overrides process_weights_after_loading (the base implementation is a no-op, unlike the W4A4 MoE method). After loading a checkpoint those attributes are never created, so the first inference request using this quantization path will raise AttributeError before any computation runs. A post-load hook is needed to unpack or alias the packed weights before apply.

Useful? React with 👍 / 👎.

[mergify]

Hi @EdalatiAli, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[dsikka]

FYI - #30881 This PR enables the NVFP4A16 pathway using the existing class which I think is preferable