[Feature]: Add SM120 (RTX 6000/5000 Blackwell) support for native NVFP4 MoE kernels

[Ramalama2]

🚀 The feature, motivation and pitch

Summary

SM120 (RTX 6000 Pro Blackwell, compute capability 12.0) is not recognized in the MXFP4 backend selection logic, causing fallback to Marlin instead of using native NVFP4 kernels.

Environment

• vLLM version: 0.13.0
• GPU: NVIDIA RTX 6000 Pro (Blackwell, SM120a)
• Compute Capability: (12, 0)
• Model: gpt-oss-120b (MXFP4 quantization)

Current Behavior

Startup log shows:
WARNING [marlin_utils_fp4.py:226] Your GPU does not have native support for FP4
computation but FP4 quantization is being used. Weight-only FP4 compression will
be used leveraging the Marlin kernel. This may degrade performance for compute-heavy workloads.

And:
INFO [mxfp4.py:161] Using Marlin backend

Root Cause

The MXFP4 backend selection in vllm/model_executor/layers/quantization/mxfp4.py only checks for SM100 family:

# Line 121-126
elif (
    current_platform.is_device_capability_family(100)  # Only matches SM10x
    and has_flashinfer()
    ...

SM120 ((12, 0)) doesn't match is_device_capability_family(100) since it's a different major version.

Similarly, the Triton backend check excludes SM120:
Line 96: and (9, 0) <= current_platform.get_device_capability() < (11, 0)

Evidence of SM120 Kernel Support

The NVFP4 SM120 kernels ARE compiled and available in vLLM 0.13.0:
-> csrc/quantization/fp4/nvfp4_scaled_mm_sm120_kernels.cu
-> csrc/quantization/fp4/nvfp4_blockwise_moe_kernel.cu (with ENABLE_NVFP4_SM120)
-> Commit e502098 - "[Kernel] Add NVFP4 MoE CUTLASS support for SM120"
-> Commit c0dfc89 - "SM120 / NVFP4: add device guard and runtime SM dispatch"

The support check also returns True:

>>> from vllm._custom_ops import cutlass_scaled_mm_supports_fp4
>>> cutlass_scaled_mm_supports_fp4(120)
True

Expected Behavior

SM120 should use native NVFP4 kernels similar to SM100, not fall back to Marlin.

Suggested Fix

Add SM120 family detection to get_mxfp4_backend() in mxfp4.py:

elif (
    (current_platform.is_device_capability_family(100) or
     current_platform.is_device_capability_family(120))
    and has_flashinfer()
    ...

Or create dedicated SM120 backend paths if the kernel interfaces differ.

Affected Hardware

• NVIDIA RTX 6000 Pro (Blackwell) - SM120
• NVIDIA RTX 5090/5080 (Blackwell) - SM120
• Other consumer/pro Blackwell cards using SM120/SM120a

Related Commits

• e502098 - [Kernel] Add NVFP4 MoE CUTLASS support for SM120
• c0dfc89 - SM120 / NVFP4: add device guard and runtime SM dispatch
• a00d889 - [EPLB] Support EPLB w/ NVFP4

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[mtcl]

This will be very useful. following this one.

please also upgrade official docker image to cu130

[mratsim]

I'm confused.

GPT-OSS-120B is mxfp4 not nvfp4 and cannot use NVFP4 code paths as data is stored very differently between mxfp4 and nvfp4.

There are no warnings for NVFP4 moe models, and there are spurious ones for NVFP4A16 (#27471).

[Ramalama2]

I'm confused.

GPT-OSS-120B is mxfp4 not nvfp4 and cannot use NVFP4 code paths as data is stored very differently between mxfp4 and nvfp4.

There are no warnings for NVFP4 moe models, and there are spurious ones for NVFP4A16 (#27471).

You're right, I conflated MXFP4 with NVFP4 - apologies for the confusion.

The actual issue is that MXFP4 on SM120 falls back to Marlin because the Triton
backend check excludes SM120:

# Line 157-158 in mxfp4.py
# SM120 needs this fix: https://github.com/triton-lang/triton/pull/8498
and (9, 0) <= current_platform.get_device_capability() < (11, 0)

That Triton PR (triton-lang/triton#8498) was merged on Oct 24, 2025.
Should SM120 now be enabled for the Triton MXFP4 backend, or is there
additional validation needed?

Otherwise the (#27471) issue, is correct either, but as far i understand its just about the warning message itself, not about fixing the root cause.

[vgoklani]

[etog-yu]

https://forums.developer.nvidia.com/t/your-gpu-does-not-have-native-support-for-fp4-computation-but-fp4-quantization-is-being-used/355494/2
This comment here notes that Triton does not offer a speedup over Marlin for gpt-oss. I have not tested it myself.

[Ramalama2]

https://forums.developer.nvidia.com/t/your-gpu-does-not-have-native-support-for-fp4-computation-but-fp4-quantization-is-being-used/355494/2 This comment here notes that Triton does not offer a speedup over Marlin for gpt-oss. I have not tested it myself.

i did tested, i fixed vllm for gpt-oss 120b with claude code, took me a day i think.
And yes Triton is much slower as Marlin.

With Marlin im getting 185 token/s generation speed (single user), and with Triton 140 token/s on a RTX 6000 Max-Q (300W):
#27542 (comment)

But this is because NVFP4/FP4 isn’t being “properly utilized”. I readed in the Nvidia Forums somewhere that there should be actually much more to gain. There is a similar issue with the DGX Spark.

So my hope is to fix the usage of the Triton Kernel properly.
If its properly fixed and is still slower as Marlin, we can still use the "VLLM_MXFP4_USE_MARLIN=1" Environment variable for the old behaviour.

[sininspira2]

https://forums.developer.nvidia.com/t/your-gpu-does-not-have-native-support-for-fp4-computation-but-fp4-quantization-is-being-used/355494/2 This comment here notes that Triton does not offer a speedup over Marlin for gpt-oss. I have not tested it myself.

i did tested, i fixed vllm for gpt-oss 120b with claude code, took me a day i think. And yes Triton is much slower as Marlin.

With Marlin im getting 185 token/s generation speed (single user), and with Triton 140 token/s on a RTX 6000 Max-Q (300W): #27542 (comment)

But this is because NVFP4/FP4 isn’t being “properly utilized”. I readed in the Nvidia Forums somewhere that there should be actually much more to gain. There is a similar issue with the DGX Spark.

So my hope is to fix the usage of the Triton Kernel properly. If its properly fixed and is still slower as Marlin, we can still use the "VLLM_MXFP4_USE_MARLIN=1" Environment variable for the old behaviour.

Been beating my head against the wall trying to get NVFP4 to work in VLLM with the sm120 blackwells. Do you mind sharing how you ended up doing it?

[Ramalama2]

https://forums.developer.nvidia.com/t/your-gpu-does-not-have-native-support-for-fp4-computation-but-fp4-quantization-is-being-used/355494/2 This comment here notes that Triton does not offer a speedup over Marlin for gpt-oss. I have not tested it myself.

i did tested, i fixed vllm for gpt-oss 120b with claude code, took me a day i think. And yes Triton is much slower as Marlin.
With Marlin im getting 185 token/s generation speed (single user), and with Triton 140 token/s on a RTX 6000 Max-Q (300W): #27542 (comment)
But this is because NVFP4/FP4 isn’t being “properly utilized”. I readed in the Nvidia Forums somewhere that there should be actually much more to gain. There is a similar issue with the DGX Spark.
So my hope is to fix the usage of the Triton Kernel properly. If its properly fixed and is still slower as Marlin, we can still use the "VLLM_MXFP4_USE_MARLIN=1" Environment variable for the old behaviour.

Been beating my head against the wall trying to get NVFP4 to work in VLLM with the sm120 blackwells. Do you mind sharing how you ended up doing it?

Same here, it took me a day.
But sadly no. I did indeed logged my changes that i made with claude to a changes.txt file.
But as my results with triton were bad (not what i expected), i dropped the entire idea and deleted that log file :-(

Afterwards this was maybe not that clever, but i thought at that time, if someone is going to fix it, he will understand what he does compared to me and do it properly.

Sorry