why nvfp4 2D 2d quantization' s scale has the same shape as 1d quantization's scale

[fy1214]

I use the test_nvfp4_quantize_exact.py to checkout the scale of 2d quantization, when i put M, N=256,1024 in the test i found out that the scale's shape are [256, 64]. Also the 1d quantization is the same shape. Is there any reason for that?

[ptrendx]

You are correct, the scale shape is the same. The computed 2D scale is broadcasted to that shape in order for the resulting FP4 tensor to be compatible with the regular FP4 gemm.

[fy1214]

You are correct, the scale shape is the same. The computed 2D scale is broadcasted to that shape in order for the resulting FP4 tensor to be compatible with the regular FP4 gemm.

Thank u. So is this for FP4 gemm tensor core? Is there any doc for this.

[ptrendx]

You can see the scaling format support of cuBLAS here: https://docs.nvidia.com/cuda/cublas/index.html#scaling-mode-support-overview

Keep in mind that we do the required swizzling of the scaling factor just before the GEMM, the ones stored in the tensor are unswizzled.

[ptrendx]

Closing this issue - please reopen if you have more questions :-)!

[fy1214]

bool use_optimized_kernel = dtype == DType::kBFloat16 && rows % 32 == 0 && cols % 32 == 0 &&
                                  output_tensor->has_data();

I want to know why the nvfp4_quantize_transpose kernel only use in bf16. Becasue the ptx only write reg.b16? @ptrendx