Skip to content

NVFP4EffMA 0.4.0

Choose a tag to compare

View all tags
@srose69 srose69 released this 21 Oct 10:53
· 6 commits to main since this release
v0.4.0
1da403b

NVFP4EffMA 0.4.0

NVFP4 GEMM, GEMV and quantization for NVIDIA GPUs without native FP4 (sm_50 … sm_90).

Install

PyTorch: pick the wheel matching your Python (cpXY), PyTorch (torchA.B) and its CUDA (cuNNN):

pip install nvfp4effma-0.4.0+<cuNNNtorchA.B>-<cpXY>-<cpXY>-linux_x86_64.whl

Other combinations: pip install ./binds/python --no-build-isolation from the source archive.

C / C++: unpack nvfp4effma-0.4.0-linux-x86_64-cuda12.tar.xz (tar -xJf) and link lib/libnvfp4effma.a (see its README).

Every artifact carries SASS for sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86, sm_87, sm_89, sm_90 (CUDA 12.0).

Assets

  • nvfp4effma-0.4.0-linux-x86_64-cuda12.tar.xz (8,6M)
  • nvfp4effma-0.4.0-src.tar.gz (136K)
  • nvfp4effma-0.4.0+cu126torch2.7-cp312-cp312-linux_x86_64.whl (57M)

What's in it

For code that builds on NVFP4EffMA:

  • Int8Tensor and quantize_int8(x): the int8 activations of linear(..., act="int8") (one FP32 scale per 16 values) as a public type. linear takes an Int8Tensor as x (bit for bit act="int8"), so quantized activations can be kept for a backward pass without reaching into the extension; act="int8" itself now goes through it.
  • get_include(): the wheel ships nvfp4effma.h and the device-side codecs nvfp4effma.cuh (E2M1 / E4M3 decode and RNE encode, scale addressing), so CUDA extensions can build against the installed package. The C archive carries them too (src/).

Kernels are unchanged from 0.3.0.

Assets 6
Loading