E5M2EffMA 0.2.0

E5M2EffMA 0.2.0

FP8 (E5M2 and E4M3) GEMM, GEMV, casts and fused AdamW for NVIDIA GPUs without native FP8 (sm_50 … sm_87).

Install

PyTorch: pick the wheel matching your Python (cpXY), PyTorch (torchA.B) and its CUDA (cuNNN):

pip install e5m2effma-0.2.0+<cuNNNtorchA.B>-<cpXY>-<cpXY>-linux_x86_64.whl

Other combinations: pip install ./binds/python --no-build-isolation from the source archive.

C / C++: unpack e5m2effma-0.2.0-linux-x86_64-cuda12.tar.gz and link lib/libe5m2effma.a (see its README).

Every artifact carries SASS for sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86, sm_87 (CUDA 12.0).

Assets

• e5m2effma-0.2.0-linux-x86_64-cuda12.tar.gz (37M)
• e5m2effma-0.2.0-src.tar.gz (108K)
• e5m2effma-0.2.0+cu126torch2.7-cp312-cp312-linux_x86_64.whl (39M)

What's in it

• get_include(): the wheel ships e5m2effma.h and the device-side codecs e5m2effma.cuh (E5M2 / E4M3 decode and RNE encode), so CUDA extensions can build against the installed package instead of a source checkout. The C archive carries them too (src/).
• GEMM: register budget of the 128-thread configurations (128 × 64, 64 × 128). They were compiled for 6 blocks per SM (~80 registers) and spilled, 67–82 ms instead of 23–24 ms at 4096³ on a GTX 1080; they now target 4 × 128 threads (≤ 128 registers). The tuned tables do not pick them on the GTX 1080, so its default kernels are unchanged.
• README: using E4M3EffMA and E5M2EffMA together.

E5M2EffMA 0.1.0

E5M2EffMA 0.1.0

FP8 (E5M2 and E4M3) GEMM, GEMV, casts and fused AdamW for NVIDIA GPUs without native FP8 (sm_50 … sm_87).

Install

PyTorch: pick the wheel matching your Python (cpXY), PyTorch (torchA.B) and its CUDA (cuNNN):

pip install e5m2effma-0.1.0+<cuNNNtorchA.B>-<cpXY>-<cpXY>-linux_x86_64.whl

Other combinations: pip install ./binds/python --no-build-isolation from the source archive.

C / C++: unpack e5m2effma-0.1.0-linux-x86_64-cuda12.tar.gz and link lib/libe5m2effma.a (see its README).

Every artifact carries SASS for sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86, sm_87 (CUDA 12.0).

Assets

• e5m2effma-0.1.0-linux-x86_64-cuda12.tar.gz (37M)
• e5m2effma-0.1.0-src.tar.gz (108K)
• e5m2effma-0.1.0+cu126torch2.7-cp312-cp312-linux_x86_64.whl (39M)

What's in it

First release. The E5M2 sibling of E4M3EffMA, built on the same
kernels, taking E5M2 and E4M3 in any pairing.

• GEMM D = α·op(A)·op(B) + bias + β·C with cuBLASLt / Transformer Engine semantics: A, B ∈ {E5M2, E4M3, BF16, FP32}, D ∈ {E5M2, E4M3, BF16, FP16, FP32}, per-tensor or row-wise scales, amax for FP8 outputs, TE-style FP32 main_grad accumulation. The backward GEMMs of TE's HYBRID format (E5M2 gradients × E4M3 weights / activations) run as they are. Exact FP8 products, FP32 accumulation, one rounding.
• GEMV path for up to 8 columns (decoding), picked automatically, for E5M2 or E4M3 weights.
• Casts between E5M2, E4M3, BF16, FP16 and FP32: FP8 Inf / NaN codes decode exactly, FP32 → FP8 saturates as cvt.rn.satfinite, and amax reports overflow (loss scaling). Fused Adam / AdamW with FP32 or BF16 moments and an optional E5M2 or E4M3 copy of the weights.
• PyTorch: matmul / linear / cast / adam_step, e5m2effma.AdamW, torch.e5m2effma (= torch.float8_e5m2), torch._scaled_mm on pre-Ada GPUs for every FP8 pairing (shared with e4m3effma ≥ 0.1.1, import order does not matter), and Float8Tensor with E5M2 or E4M3 data.
• No stack or local memory in any kernel on any architecture.

On a GTX 1080: GEMM at cuBLAS SGEMM level per clock (77 % of FP32 peak), GEMV n = 1 at 89 % of DRAM bandwidth (4.4× cuBLAS SGEMV), AdamW with BF16 parameters and moments 2.65–2.79× torch's fused AdamW. Details: docs/performance.md, benches/.

Tested and tuned by the author on a GTX 1080 (sm_61) only; an RTX 3090 (sm_86) passed the test suite of E4M3EffMA, which shares these kernels, in a community run. Other architectures are compiled, not run.

The wheel here is built for Python 3.12, PyTorch 2.7 and CUDA 12.6.