mlops-norm 0.1.0

mlops-norm 0.1.0

Part of mlops: CUDA operations for NVIDIA GPUs sm_50 … sm_90.

Install

Nothing here downloads anything. Install first:

• PyTorch (CUDA build);

Then, in the directory with the downloaded wheel (cpXY = your Python, torchA.B / cuNNN = your PyTorch and its CUDA):

pip install --no-index -f . mlops-norm

--no-index takes mlops-norm from this directory and its dependencies from what is installed, never from PyPI.
Other combinations: ./setup.sh norm from the source archive.

C / C++: unpack mlops-norm-0.1.0-linux-x86_64-cuda12.tar.xz (tar -xJf) and link lib/libmlops_norm.a (see its README).

Every artifact carries SASS for sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86, sm_87, sm_89, sm_90 (CUDA 12.0), built against NVFP4EffMA 0.6.0.

Assets

• mlops-norm-0.1.0-linux-x86_64-cuda12.tar.xz (1,4M)
• mlops-norm-0.1.0-src.tar.gz (60K)
• mlops_norm-0.1.0+cu126torch2.7-cp312-cp312-linux_x86_64.whl (12M)

Changes

First release. RMSNorm and LayerNorm for sm_50 .. sm_90, with a residual add before and an INT8 or NVFP4 output after.

• Flavors. RMSNorm and LayerNorm; the weight, 1 + weight (Gemma, Qwen3-Next, LayerNorm1P) or none; a bias; Hugging
  Face's order of roundings (round(round(v r) w), Llama / Qwen) or a single rounding (Gemma, LayerNorm, biases,
  gates); silu gates before or after the norm (Mamba-2, Nemotron-H, Qwen3-Next); groups; x in BF16, FP16 or FP32 with
  weights in BF16 / FP16 / FP32. 105 kernels, none with spills on sm_61 .. sm_89.
• Quantized outputs. INT8 (amax / 127 per 16 values) and NVFP4 (E2M1 with E4M3 scales and a decode-factor
  global): the bytes nvfp4a_quantize of NVFP4EffMA makes of the same y, bit for bit, without a pass over y. The E4M3
  and E2M1 primitives are NVFP4EffMA's (its headers are a build dependency, nothing at run time); the E2M1 conversion
  is rewritten in fewer instructions and checked against NVFP4EffMA's for all 2^32 floats.
• Reproducible sums. The squares are summed in units of 256 elements in a fixed order (the same as mlops_embed and
  mlops_lm_head use), so the result does not depend on the variant, the element count or the GPU (the gates' exp
  aside). From sm_80 a BF16 pair is rounded by cvt.rn.bf16x2.f32, below it by integer instructions with the same
  results.
• Variants and items. mlops_norm_variants lists the instances (1, 2, 4, 8 warps) as data; mlops_norm_items.h
  describes a call as items for an executor, with their NVRTC entries and access ranges. A group is a multiple of 256
  values; per-head norms of 64 or 128 belong to mlops_fbla_qkv_at.
• Python mlops.norm: norm, rms_norm, layer_norm, RMSNorm, LayerNorm, norm_variants. order=None picks
  Hugging Face's order for an RMSNorm with a plain weight and no bias or gate, a single rounding otherwise.
• 2048 x 4096 BF16 RMSNorm: 160 us on a GTX 1080 (127 - 130 us with an INT8 or NVFP4 output), 20 us on an L40S (18 - 19 us).
• Tested: GTX 1080: the C tests (75 checks: bit for bit against a host emulation, the quantized bytes against
  nvfp4a_quantize, items through NVRTC), pytest (26), the samples, the wheel in a fresh venv, the C archive from plain
  C. sm_89 (48 GB): the C tests and pytest.