v0.2.4 — differentiable grouped NF4 GEMM (training) + NVMe arena surface
v0.2.4 — the differentiable grouped NF4 GEMM (training half) + NVMe arena surface
nf4_qlora makes the fused kernel trainable: an autograd.Function whose backward
re-decodes one expert at a time, plus fused_grouped_lora composing the frozen
4-bit projection with the trainable B(Ax) delta at the pre-activation point.
Verified on real hardware before this tag:
- gradient correctness -- 5/5 on sm_86 and sm_89, gradient relative error
exactly 0.0 against the dequantize-then-matmul reference, forward 0.0027 - offload safety at 48 layers (Qwen3-30B-A3B, RTX 4090): 0.756x the reference
peak, no OOM, where holding weights on the ctx OOMed at 22.41 GB - frozen experts bit-identical after training -- 192 tensors, 16.3 GB hashed,
zero placeholders, with a positive control proving a single flipped byte IS
detected - loss parity with the reference on varied data and a held-out eval:
0.337073 vs 0.340271 (0.95%), train median delta 0.0136 (band 0.05)
Also: triton is now a Linux-only dependency, so the pure-torch provenance and
NVMe-arena surface installs on macOS/Windows; nvme_arena, nvme_bake_nf4 and
nvme_reader ship in the wheel with the smoke exercising the CLI and the O_DIRECT
alignment guard.
0.2.3 was committed but never tagged or published; PyPI goes 0.2.2 -> 0.2.4.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
-----BEGIN PGP SIGNATURE-----
iJEEABYKADkWIQQ7n/u5ZVSf3n/PS1XqrS1lj4+/ngUCamqZMRsUgAAAAAAEAA5t
YW51MiwyLjUrMS4xMiwwLDMACgkQ6q0tZY+Pv54NXAD9FrbYfM6cdN3SNnF2kVep
CuWRNZjY02/kcwG8andZxU4A/2BTNbZs6adxX9nNjtfhSRXKMDp+/R5Z4hwZdODC
hzEG
=n67z
-----END PGP SIGNATURE-----