trnsparse 0.2.0 — NKI SpMM validated on trn1
Phase 1 lands. First hardware-validated NKI kernel in trnsparse, and the suite's first torch.autograd.Function-wrapped NKI kernel (closes trnsci/trnsci#3 for this repo).
Added
- NKI SpMM kernel (
trnsparse/nki/kernels.py::_spmm_dense_kernel) — stationary A-tile reuse GEMM on the Tensor Engine.TILE_M = TILE_K = 128, TILE_N = 512. - Autograd wrapping (
trnsparse.nki.dispatch._SpMMFunction) — analytic backward (dA = dC @ Bᵀprojected,dB = Aᵀ @ dC).torch.autograd.gradcheckpasses atatol=1e-4. - Dispatch wiring —
set_backend("nki")routestrnsparse.spmmthrough the NKI path; v0.1.3torch.sparsefallback unchanged otherwise. tests/test_nki_spmm.py— 8@pytest.mark.neurontests: parity across aligned + unaligned + low-density shapes, gradcheck, end-to-endloss.backward()smoke.benchmarks/bench_spmm.py— four-backend SpMM table (scipy / torch.sparse / trnsparse pytorch / trnsparse nki) in one pytest pass.docs/benchmarks.mdpopulated with realtrn1.2xlargenumbers.
Hardware validation (trn1.2xlarge, Neuron SDK 2.24)
| Test | Result |
|---|---|
| 5 parity cases (aligned + unaligned + density 0.01–0.1) | ✅ all pass at atol=1e-3, rtol=1e-4 |
torch.autograd.gradcheck |
✅ pass at atol=1e-4 |
loss.backward() smoke |
✅ finite gradients through full stack |
Known limits
- NKI is slower than CPU in v0.2.0 — the kernel materializes the CSR into a dense
(M, K)tile before the matmul. At density 0.001 on1024 × 1024, this means ~1000× more work than scipy does. Seedocs/benchmarks.mdfor numbers. Sparse speedup comes from row-bucketing + gather-matmul-scatter, which is #15 / v0.3.0 / Phase 3. - SpMV stays on PyTorch — single-column NKI matmul doesn't amortize the compile + dispatch overhead.