Skip to content

trnsparse 0.2.0 — NKI SpMM validated on trn1

Choose a tag to compare

@scttfrdmn scttfrdmn released this 13 Apr 20:51
· 40 commits to main since this release

Phase 1 lands. First hardware-validated NKI kernel in trnsparse, and the suite's first torch.autograd.Function-wrapped NKI kernel (closes trnsci/trnsci#3 for this repo).

Added

  • NKI SpMM kernel (trnsparse/nki/kernels.py::_spmm_dense_kernel) — stationary A-tile reuse GEMM on the Tensor Engine. TILE_M = TILE_K = 128, TILE_N = 512.
  • Autograd wrapping (trnsparse.nki.dispatch._SpMMFunction) — analytic backward (dA = dC @ Bᵀ projected, dB = Aᵀ @ dC). torch.autograd.gradcheck passes at atol=1e-4.
  • Dispatch wiringset_backend("nki") routes trnsparse.spmm through the NKI path; v0.1.3 torch.sparse fallback unchanged otherwise.
  • tests/test_nki_spmm.py — 8 @pytest.mark.neuron tests: parity across aligned + unaligned + low-density shapes, gradcheck, end-to-end loss.backward() smoke.
  • benchmarks/bench_spmm.py — four-backend SpMM table (scipy / torch.sparse / trnsparse pytorch / trnsparse nki) in one pytest pass.
  • docs/benchmarks.md populated with real trn1.2xlarge numbers.

Hardware validation (trn1.2xlarge, Neuron SDK 2.24)

Test Result
5 parity cases (aligned + unaligned + density 0.01–0.1) ✅ all pass at atol=1e-3, rtol=1e-4
torch.autograd.gradcheck ✅ pass at atol=1e-4
loss.backward() smoke ✅ finite gradients through full stack

Known limits

  • NKI is slower than CPU in v0.2.0 — the kernel materializes the CSR into a dense (M, K) tile before the matmul. At density 0.001 on 1024 × 1024, this means ~1000× more work than scipy does. See docs/benchmarks.md for numbers. Sparse speedup comes from row-bucketing + gather-matmul-scatter, which is #15 / v0.3.0 / Phase 3.
  • SpMV stays on PyTorch — single-column NKI matmul doesn't amortize the compile + dispatch overhead.

Closes #14. Closes #4.