Skip to content

KL-SOAP, ECC with SR, faster PSGD

Choose a tag to compare

View all tags
@ClashLuke ClashLuke released this 26 Apr 21:55
· 45 commits to main since this release
v3.1.0
d972fe6
  • #88 implements SR on ECC -- ECC previously disabled SR (3.0.0) or was incorrectly implemented (2.3.0); @josejg pointed out a fix
  • New optimizer: KLSOAP. It's aware of SOAP learning its precond factors jointly, following https://arxiv.org/abs/2509.03378
  • Removes CPU-GPU syncs in PSGD hotpath after analyzing generated code

Contributors

josejg
Assets 2
Loading