-
Report stable rank (froebenius norm ^2 / spectral norm ^2)
-
Histogram of eigenvalues, potentially using the same code as lovely_tensors provides in the tensor.plt call https://github.com/xl0/lovely-tensors
-
If preconditioning fails due to instability add dampening using the heuristic in the EK-FAC paper. This happens less often with TrackStar due to the down projection but it would be nice to make it generally available for all our methods.
https://arxiv.org/html/2407.21594v1
https://arxiv.org/abs/2308.03296