Investigate the impact of LU-variants on randomized SVD with scipy's lu_no_fortran branch

[ogrisel]

Experiment with the lu_no_fortan branch of scipy (scipy/scipy#18358) to:

• evaluate the impact of using index-based permutation in the randomized SVD solver of scikit-learn
• explore the impact of discarding the permutation info and therefore confirming @tygert's intuitions expressed in RFC: add support for LU factorization in the linalg extension data-apis/array-api#627 (comment).

Here are the resulting plots of the randomized SVD benchmarks with various normalizers:

So in conclusion:

• Using LU-based normalization while completely discarding the permutation info is sometimes better and sometimes worse (!!!) than no normalization at all, but always worse than an LU-based normalizer that takes the permutation into account one way or another.
• Using index-based permutation (using matrix_p=False followed by row-wise fancing indexing) seems to be approximately as fast as letting scipy precompute the permutation with permute_l=True).
• LU-based normalization with the new Cython-based scipy branch is still slightly faster than QR-based normalization (on average).

[lezcano]

Fair enough. Thank you for the benchmarks!

There's just one bit I don't understand. How come in the last graph "LU_no_permute" and "QR" seem to "come back in time" (iter 0 happens AFTER iter 1)?

[ogrisel]

There's just one bit I don't understand. How come in the last graph "LU_no_permute" and "QR" seem to "come back in time" (iter 0 happens AFTER iter 1)?

Those are random fluctuation for the small times. Ideally, I should re-run the benchmarks many times and plot the average with horizontal error bars, but life is too short ;)