support for implicitly scaling/centering sparse matrix

[robmaz]

irlba has center= and scale= arguments to implicitly scale and center matrices while calculating the partial svd. This is super useful for large sparse matrices, because centering explicitly beforehand would usually turn them into dense matrices and make the svd (or even keeping them in memory) unfeasible.

Do you think this can be supported somehow? I find that Rspectra can still handle some large matrices (~1 billion entries) where irlba already croaks, which is very nice, but principal-component regression basically needs the centering step ...

[yixuan]

If I understand correctly you hope this to be implemented in the RSpectra package, right?

[robmaz]

Ultimately I (and, I guess, most people interested in principal-component regression) would use it via Rspectra, but I think it would need to be implemented on a deeper level, e.g., the crossproduct X*t(X) for sparse matrices, which I have seen somewhere in the algorithm. I do not know how the modified algorithm goes, but it certainly must exploit the fact that instead of 0s, you now have many elements with known fixed values -m/s. Basically a concept of generalized sparsity.

[yixuan]

I have added this feature to the development version of RSpectra, see the example here. I just need more time to document and push the new version to CRAN.

[robmaz]

Wow, that was quick. Thank you very much, I look forward to trying this out.