better solve performance with lsqr

[grinsted]

I get better performance when I use the scipy.sparse.linalg.lsqr iterative solver instead of np.linalg.lstsq.

I get 16.7s vs 19.8s for a test with a big data set.

[efiring]

That's a very small difference. And, I'm puzzled by your use of a sparse solver; the matrix involved is not sparse.

[grinsted]

Those time measurements includes "matrix prep" time which makes up about two thirds of the total run time for solve. When I only measure the LSQR vs LSTSQ statetement then the relative improvement is much greater (9s vs 14s -on another machine). That is IMO substantial. Matrix prep time can also be optimized i believe.

I looked into LSQR because I had prior experience (from matlab) where it gave huge performance boosts on other large non-sparse matrix systems. Perhaps there are other iterative methods that are even better.(?)

[grinsted]

Here's a little snippet comparing performance of LSTSQ, LSQR, LSMR: scipy/scipy#11204

For matrices that are 100000x20 and 100000x60 then i get:

                    wall_time
n      p  solver             
100000 20 la.lstsq   0.063300
          spa.lsmr   0.010690
          spa.lsqr   0.016128
       60 la.lstsq   0.215469
          spa.lsmr   0.029070
          spa.lsqr   0.041543

[efiring]

Closing, see #81 (comment). This can be reopened as needed.