Reduce onenormest memory usage

[pv]

Replace Python sort operations by Numpy sort operations (significantly
smaller memory usage and higher speed). Make some operations
block-wise, to avoid large temporary arrays. Delete local variables when
possible, to avoid keeping unused large arrays around.

Example:

import numpy as np
from scipy.sparse.linalg import aslinearoperator, onenormest
from scipy import sparse
n = 10e6
k = 100
np.random.seed(1234)
A = sparse.rand(n, k, 1e-6)
B = sparse.rand(k, n, 1e-6)
C = aslinearoperator(A) * aslinearoperator(B)
print(onenormest(C))

# BEFORE
$ /usr/bin/time python runtests.py --python test.py
9.02604258152
10.69user 1.54system 0:11.48elapsed 106%CPU (0avgtext+0avgdata 3207060maxresident)k
0inputs+728outputs (0major+558909minor)pagefaults 0swaps
# AFTER
$ /usr/bin/time python runtests.py --python test.py
9.02604258152
2.15user 1.04system 0:02.67elapsed 119%CPU (0avgtext+0avgdata 837128maxresident)k
0inputs+624outputs (0major+97560minor)pagefaults 0swaps

Benchmark results:

$ python runtests.py --bench-compare master sparse_linalg_onenormest
BENCHMARK                                    BEFORE      AFTER     FACTOR
...mEst.time_onenormest(3, 'onenormest')    31.87ms    36.12ms   1.13336139x
...mEst.time_onenormest(5, 'onenormest')    32.62ms    36.84ms   1.12939072x
...Est.time_onenormest(10, 'onenormest')    33.56ms    36.85ms   1.09794512x
...Est.time_onenormest(30, 'onenormest')    34.37ms    37.28ms   1.08443168x
...st.time_onenormest(100, 'onenormest')    47.18ms    44.87ms   0.95098888x
...t.time_onenormest(1000, 'onenormest')      1.10s      1.04s   0.94261184x
...st.time_onenormest(500, 'onenormest')   277.79ms   246.67ms   0.88795003x
...st.time_onenormest(300, 'onenormest')   127.40ms   108.53ms   0.85191679x
...ime_onenormest(10000.0, 'onenormest')     4.68ms     1.91ms   0.40855572x
...e_onenormest(1000000.0, 'onenormest')   604.86ms   182.65ms   0.30197286x
...me_onenormest(100000.0, 'onenormest')   136.04ms    24.26ms   0.17836432x

There's a minor slowdown for very small matrices --- but using onenormest for those doesn't make much sense anyway.

[argriffing]

Thanks! I think this PR together with the merged PR #4546 will have covered all of the modifications in #4095.

I think the slowdown for small matrices is OK for the reasons you've given.

One nitpick is that this PR uses scipy.sparse.rand which is fine in many contexts but which is a particularly weak choice for testing or benchmarking onenormest because rand samples only non-negative values. A feature of the onenormest implementation is that it computes one-norms of entrywise non-negative matrices 'exactly' (with the usual floating point error, but without the more interesting sources of error), basically using the following strategy:

>>> A = np.random.rand(20, 20)
>>> np.linalg.norm(A, ord=1)
12.716998703114299
>>> A.T.dot(np.ones(20)).max()
12.716998703114299

It does this implicitly without having to look at the entries of the matrix. The cleverness of the algorithm does not come into play for these matrices, so tests or benchmarks that use them could be improved by also using linear operators that have entrywise mixed signs.

Edit: I'd intended the PR #4679 to let you just replace rand -> randn to address this, but it emerged from review in a more general (and more verbose) form.

[argriffing]

Merged, thanks @pv!