Add generalized ufunc linalg functions and make numpy.linalg use them

[pv]

This is a followup to gh-2954 (list of new commits), which replaces numpy.linalg routines with the corresponding umath_linalg implementations, while preserving backward compatibility for ndim <= 2 inputs. The new code is all under numpy/linalg.

Numpy's test suite and those of Scipy, Pandas, and Nipy pass with these changes, so there probably aren't obvious problems with backward compat.

This PR doesn't add new functions to numpy.linalg, only replaces existing ones. That, and other things still to do are summarized in issue gh-3217.

[charris]

Looks like it needs a rebase.

[pv]

Rebased.

[njsmith]

Is there a test for the FP flags thing (63a8aef)? That seems like it could use a test.

For the double/single issue, does it work to use the ufunc dtype= argument? Theoretically this ought to let the internal ufunc machinery make any necessary promotions one-buffer-at-a-time, which is nicer than copying entire arrays.

[pv]

@njsmith: test_linalg.py:test_reduced_rank fails if the flags aren't cleared (for SVD, but not all of the routines). Not sure how to exactly best test that otherwise, as the issue is floating point errors (0/0 et al) inside LAPACK. Matrices full of inf cause those, but in that case LAPACK also signals a failure... I can't think of a way to directly test this.

The dtype keyword seems buggy for gufuncs, btw (gh-3222):

>>> import numpy as np
>>> a = np.array([[1.+2j,2+3j], [3+4j,4+5j]], dtype=np.cdouble)
>>> np.linalg._umath_linalg.eigvals(a, dtype=np.complex128)
__main__:1: DeprecationWarning: Implicitly casting between incompatible kinds. In a future numpy release, this will raise an error. Use casting="unsafe" if this is intentional.
__main__:1: ComplexWarning: Casting complex values to real discards the imaginary part
array([-0.37228132+0.j,  5.37228132+0.j])
>>> np.linalg._umath_linalg.eigvals(a)
array([-0.35670389-0.26307815j,  5.35670389+7.26307815j])
>>> b = np.array([[1.,2], [3,4]], dtype=np.float64)
>>> np.linalg._umath_linalg.eigvals(b, dtype=np.float64)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: No loop matching the specified signature was found for ufunc eigvals

The signature keyword however seems to work (but is much messier to use) --- does also that do the cast per-block?

[njsmith]

@pv: I think that dtype=X is just a shorthand for sig=<all X>? But I haven't checked.

[pv]

Using signature= for type selection, and do some ugly stuff to make the FP invalid flag reflect the LAPACK error status from all iterations. It would be nice to get the better error handling into ufuncs, but perhaps some ugliness can be tolerated before that.

[pv]

And then fixes for a GIL bug if a LAPACK error condition occurs.

[charris]

@pv Do you feel this is in a state to go in?

[pv]

I'm not aware of anything that breaks with this change (scipy, nipy, scikit-learn, pandas, statsmodels, networkx seem to suffer no regressions). Although this PR doesn't implement everything in the roadmap, it stands alone OK, and I think there's not more to add to it.

Should be advertised on the ML after (or before) merging.

[charris]

The list post is getting much (any) response, but I'll wait a few more days.

@pv What about something like npymath, say npylapack_lite, for the combined library?

[pv]

@charris: I don't see immediate use cases for npylapack_lite, as BLAS is widely available, if that's what you meant.

[pv]

No further input seems to be coming, so maybe if there are no remaining concerns, we can merge and be done with it.

[charris]

OK, in it goes.