ENH: Add ordered QZ decomposition #3107

Merged
merged 9 commits into from Nov 24, 2015

Projects

None yet

9 participants

@jseabold

This currently segfaults. I don't know why. I spent some time checking the wrapper definitions and stepping through with gdb. I didn't notice anything that was obvious to me. If someone else wants to give it a go, this was the working script I was using. If someone can figure out the segfault, I'll take it the rest of the way, but I just don't have time to mess with this.

A = [[-21.10-22.50j, 53.5-50.5j, -34.5+127.5j,   7.5+ 0.5j],
[ -0.46- 7.78j, -3.5-37.5j, -15.5+ 58.5j, -10.5- 1.5j],
[  4.30- 5.50j, 39.7-17.1j, -68.5+ 12.5j,  -7.5- 3.5j],
[  5.50+ 4.40j, 14.4+43.3j, -32.5- 46.0j, -19.0-32.5j]]
B = [[1.0-5.0j,  1.6+1.2j, -3+0j,  0.0-1.0j],
[0.8-0.6j,  3.0-5.0j, -4+3j, -2.4-3.2j],
[1.0+0.0j,  2.4+1.8j, -4-5j,  0.0-3.0j],
[0.0+1.0j, -1.8+2.4j,  0-4j,  4.0-5.0j]]

A = [[3.9, 12.5, -34.5, -0.5],
[4.3, 21.5, -47.5,  7.5],
[4.3, 21.5, -43.5,  3.5],
[4.4, 26.0, -46.0,  6.0]]

B = [[  1,    2,   -3,    1],
[  1,    3,   -5,    4],
[  1,    3,   -4,    3],
[  1,    3,   -4,    4]]


from scipy.linalg._decomp_qz import ordqz, _qz, _select_function, get_lapack_funcs
output = 'real'
lwork = None
overwrite_a = False
overwrite_b = False
check_finite = True
sort = 'lhp'
result, typ = _qz(A, B, output=output, lwork=lwork, sort=None,
                 overwrite_a=overwrite_a, overwrite_b=overwrite_b,
                 check_finite=check_finite)
AA, BB, Q, Z = result[0], result[1], result[-4], result[-3]
alpha, beta = result[3], result[4]
sfunction = _select_function(sort, typ)
select = sfunction(alpha, beta)
tgsen, = get_lapack_funcs(('tgsen',), (AA,BB))

n = len(A)
m = 0.
lwork = 2*m*(n-m)
liwork = n+2
#result = tgsen(select, AA, BB, Q, Z, lwork=-1, liwork=AA.shape[1]*2)
result = tgsen(select, AA, BB, Q, Z)
@andrenarchy

I also don't have the time to dig around, but if anybody else wants to look into this it would be nice if you could add information about your setup and the exact failure (OS, compiler, BLAS/LAPACK, backtrace, ...)?

@jseabold

OS: Linux
Compiler: gcc 4.8.1
Using recent dev version of OpenBlas. Not sure the included LAPACK version.
Compiled scipy with the following compiler flags

export FOPT="-O0 -ggdb"
export OPT="-O0 -ggdb"

Gdb backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff5a055cb in dtgsen_ () from /home/skipper/.local/lib/libopenblas.so.0
(gdb) bt
#0  0x00007ffff5a055cb in dtgsen_ ()
   from /home/skipper/.local/lib/libopenblas.so.0
#1  0x00007ffff08c732e in f2py_rout__flapack_dtgsen (
    capi_self=<optimised out>, capi_args=<optimised out>, 
    capi_keywds=<optimised out>, f2py_func=0x7ffff5a05570 <dtgsen_>)
    at build/src.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/linalg/_flapackmodule.c:2166
#2  0x000000000052e3cb in PyObject_Call (kw=0x0, arg=0xa10470, func=0xfa32b0)
    at ../Objects/abstract.c:2529
#3  do_call (nk=<optimised out>, na=<optimised out>, pp_stack=0x7fffffffd8e0, 
    func=0xfa32b0) at ../Python/ceval.c:4239
#4  call_function (oparg=<optimised out>, pp_stack=0x7fffffffd8e0)
    at ../Python/ceval.c:4044
#5  PyEval_EvalFrameEx (f=f@entry=0xa56150, throwflag=throwflag@entry=0)
    at ../Python/ceval.c:2666
#6  0x0000000000567cdc in PyEval_EvalCodeEx (closure=0x0, defcount=0, 
    defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=0x9410a0, 
    globals=0x7ffff7f18c30, co=<optimised out>) at ../Python/ceval.c:3253
#7  PyEval_EvalCode (co=co@entry=0x7ffff7f18c30, 
    globals=globals@entry=0x97e690, locals=locals@entry=0x97e690)
    at ../Python/ceval.c:667
#8  0x0000000000451adb in run_mod.42568 (mod=<optimised out>, 
    filename=<optimised out>, globals=0x97e690, locals=0x97e690, 
    flags=<optimised out>, arena=<optimised out>) at ../Python/pythonrun.c:1365
#9  0x0000000000451e5b in PyRun_FileExFlags (fp=fp@entry=0xa56ba0, 
    filename=filename@entry=0x7fffffffe059 "ordqz_dbg.py", 
    start=start@entry=257, globals=globals@entry=0x97e690, 
    locals=locals@entry=0x97e690, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffdab0) at ../Python/pythonrun.c:1351
#10 0x0000000000452394 in PyRun_SimpleFileExFlags (fp=fp@entry=0xa56ba0, 
    filename=<optimised out>, filename@entry=0x7fffffffe059 "ordqz_dbg.py", 
    closeit=closeit@entry=1, flags=flags@entry=0x7fffffffdab0)
    at ../Python/pythonrun.c:943
#11 0x0000000000452490 in PyRun_AnyFileExFlags (fp=fp@entry=0xa56ba0, 
    filename=filename@entry=0x7fffffffe059 "ordqz_dbg.py", 
    closeit=closeit@entry=1, flags=flags@entry=0x7fffffffdab0)
    at ../Python/pythonrun.c:747
#12 0x0000000000453ead in Py_Main (argc=<optimised out>, argv=0x7fffffffdc68)
    at ../Modules/main.c:640
#13 0x00007ffff7816de5 in __libc_start_main (main=0x453f6b <main>, argc=2, 
    ubp_av=0x7fffffffdc68, init=<optimised out>, fini=<optimised out>, 
    rtld_fini=<optimised out>, stack_end=0x7fffffffdc58) at libc-start.c:260
#14 0x00000000005786be in _start ()

Not much there, and as I mentioned stepping through the execution nothing looked off to me.

@jseabold

It would be great if someone could just confirm that it's not my openblas...

It also looks like I committed a non-working build after some tinkering with function argument types. Will fix this at least.

@ev-br
SciPy member

Fails to build on 64bit ubuntu 12.04 with stock atlas. A (hopefully) relevant part of the build log: https://gist.github.com/ev-br/7767310

@jseabold

Yes, I committed a borked .pyf file that I had been messing with. I'm fixing it to at least build now. Will commit shortly.

@jseabold

I actually think I figured it out. Let me finish the tests and see.

@jseabold

Ok, this is ready to go. Would appreciate a second set of eyes on the docs, etc. This didn't have my full attention.

The test are against R's QZ::ordqz.

Also need to make sure this works on Windows.

Are there release notes I need to update?

Closes #2236.

// cc @ricardomayerb

@andrenarchy

Thx! However, the tests are failing...

@jseabold

Right. They pass locally, and this is why I wanted to test them on another architecture. See the comment note in the tests about this type of failure.

From what I understand these should be equal up to an arbitrary scaling. So that means often the results will differ by a * -1, which is the case here. For the QZ decomp, I think I just ended up adding np.abs to all the tests, and for this one I added np.abs to one of the complex re-orderings. I'm not really sure what the 'right' thing to do here is. I think I should just test the abs/modulus for all of them. Thoughts?

@jseabold

The other failing test is just a broken test since I decided to raise a warning about the sort keyword in qz. I didn't want to just remove it, but given it doesn't work, maybe we should just raise a ValueError that says use ordqz.

@jseabold

The QZ package in R also ships with its own reference implementations of these LAPACK routines, which I assume is why I'm getting differences on my machine. It wasn't built against the same LAPACK. I'd just like someone to weigh in on the theory with some measure of confidence before I just add abs everywhere. It's been a while since I went through this, though I recall other decompositions (e.g., QR do not agree across LAPACKs/platforms up to a scalar, usually a sign difference.)

@jseabold

Yes, they're both correct. I recall doing this for the QZ now too. I'll just test them in a sign agnostic way.

import numpy as np
from scipy.linalg import ordqz

A2 = [[3.9, 12.5, -34.5, -0.5],
      [4.3, 21.5, -47.5,  7.5],
      [4.3, 21.5, -43.5,  3.5],
      [4.4, 26.0, -46.0,  6.0]]

B2 = [[1,    2,   -3,    1],
      [1,    3,   -5,    4],
      [1,    3,   -4,    3],
      [1,    3,   -4,    4]]

AA, BB, alpha, beta, Q, Z = ordqz(A2, B2, sort=lambda x,y : (x/y).imag != 0)

np.testing.assert_almost_equal(Q.dot(AA).dot(Z.T), A2)
np.testing.assert_almost_equal(Q.dot(BB).dot(Z.T), B2)

# from R
AAr = np.array([-38.566045,6.827259,0,0,41.488305,-5.2440468,0,0,37.280946,-12.954511,0.6172134,0,65.427303,-15.481901,3.2524997,4]).reshape(4,4, order='F')
BBr = np.array([-3.3680259,0,0,0,0,0.96209781,0,0,4.9227912,-1.1839165,0.3086067,0,9.6963058,-2.9877598,1.0271052,1]).reshape(4,4, order='F')
Qr = np.array([-0.55213781,-0.51056501,-0.51056501,-0.41688197,-0.6787574,0.069936641,0.069936641,0.72767172,0.48418203,-0.48418203,-0.48418203,0.54470478,-5.1446441e-17,-0.70710678,0.70710678,-2.6129692e-16]).reshape(4,4, order='F')
Zr = np.array([0.8775433,0.17550866,-0.31552691,-0.31552691,0.4375571,0.08751142,0.63280547,0.63280547,0.19611614,-0.98058068,-2.3869795e-15,-2.4980018e-15,0,5.1446441e-17,0.70710678,-0.70710678]).reshape(4,4, order='F')

# this fails because of sign errors
np.testing.assert_almost_equal(AA, AAr)

np.testing.assert_almost_equal(Qr.dot(AAr).dot(Zr.T), A2, 6)
np.testing.assert_almost_equal(Qr.dot(BBr).dot(Zr.T), B2, 6)
@coveralls

Coverage Status

Coverage remained the same when pulling 8fbab9f on jseabold:add-ordqz into c4314b0 on scipy:master.

@jseabold

Hmm, I don't know what this will do now that #3122 has been merged. I might need to fix this PR to reflect that.

@andrenarchy

Uh, yeah... The commit bbd1111 should be removed from this PR. Is there some git rebase wizardry that can do this? ;)

@jseabold

Yeah. Should be fixed. Rebased on master.

@andrenarchy

Nice, thx! 😄

@coveralls

Coverage Status

Coverage remained the same when pulling 1b7c11c on jseabold:add-ordqz into 5739a8b on scipy:master.

@andrenarchy

There are still issues with complex input matrices. With A and B defined by:

import numpy
from scipy.linalg import ordqz
A = numpy.random.rand(4,4)
B = numpy.random.rand(4,4)

the call ordqz(A,1j*B) results in

too many axes: 2 (effrank=2), expected rank=1
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-5-0a1db285eb6a> in <module>()
----> 1 ordqz(A,1j*B)

/home/andre/.pyvirt/local/lib/python2.7/site-packages/scipy/linalg/_decomp_qz.pyc in ordqz(A, B, sort, output, overwrite_a, overwrite_b, check_finite)
    311 
    312     if lwork is None or lwork == -1:
--> 313         result = tgsen(select, AA, BB, Q, Z, lwork=-1)
    314         lwork = result[-3][0]
    315     liwork = None

error: failed in converting 1st argument `select' of _flapack.ztgsen to C/Fortran array

Calling ordqz(1j*A,B) or ordqz(1j*A,1j*B) fails with

/home/andre/.pyvirt/local/lib/python2.7/site-packages/scipy/linalg/_decomp_qz.py:320: ComplexWarning: Casting complex values to real discards the imaginary part
  result = tgsen(select, AA, BB, Q, Z, lwork=lwork, liwork=liwork)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-94236e25ddab> in <module>()
----> 1 ordqz(A*1j,B)

/home/andre/.pyvirt/local/lib/python2.7/site-packages/scipy/linalg/_decomp_qz.pyc in ordqz(A, B, sort, output, overwrite_a, overwrite_b, check_finite)
    318         liwork = result[-2][0]
    319 
--> 320     result = tgsen(select, AA, BB, Q, Z, lwork=lwork, liwork=liwork)
    321 
    322     info = result[-1]

ValueError: On entry to ZTGSYL parameter number 20 had an illegal value

Any idea?

@jseabold

Sorry just now seeing this. Yes, both the inputs should be of the same type. R fails on this too, but more elegantly. I can add an error message.

It looks like you removed the tests on complex input arrays. Any reason why?

@jseabold

Oh, I see that you tried both. I'll have to look into this. The complex value tests got nixed, so I missed this.

@jseabold

The ValueError: On entry to ZTGSYL parameter number 20 had an illegal value is weird. Looking at the source it looks like ZTGSEN passes the wrong LWORK to ZTGSYL. Either that or the workspace lookup returns a wrong value. It needs be greater than or equal to 1 but it passes in LWORK - 2*M*(N-M), which is zero. I can just add one to the workspace lookup and it works fine...

@jseabold

The issue with the mixed type dtypes is that I was only checking the dtype of the first array. So complex, float works but float, complex doesn't. I'm just going to change it to check that both types are the same unless anyone thinks it's really necessary to support mixed type inputs.

@jseabold

I added back in the tests for complex input and now raise on mixed real/complex dtype inputs. Rebased on master. If travis is green, this should be good to merge.

@coveralls

Coverage Status

Coverage remained the same when pulling 25f5e43 on jseabold:add-ordqz into 3879d21 on scipy:master.

@pv pv removed the needs-work label Feb 28, 2014
@rgommers rgommers added this to the 0.15.0 milestone Feb 28, 2014
@andrenarchy

@jseabold: how do other functions like eig handle mixed inputs? Are there any scipy guidelines for mixed inputs? In my opinion, it's annoying (from the user's perspective) to always perform type checks and/or casting before calling a scipy function. Other opinions?

@jseabold

Good idea to check eig. I fixed this to check the typecode of the returned lapack function, so it supports mixed inputs now.

@coveralls

Coverage Status

Coverage remained the same when pulling 0c8aa21 on jseabold:add-ordqz into 9ccc684 on scipy:master.

@rgommers
SciPy member

I don't want a repeat of gh-2236, so need to test the Python 3.2 SSE3 binaries before merging this. Will try to do so asap.

@jseabold

AFAIK, these ordqz functions exist because of that LAPACK bug (?) that select doesn't properly work. That's why this is implemented as it is. See comments at end of #303. Let me know if there are any problems.

@rgommers
SciPy member

Unfortunately:

test_cef (test_decomp.TestOrdQZ) ... wine: Unhandled page fault on read access to 0xffffffff at 
address 0x1b9ed45 (thread 001e), starting debugger...
Unhandled exception: page fault on read access to 0xffffffff in 32-bit code (0x01b9ed45).
Register dump:

The rest at https://gist.github.com/rgommers/9608237

@jseabold

So...this is a 32-bit windows problem? Super fun. Do you know which tests, if any, ran before the segfault and/or is test_cef the only one that segfaults?

@jseabold

And to be clear, that's Python 3.2 32-bit Python only. No other segfaults on windows platforms / python versions?

@jseabold

If this is the same or similar to #2236 then it's either a compiler issue or a LAPACK bug that's long since been fixed, but we can't update the LAPACK for that build because of the other compiler issue? I don't want to spend any more time trying to solve this, if that's the case. Pretty frustrating.

How much work is it to get set up a build environment to reproduce this with a backtrace I can step through?

@rgommers
SciPy member

I don't have 64-bit Windows, not sure if it's an issue there. Also not sure about Python 3.3/3.4, also not set up now. If it turns out it's 3.2 specific, we can probably stop supporting that Python version for the next release.

@rgommers
SciPy member

https://github.com/certik/numpy-vendor should set up everything to reproduce this in a Vagrant VM on Linux, so shouldn't be too hard (assuming that that works for you).

@rgommers
SciPy member

This is with ATLAS 3.8.3, not the very latest but certainly not ancient.

@charris
SciPy member

I'd like to drop Python 3.2 at the earliest opportunity as it doesn't support the u"unicode" syntax and working around that clutters up the code. Python 3.4 has further improvements along that line, but we can't drop 3.3 quite yet ;)

@rgommers
SciPy member

Want to bring that up on the mailing list to make it official for numpy 1.9 and scipy 0.15?

@charris
SciPy member

@rgommers Done.

@jseabold

Thanks. The ATLAS is 5 years old. The LAPACK is 7. From what I understand it can't be updated because it's now Fortran 90 as of 2008. I haven't, actually checked the errata, so it may not make a difference. Either way it looks like a compiler issue, given that it only shows up on windows with SSE (presumably).

I'll see if I can't replicate this, eventually. I'm curious if R and the like segfault when built with the same compiler and linked against the same ATLAS/LAPACK you linked to. I might look into that as well.

@jseabold

From the errata. Won't know if any of these are the cause without debugging, though there are known stability bugs in the *gsen routines with lapack <= 3.2.2.

Fixed in 3.2.1: http://www.netlib.org/lapack/Change-3.2.1/013.txt
Fixed in 3.2.2: http://www.netlib.org/lapack/lapack-3.2.2.html

@rgommers
SciPy member

@cournape would you be able to build newer ATLAS libs for Windows sometime? I've been hesitant to try on Windows.....

@rgommers
SciPy member

@jseabold the dropping <Win7 and moving to gfortran starts to look like a decent option. That would solve the f90 issue as well as make it easier to update lapack.

@jseabold

Maybe there's an "easy" way to link against MKL provided by enthought or something? Just to see if it is a LAPACK issue. I'm not going to be able to be a ton of help on windows.

@rgommers
SciPy member

I'm not much better in dealing with Windows:(

@cgohlke and @cournape are set up to compile with MKL. Any help or advice you can provide?

@cournape
SciPy member
@jseabold

Rebased.

@pv pv removed the PR label Aug 13, 2014
@ev-br
SciPy member

This needs another rebase :-(. Probably due to the change of the lwork handling

@rgommers rgommers removed this from the 0.15.0 milestone Sep 16, 2014
@ricardomayerb

Since the work to put together release 0.16.0 it's about to start, I humbly ask if there any chance to resolve this QZ issue for this upcoming release.

Also, it could be useful if someone could state what are the skills needed to resolve this, because if no one has the time to work on this I'd be interested in contributing to it, but I suspect there is a lot I would have to learn before I can be of any use.

@jseabold

Rebased. To go in, someone needs to debug the issue that apparently only shows up on Python 3.2 32-bit windows unless we've since updated our ATLAS/LAPACK and the problem has gone away.

@rgommers
SciPy member

pep8 scipy is still quite unhappy

@rgommers
SciPy member

I'm right now messing with the Windows ATLAS binaries, because they're missing some routines that were wrapped for 0.16.0. I'll have a look at this again.

@jseabold

Travis is happy now. Let me know anything else I can do.

@rgommers rgommers and 1 other commented on an outdated diff May 23, 2015
scipy/linalg/flapack.pyf.src
subroutine <prefix2>gges(jobvsl,jobvsr,sort_t,<prefix2>select,n,a,lda,b,ldb,sdim,alphar,alphai,beta,vsl,ldvsl,vsr,ldvsr,work,lwork,bwork,info)
- ! For a pair of N-by-N real nonsymmetric matrices (A,B) computes
+ ! For a pair of N-by-N real nonsymetric matrices (A,B) computes
@rgommers
rgommers May 23, 2015

introducing a typo here, can you revert?

@rgommers rgommers commented on the diff May 23, 2015
scipy/linalg/tests/test_decomp.py
assert_(all(diag(BB) >= 0))
+def _make_pos(X):
+ # the decompositions can have different signs than verified results
+ return np.sign(X)*X
+
+
+class TestOrdQZ(TestCase):
+ @classmethod
+ def setupClass(cls):
@rgommers
rgommers May 23, 2015

Any reason not to change the above two lines to the standard def setUp(self):?

@rgommers
rgommers May 23, 2015

and the cls. below to self.

@jseabold
jseabold Jul 14, 2015

Just saves some time so they aren't instantiated before each test method is run.

@rgommers rgommers commented on an outdated diff May 23, 2015
scipy/linalg/_decomp_qz.py
+ sfunction = lambda x, y: (abs(x/y) < 1.0)
+ elif sort == 'ouc':
+ sfunction = lambda x, y: (abs(x/y) > 1.0)
+ else:
+ raise ValueError("sort parameter must be None, a callable, or "
+ "one of ('lhp','rhp','iuc','ouc')")
+
+ return sfunction
+
+
+def _qz(A, B, output='real', lwork=None, sort=None, overwrite_a=False,
+ overwrite_b=False, check_finite=True):
+ if sort is not None:
+ # Disabled due to segfaults on win32, see ticket 1717.
+ raise ValueError("The 'sort' input of qz() has to be None and will be "
+ "removed in 0.15. Use ordqz instead.")
@rgommers
rgommers May 23, 2015

let's make this a future release instead of 0.15

@rgommers rgommers commented on an outdated diff May 23, 2015
scipy/linalg/_decomp_qz.py
+ overwrite_b=overwrite_b, sort_t=0)
+
+ info = result[-1]
+ if info < 0:
+ raise ValueError("Illegal value in argument %d of gges" % -info)
+ elif info > 0 and info <= a_n:
+ warnings.warn("The QZ iteration failed. (a,b) are not in Schur "
+ "form, but ALPHAR(j), ALPHAI(j), and BETA(j) should be "
+ "correct for J=%d,...,N" % info-1, UserWarning)
+ elif info == a_n + 1:
+ raise LinAlgError("Something other than QZ iteration failed")
+ elif info == a_n + 2:
+ raise LinAlgError("After reordering, roundoff changed values of some "
+ "complex eigenvalues so that leading eigenvalues "
+ "in the Generalized Schur form no longer satisfy "
+ "sort=True. This could also be caused due to "
@rgommers
rgommers May 23, 2015

remove caused here

@rgommers rgommers commented on an outdated diff May 23, 2015
scipy/linalg/_decomp_qz.py
@@ -160,86 +214,139 @@ def qz(A, B, output='real', lwork=None, sort=None, overwrite_a=False,
[-0.79813178, 0.58842606, 0.12938478],
[-0.54861681, -0.6210585 , -0.55973739]])
+ See also
+ --------
+ scipy.linalg.ordqz
@rgommers
rgommers May 23, 2015

simply ordqz is enough

@rgommers rgommers commented on an outdated diff May 23, 2015
scipy/linalg/_decomp_qz.py
- typb = 'D'
- else:
- b1 = b1.astype('F')
- typb = 'F'
+ Parameters
+ ----------
+ A : (N, N) array_like
+ 2d array to decompose
+ B : (N, N) array_like
+ 2d array to decompose
+ sort : {callable, 'lhp', 'rhp', 'iuc', 'ouc'}, optional
+ Specifies whether the upper eigenvalues should be sorted. A callable
+ may be passed that, given a eigenvalue, returns a boolean denoting
+ whether the eigenvalue should be sorted to the top-left (True). For
+ real matrix pairs, the sort function takes three real arguments
+ (alphar, alphai, beta). The eigenvalue x = (alphar + alphai*1j)/beta.
@rgommers
rgommers May 23, 2015

double backticks needed around code snippets

@rgommers
rgommers May 23, 2015

same 3 lines below

@rgommers rgommers commented on an outdated diff May 23, 2015
scipy/linalg/_decomp_qz.py
- gges, = get_lapack_funcs(('gges',), (a1,b1))
+ output : str {'real','complex'}
+ Construct the real or complex QZ decomposition for real matrices.
+ Default is 'real'.
+ overwrite_a : bool
+ If True, the contents of A are overwritten.
+ overwrite_b : bool
+ If True, the contents of B are overwritten.
+ check_finite : boolean
@rgommers
rgommers May 23, 2015

bool and for all parameters except A and B, add , optional

@rgommers
SciPy member

WIth current setup in numpy-vendor, there are four tests that crash: test_cef, test_iuc, test_lhp, test_ref. Both with SSE2 and SSE3 builds, on Python 2.7. Didn't check other Python versions, but very likely will be the same issue. All looks like:

.............................wine: Unhandled page fault on read access to 0xffffffff at address 0x117e5d5 (thread 0046), starting debugger...
Application tried to create a window, but no driver could be loaded.
Make sure that your X server is running and that $DISPLAY is set correctly.
Unhandled exception: page fault on read access to 0xffffffff in 32-bit code (0x0117e5d5).
Register dump:
 CS:0073 SS:007b DS:007b ES:007b FS:0033 GS:003b
 EIP:0117e5d5 ESP:0042b92c EBP:00000001 EFLAGS:00010202(  R- --  I   - - - )
 EAX:0042c0a4 EBX:0042c09c ECX:00000001 EDX:00000040
 ESI:0042c084 EDI:00000004
Stack dump:
0x0042b92c:  00000006 00000040 0042c09c 00000010
0x0042b93c:  0042c09c 01c600c8 00000008 0117ee2f
0x0042b94c:  0042c07c 01c600c0 00000002 00000003
0x0042b95c:  00000001 00000002 0042c0bc 00000003
0x0042b96c:  00000003 0117ed10 0042c07c 011288c1
0x0042b97c:  00000001 00000003 00000000 3ff00000
Backtrace:
=>0 0x0117e5d5 in _flapack.pyd (+0x3be5d5) (0x00000001)
0x0117e5d5: 
Modules:
Module  Address                 Debug info      Name (84 modules)
PE        440000-  449000       Deferred        _multiprocessing.pyd
PE        d10000-  db9000       Deferred        mtrand.pyd
PE        dc0000- 1516000       Stabs           _flapack.pyd
...
@rgommers
SciPy member

Will try again when I have https://github.com/numpy/numpy/wiki/Mingw-static-toolchain set up, maybe OpenBLAS works better.

@jseabold

@rgommers addressed comments and rebased. Updated the versionadded to 0.17.0.

@jseabold

Cleaned up the remaining pep-8 issues.

@rgommers rgommers added this to the 0.17.0 milestone Aug 20, 2015
@rgommers
SciPy member

(adding 0.17.0 milestone to not forget to look at this again)

@rgommers
SciPy member

After the last rebase I cannot reproduce the crash anymore. Tried with the SSE3 builds of Python 2.7 and 3.4. Maybe one of the lwork fixed did the trick here, I'm not sure.

Also note that previously 3.2 seemed to be in the worst shape, but that's a don't care anymore by now.

@rgommers
SciPy member

@jseabold one last rebase?

@jseabold

Rebase was an odd one. Could probably use another once over, provided that CI test pass.

@rgommers
SciPy member

@jseabold all green. What do you mean with odd rebase? Some commits could be squashed, but other than that it looks OK to me.

@jseabold

There were some conflicts in which the diffs didn't really make sense to me, so I did my best. I can clean up the commit history a bit.

@jseabold

Did a bit of cleanup.

@rgommers
SciPy member

@jseabold can you edit the commit message of the last commit to remove [ci skip]. The TravisCI tests won't run otherwise.

@jseabold jseabold STY: PEP-8 cleanup.
STY: Fix error messages

DOC: Fix docstring

STY: PEP-8 Cleanup

DOC: Fix versionadded.

STY: Use double-backticks for code.
38f0ac5
@jseabold
@rgommers
SciPy member

Allright, let's say a little prayer and hit the green button.....

@rgommers rgommers merged commit 60b5baa into scipy:master Nov 24, 2015

1 check passed

Details continuous-integration/travis-ci/pr The Travis CI build passed
@rgommers
SciPy member

Merged, thanks Skipper!

@ricardomayerb

Thank you so much, Skipper and Ralph! This is great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment