New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

arpack tests fail when installed on mac os x #2547

Closed
joernhees opened this Issue Jun 7, 2013 · 34 comments

Comments

Projects
None yet
7 participants
@joernhees
Contributor

joernhees commented Jun 7, 2013

I ran into this problem several times already with different installation methods. This time i used brew to install scipy:

brew tap homebrew/science
brew tap samueljohn/python
brew update
brew install python
pip install nose
brew install scipy

If you then run the tests the following errors occur: https://gist.github.com/joernhees/5733199

A shortened version here:

$ ipython
Python 2.7.5 (default, Jun 8 2013, 00:27:06)
Type "copyright", "credits" or "license" for more information.

IPython 0.13.2 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: import scipy

In [2]: scipy.test()
Running unit tests for scipy
NumPy version 1.7.1
NumPy is installed in /usr/local/lib/python2.7/site-packages/numpy
SciPy version 0.12.0
SciPy is installed in /usr/local/lib/python2.7/site-packages/scipy
Python version 2.7.5 (default, Jun 8 2013, 00:27:06) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)]
nose version 1.3.0
/usr/local/lib/python2.7/site-packages/numpy/lib/utils.py:139: DeprecationWarning: `scipy.lib.blas` is deprecated, use `scipy.linalg.blas` instead!
warnings.warn(depdoc, DeprecationWarning)
/usr/local/lib/python2.7/site-packages/numpy/lib/utils.py:139: DeprecationWarning: `scipy.lib.lapack` is deprecated, use `scipy.linalg.lapack` instead!
warnings.warn(depdoc, DeprecationWarning)
...............................................K...............................................K..K........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................SSSSSS......SSSSSS......S..........K..............................................................................................................................................................................................................................................................................................K...................................................................................................................................................................................................../usr/local/lib/python2.7/site-packages/scipy/misc/pilutil.py:230: DeprecationWarning: fromstring() is deprecated. Please call frombytes() instead.
image = Image.fromstring('L',shape,bytedata.tostring())
/usr/local/lib/python2.7/site-packages/scipy/misc/pilutil.py:230: DeprecationWarning: fromstring() is deprecated. Please call frombytes() instead.
image = Image.fromstring('L',shape,bytedata.tostring())
/usr/local/lib/python2.7/site-packages/scipy/misc/pilutil.py:230: DeprecationWarning: fromstring() is deprecated. Please call frombytes() instead.
image = Image.fromstring('L',shape,bytedata.tostring())
/usr/local/lib/python2.7/site-packages/scipy/misc/pilutil.py:230: DeprecationWarning: fromstring() is deprecated. Please call frombytes() instead.
image = Image.fromstring('L',shape,bytedata.tostring())
/usr/local/lib/python2.7/site-packages/scipy/misc/pilutil.py:230: DeprecationWarning: fromstring() is deprecated. Please call frombytes() instead.
image = Image.fromstring('L',shape,bytedata.tostring())
./usr/local/lib/python2.7/site-packages/scipy/misc/pilutil.py:230: DeprecationWarning: fromstring() is deprecated. Please call frombytes() instead.
image = Image.fromstring('L',shape,bytedata.tostring())
./usr/local/lib/python2.7/site-packages/scipy/misc/pilutil.py:230: DeprecationWarning: fromstring() is deprecated. Please call frombytes() instead.
image = Image.fromstring('L',shape,bytedata.tostring())
.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FFF.....FFF..FF......FF..F..........FFF.....FFF..............FF..FF.FFF..FF..FF..FF..F..........FF..FFF.FFF.............F.......F....F.......F..............FFF......FF.............F...F...F....F...F...F..............FFF.FFF.FFF..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................K....................................................................K......................................................................................................................................................................KK.................................................................................................................................................................................................................................................................................................................................................................................................................................K........K.......................................................................................................................................................................................................................S...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
FAIL: test_arpack.test_symmetric_modes(True, <std-symmetric>, 'f', 2, 'LM', None, 0.5, <class 'scipy.sparse.csr.csr_matrix'>, None, 'normal')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", line 259, in eval_evec
assert_allclose(LHS, RHS, rtol=rtol, atol=atol, err_msg=err)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 1179, in assert_allclose
verbose=verbose, header=header)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 645, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=0.00178814, atol=0.000357628
error for eigsh:standard, typ=f, which=LM, sigma=0.5, mattype=csr_matrix, OPpart=None, mode=normal
(mismatch 100.0%)
x: array([[ 2.38156418e-01, -6.75444982e+09],
[ -1.07853470e-01, -8.01245676e+09],
[ 1.24683023e-01, -5.19757686e+09],...
y: array([[ 2.38156418e-01, -5.70949789e+08],
[ -1.07853470e-01, -4.05829392e+08],
[ 1.24683023e-01, 6.25800146e+07],...

======================================================================
FAIL: test_arpack.test_symmetric_modes(True, <std-symmetric>, 'f', 2, 'LM', None, 0.5, <class 'scipy.sparse.csr.csr_matrix'>, None, 'buckling')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", line 259, in eval_evec
assert_allclose(LHS, RHS, rtol=rtol, atol=atol, err_msg=err)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 1179, in assert_allclose
verbose=verbose, header=header)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 645, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=0.00178814, atol=0.000357628
error for eigsh:standard, typ=f, which=LM, sigma=0.5, mattype=csr_matrix, OPpart=None, mode=buckling
(mismatch 100.0%)
x: array([[ 3.53755447e-01, -2.29114355e+04],
[ -1.60204595e-01, -6.65625445e+04],
[ 1.85203065e-01, -2.69012500e+04],...
y: array([[ 3.53755447e-01, -8.88255444e+05],
[ -1.60204595e-01, -2.39343354e+06],
[ 1.85203065e-01, -3.96842525e+04],...

======================================================================
FAIL: test_arpack.test_symmetric_modes(True, <std-symmetric>, 'f', 2, 'LM', None, 0.5, <class 'scipy.sparse.csr.csr_matrix'>, None, 'cayley')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", line 259, in eval_evec
assert_allclose(LHS, RHS, rtol=rtol, atol=atol, err_msg=err)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 1179, in assert_allclose
verbose=verbose, header=header)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 645, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=0.00178814, atol=0.000357628
error for eigsh:standard, typ=f, which=LM, sigma=0.5, mattype=csr_matrix, OPpart=None, mode=cayley
(mismatch 100.0%)
x: array([[ -2.38156418e-01, 1.04661597e+09],
[ 1.07853470e-01, 1.39930271e+09],
[ -1.24683023e-01, 9.56906461e+08],...
y: array([[ -2.38156418e-01, 7.63721281e+07],
[ 1.07853470e-01, 1.25169905e+08],
[ -1.24683023e-01, 2.91283130e+07],...

======================================================================
...
======================================================================
FAIL: test_arpack.test_symmetric_modes(True, <gen-symmetric>, 'd', 2, 'SA', None, 0.5, <function asarray at 0x10ad03e60>, None, 'cayley')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", line 259, in eval_evec
assert_allclose(LHS, RHS, rtol=rtol, atol=atol, err_msg=err)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 1179, in assert_allclose
verbose=verbose, header=header)
File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 645, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=4.44089e-13, atol=4.44089e-13
error for eigsh:general, typ=d, which=SA, sigma=0.5, mattype=asarray, OPpart=None, mode=cayley
(mismatch 100.0%)
x: array([[-0.36892684, -0.01935691],
[-0.26850996, -0.11053158],
[-0.40976156, -0.13223572],...
y: array([[-0.43633077, -0.01935691],
[-0.25161386, -0.11053158],
[-0.36756684, -0.13223572],...

----------------------------------------------------------------------
Ran 4831 tests in 54.169s

FAILED (KNOWNFAIL=11, SKIP=14, failures=63)
Out[2]: <nose.result.TextTestResult run=4831 errors=0 failures=63>

In [3]:
@joernhees

This comment has been minimized.

Show comment
Hide comment
@joernhees

joernhees Jun 8, 2013

Contributor

I actually found a workaround by using openblas (a bit slower is better than wrong IMO):

First make sure you have a gfortran version >= 4.7: gfortran --version (If not you probably followed some popular mac os x scipy install howto before. In my case it used brew's apple-gcc42, so brew unlink apple-gcc42 && brew install gfortran could solve that).

Then run:

brew install openblas
brew reinstall scipy --with-openblas

As mentioned in Homebrew/homebrew-python#12 no one wants to be responsible for these errors...
IMHO scipy should at least warn users in case a library is used which is known to cause errors of this magnitude.

Contributor

joernhees commented Jun 8, 2013

I actually found a workaround by using openblas (a bit slower is better than wrong IMO):

First make sure you have a gfortran version >= 4.7: gfortran --version (If not you probably followed some popular mac os x scipy install howto before. In my case it used brew's apple-gcc42, so brew unlink apple-gcc42 && brew install gfortran could solve that).

Then run:

brew install openblas
brew reinstall scipy --with-openblas

As mentioned in Homebrew/homebrew-python#12 no one wants to be responsible for these errors...
IMHO scipy should at least warn users in case a library is used which is known to cause errors of this magnitude.

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 9, 2013

Member

It's not "doesn't want to be responsible", it's simply quite hard to solve. Duplicate of gh-2372, so closing.

Member

rgommers commented Jun 9, 2013

It's not "doesn't want to be responsible", it's simply quite hard to solve. Duplicate of gh-2372, so closing.

@rgommers rgommers closed this Jun 9, 2013

@joernhees

This comment has been minimized.

Show comment
Hide comment
@joernhees

joernhees Jun 10, 2013

Contributor

@rgommers i'm quite sure this isn't a dup of that, i'm running on mac os x and afaik it uses Accelerate.framework in LAPACK.

Also i seem to judge the severity of this a bit different: Many of my researcher colleagues use macs and all of whom I asked to run the scipy test were surprised that these tests fail. I'm not 100 % sure how bad the errors actually are, but i get a bad feeling that many scientific publications could depend on this and never notice the errors.

What i suggest is not hard to solve: i suggested that scipy should check and notify the users if the wrong library is used which is known to cause problems. You point out yourself that not using Accelerate and llvm is the best in your opinion ( Homebrew/homebrew-python#12 ). Why not check for it?

Contributor

joernhees commented Jun 10, 2013

@rgommers i'm quite sure this isn't a dup of that, i'm running on mac os x and afaik it uses Accelerate.framework in LAPACK.

Also i seem to judge the severity of this a bit different: Many of my researcher colleagues use macs and all of whom I asked to run the scipy test were surprised that these tests fail. I'm not 100 % sure how bad the errors actually are, but i get a bad feeling that many scientific publications could depend on this and never notice the errors.

What i suggest is not hard to solve: i suggested that scipy should check and notify the users if the wrong library is used which is known to cause problems. You point out yourself that not using Accelerate and llvm is the best in your opinion ( Homebrew/homebrew-python#12 ). Why not check for it?

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jun 10, 2013

Member

Yes, disabling support for Accelerate completely could be sensible until this is resolved.

Member

pv commented Jun 10, 2013

Yes, disabling support for Accelerate completely could be sensible until this is resolved.

@pv pv reopened this Jun 10, 2013

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 10, 2013

Member

Sorry about linking the wrong ticket. It is a duplicate of gh-2256 as well as gh-2248.

I'm not sure about disabling Accelerate support - it will create a lot of build issues and a flood of ticket / complaints on the mailing list. Disabling this part of ARPACK functionality (everywhere or only on OS X?) is probably a better alternative. AFAIK it's only single precision that has a problem and that's not widely used.

I suggest continuing this discussion on gh-2248; I'll add that to the 0.13.0 Milestone so we can't just forget about this. Closing again as duplicate.

Member

rgommers commented Jun 10, 2013

Sorry about linking the wrong ticket. It is a duplicate of gh-2256 as well as gh-2248.

I'm not sure about disabling Accelerate support - it will create a lot of build issues and a flood of ticket / complaints on the mailing list. Disabling this part of ARPACK functionality (everywhere or only on OS X?) is probably a better alternative. AFAIK it's only single precision that has a problem and that's not widely used.

I suggest continuing this discussion on gh-2248; I'll add that to the 0.13.0 Milestone so we can't just forget about this. Closing again as duplicate.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jun 10, 2013

Member

@rgommers: I meant that we disable also Veclib/Accelerate detection. This is certainly doable.

The problem is that the "part of ARPACK functionality" in question is not precisely defined. It's not even only single precision, also apparently double precision fails (see 'd' in the above tests).

Member

pv commented Jun 10, 2013

@rgommers: I meant that we disable also Veclib/Accelerate detection. This is certainly doable.

The problem is that the "part of ARPACK functionality" in question is not precisely defined. It's not even only single precision, also apparently double precision fails (see 'd' in the above tests).

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 10, 2013

Member

I understood that that's what you meant. It's doable, but will create lots of issues for users. We'll be disabling the only BLAS/LAPACK shipped with the OS, forcing users to hunt for other binaries (which may have their own issues) or install a BLAS/LAPACK from source (very much nontrivial). So it's not a decision to take lightly.

OpenBLAS support in numpy.distutils is still shaky by the way: http://thread.gmane.org/gmane.comp.python.numeric.general/53541/focus=53547

Member

rgommers commented Jun 10, 2013

I understood that that's what you meant. It's doable, but will create lots of issues for users. We'll be disabling the only BLAS/LAPACK shipped with the OS, forcing users to hunt for other binaries (which may have their own issues) or install a BLAS/LAPACK from source (very much nontrivial). So it's not a decision to take lightly.

OpenBLAS support in numpy.distutils is still shaky by the way: http://thread.gmane.org/gmane.comp.python.numeric.general/53541/focus=53547

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 10, 2013

Member

You're right by the way, missed the 'd'.

Member

rgommers commented Jun 10, 2013

You're right by the way, missed the 'd'.

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 10, 2013

Member

It's well-defined enough - disable everything that's failing on OS X 10.8 right now.

Member

rgommers commented Jun 10, 2013

It's well-defined enough - disable everything that's failing on OS X 10.8 right now.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jun 10, 2013

Member

I think disabling is not a satisfactory solution, because: the tests show that something fails, we do not understand why. However, they do not show that the parts for which the tests pass function correctly, rather, they show they work in some test cases.

It will also break portability of Scipy code across platforms.

Moreover, the situation has apparently got worse with new OSX versions, as previously it used to be so that the problems were restricted to single precision. Next version of OSX --- more problems?

Member

pv commented Jun 10, 2013

I think disabling is not a satisfactory solution, because: the tests show that something fails, we do not understand why. However, they do not show that the parts for which the tests pass function correctly, rather, they show they work in some test cases.

It will also break portability of Scipy code across platforms.

Moreover, the situation has apparently got worse with new OSX versions, as previously it used to be so that the problems were restricted to single precision. Next version of OSX --- more problems?

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 10, 2013

Member

Could be that OS X 10.9 has even more issues, we'll find out soon enough.

There's also the -ff2c solution still, gh-280. On that same PR David says that MKL has similar issues as Accelerate. OpenBLAS and GotoBLAS are not well supported by numpy.distutils. Bento support is better, but has 64-bit issues. So what are we going to recommend, use Netlib BLAS/LAPACK or ATLAS only?

I'm going to try one more time: all test failures are in the generalized-symmetric support, as added in gh-25. It was added for 0.10.0, and started given issues soon after. Disabling generalized-symmetric support (a feature used by a tiny fraction of users) either on OS X or on all platforms would be a lower-impact change than completely removing support for Accelerate.

@jakevdp @cournape any opinion on this? Time to have a go at fixing it? I believe you both have an OS X 10.8 system:)

Member

rgommers commented Jun 10, 2013

Could be that OS X 10.9 has even more issues, we'll find out soon enough.

There's also the -ff2c solution still, gh-280. On that same PR David says that MKL has similar issues as Accelerate. OpenBLAS and GotoBLAS are not well supported by numpy.distutils. Bento support is better, but has 64-bit issues. So what are we going to recommend, use Netlib BLAS/LAPACK or ATLAS only?

I'm going to try one more time: all test failures are in the generalized-symmetric support, as added in gh-25. It was added for 0.10.0, and started given issues soon after. Disabling generalized-symmetric support (a feature used by a tiny fraction of users) either on OS X or on all platforms would be a lower-impact change than completely removing support for Accelerate.

@jakevdp @cournape any opinion on this? Time to have a go at fixing it? I believe you both have an OS X 10.8 system:)

@jakevdp

This comment has been minimized.

Show comment
Hide comment
@jakevdp

jakevdp Jun 11, 2013

Member

I hate to admit it, but I still haven't been able to successfully build scipy from source on my mac. I use macports there, and have been doing all my scipy development work on my old linux machine 😬

Regarding generalized eigenvalue problems... I'd hate to completely remove it, because it works fine on most systems and there is probably someone out there who finds it useful. What if we move it to a category along the lines of "unsupported and untested: use at your own peril"? Is there any precedent for that?

Member

jakevdp commented Jun 11, 2013

I hate to admit it, but I still haven't been able to successfully build scipy from source on my mac. I use macports there, and have been doing all my scipy development work on my old linux machine 😬

Regarding generalized eigenvalue problems... I'd hate to completely remove it, because it works fine on most systems and there is probably someone out there who finds it useful. What if we move it to a category along the lines of "unsupported and untested: use at your own peril"? Is there any precedent for that?

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jun 11, 2013

Member

I am still not convinced that disabling features solves the issue --- we may certainly get fewer test failures, but this is just hiding the issue. We know that some calls to Accelerate fails to give correct results. What we do not know is whether some of these calls are also made in the usual eigenproblem case, in code paths not hit by the simple set of test cases.

Moreover, the problem is not limited to ARPACK --- remember that also scipy.linalg has had failures reported on OSX: http://scipy-user.10969.n7.nabble.com/Fwd-scipy-test-fails-for-0-11-0-on-OS-X-10-8-2-td4039.html

Basically, anything in Scipy that calls certain LAPACK/BLAS routines (which?) is suspect, and we cannot just go removing these features.

One possible fix is to go an try to plug all lapack FUNCTION calls even for double precision, as this is one place where the ABI incompatibility issue can come and bite.

Member

pv commented Jun 11, 2013

I am still not convinced that disabling features solves the issue --- we may certainly get fewer test failures, but this is just hiding the issue. We know that some calls to Accelerate fails to give correct results. What we do not know is whether some of these calls are also made in the usual eigenproblem case, in code paths not hit by the simple set of test cases.

Moreover, the problem is not limited to ARPACK --- remember that also scipy.linalg has had failures reported on OSX: http://scipy-user.10969.n7.nabble.com/Fwd-scipy-test-fails-for-0-11-0-on-OS-X-10-8-2-td4039.html

Basically, anything in Scipy that calls certain LAPACK/BLAS routines (which?) is suspect, and we cannot just go removing these features.

One possible fix is to go an try to plug all lapack FUNCTION calls even for double precision, as this is one place where the ABI incompatibility issue can come and bite.

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 11, 2013

Member

OK let's give it a try. Accelerate in numpy should be removed as well for the same reason then. Guess we'll need to write a good guide on how to compile with other BLAS/LAPACK libs and refer anyone who can't figure it out to HomeBrew.

I'll bring it up on the mailing list - maybe that triggers someone on OS X 10.8 to have a go at fixing things.

Member

rgommers commented Jun 11, 2013

OK let's give it a try. Accelerate in numpy should be removed as well for the same reason then. Guess we'll need to write a good guide on how to compile with other BLAS/LAPACK libs and refer anyone who can't figure it out to HomeBrew.

I'll bring it up on the mailing list - maybe that triggers someone on OS X 10.8 to have a go at fixing things.

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 11, 2013

Member

We have a hosted 10.8 Mac Mini for testing by the way. I'll set up some different lapacks there. I can create an account for you as well if you want.

Member

rgommers commented Jun 11, 2013

We have a hosted 10.8 Mac Mini for testing by the way. I'll set up some different lapacks there. I can create an account for you as well if you want.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jun 11, 2013

Member

Yes --- actually, I think it's overall a bit dangerous that Numpy finds Accelerate/Veclib but does not provide compiler flags that ensure the correct Fortran ABI is used.

Scipy can perhaps hack around this by avoiding FUNCTION calls, but this is not true for third-party modules that just want get_info('lapack_opt')

Member

pv commented Jun 11, 2013

Yes --- actually, I think it's overall a bit dangerous that Numpy finds Accelerate/Veclib but does not provide compiler flags that ensure the correct Fortran ABI is used.

Scipy can perhaps hack around this by avoiding FUNCTION calls, but this is not true for third-party modules that just want get_info('lapack_opt')

@ghost ghost assigned cournape Jun 11, 2013

@cournape

This comment has been minimized.

Show comment
Hide comment
@cournape

cournape Jun 11, 2013

Member

I am a bit swamped at work ATM, but we can look at that during scipy conference.

Member

cournape commented Jun 11, 2013

I am a bit swamped at work ATM, but we can look at that during scipy conference.

@joernhees

This comment has been minimized.

Show comment
Hide comment
@joernhees

joernhees Jun 11, 2013

Contributor

@rgommers i've summarized how to get scipy running from scratch here: http://joernhees.de/blog/2013/06/08/mac-os-x-10-8-scientific-python-with-homebrew/
Summarizing for scipy:

# set up some taps and update brew
brew tap homebrew/science # a lot of cool formulae for scientific tools
brew samueljohn/python # numpy, scipy
brew update && brew upgrade

# install a brewed python
brew install python

# install openblas (otherwise scipy's arpack tests will fail)
brew install openblas

brew install numpy --with-opneblas
brew install scipy --with-openblas

That way all tests in scipy

Ran 4831 tests in 54.472s

OK (KNOWNFAIL=11, SKIP=14)

and numpy

Ran 4771 tests in 19.594s

OK (KNOWNFAIL=5, SKIP=6)

pass.

Thanks to @samueljohn it should be fairly simple to extract the needed compile options.

Contributor

joernhees commented Jun 11, 2013

@rgommers i've summarized how to get scipy running from scratch here: http://joernhees.de/blog/2013/06/08/mac-os-x-10-8-scientific-python-with-homebrew/
Summarizing for scipy:

# set up some taps and update brew
brew tap homebrew/science # a lot of cool formulae for scientific tools
brew samueljohn/python # numpy, scipy
brew update && brew upgrade

# install a brewed python
brew install python

# install openblas (otherwise scipy's arpack tests will fail)
brew install openblas

brew install numpy --with-opneblas
brew install scipy --with-openblas

That way all tests in scipy

Ran 4831 tests in 54.472s

OK (KNOWNFAIL=11, SKIP=14)

and numpy

Ran 4771 tests in 19.594s

OK (KNOWNFAIL=5, SKIP=6)

pass.

Thanks to @samueljohn it should be fairly simple to extract the needed compile options.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jun 11, 2013

Member

@joernhees: This is BTW likely an incompatible Fortran ABI issue, so recompiling with -ff2c compilation and link flags may also fix the issue with Accelerate (I remember someone reporting that this worked). Environment variables FOPT, FFLAGS and LDFLAGS control these AFAIK.

Macports includes the ff2c for Octave, which probably has similar problems. On OSX Scipy includes essentially a workaround similar to dotwrp, which worked OK for previous OSX versions, but something apparently changed on 10.8 (and 10.7?).

Member

pv commented Jun 11, 2013

@joernhees: This is BTW likely an incompatible Fortran ABI issue, so recompiling with -ff2c compilation and link flags may also fix the issue with Accelerate (I remember someone reporting that this worked). Environment variables FOPT, FFLAGS and LDFLAGS control these AFAIK.

Macports includes the ff2c for Octave, which probably has similar problems. On OSX Scipy includes essentially a workaround similar to dotwrp, which worked OK for previous OSX versions, but something apparently changed on 10.8 (and 10.7?).

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Jun 11, 2013

Member

Thanks @joernhees.

@pv there's apparently also a new build issue with Accelerate (maybe came with OS X update), I've seen it reported in a couple of places now:
http://stackoverflow.com/questions/16269161/installing-issues-for-scipy-on-mac
http://charlesmartinreid.com/wiki/Scipy_vImage.h_Problem

Member

rgommers commented Jun 11, 2013

Thanks @joernhees.

@pv there's apparently also a new build issue with Accelerate (maybe came with OS X update), I've seen it reported in a couple of places now:
http://stackoverflow.com/questions/16269161/installing-issues-for-scipy-on-mac
http://charlesmartinreid.com/wiki/Scipy_vImage.h_Problem

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jun 27, 2013

Member

Ok, I just managed to get the tests to fail on OSX 10.8.4 + clang + gfortran 4.2, when using ff2c. (@rgommers this is on the scipy test machine).

Moreover, I isolated a pure-Fortran example that fails when linked with Arpack --- everyone can try this out: https://gist.github.com/pv/5873146

So the Fortran ABI issue is partly a red herring, there's a real bug somewhere either in ARPACK or in Accelerate.
It sounds like the bug is in Accelerate, as the tests pass on other BLAS+LAPACK libraries. However, it may also be an ARPACK bug (e.g. depending on a certain level of rounding error --- but that sounds still unlikely if it works on MKL etc.), or, an incompatibility in the Fortran ABI that is not fixed by -ff2c (also this sounds unlikely, and it would then be an Accelerate bug).

It seems the plan of action for Scipy should be to drop support for Accelerate.

Member

pv commented Jun 27, 2013

Ok, I just managed to get the tests to fail on OSX 10.8.4 + clang + gfortran 4.2, when using ff2c. (@rgommers this is on the scipy test machine).

Moreover, I isolated a pure-Fortran example that fails when linked with Arpack --- everyone can try this out: https://gist.github.com/pv/5873146

So the Fortran ABI issue is partly a red herring, there's a real bug somewhere either in ARPACK or in Accelerate.
It sounds like the bug is in Accelerate, as the tests pass on other BLAS+LAPACK libraries. However, it may also be an ARPACK bug (e.g. depending on a certain level of rounding error --- but that sounds still unlikely if it works on MKL etc.), or, an incompatibility in the Fortran ABI that is not fixed by -ff2c (also this sounds unlikely, and it would then be an Accelerate bug).

It seems the plan of action for Scipy should be to drop support for Accelerate.

@mandli

This comment has been minimized.

Show comment
Hide comment
@mandli

mandli Jun 30, 2013

I tried to tease out an example as well but I think @pv did a better job of isolating the problem and I can confirm that the gist fails for me as well and that it is not fixed by -ff2c.

mandli commented Jun 30, 2013

I tried to tease out an example as well but I think @pv did a better job of isolating the problem and I can confirm that the gist fails for me as well and that it is not fixed by -ff2c.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jul 1, 2013

Member

Could one of you guys send a bug report to Apple, now that we have a reproducible Fortran-only test case? This issue affects everyone using ARPACK on OSX. (Also, I guess Apple has some recommended Fortran compiler, so maybe check with that too?) I don't use OSX, so I'd be happy if someone else does this.

Member

pv commented Jul 1, 2013

Could one of you guys send a bug report to Apple, now that we have a reproducible Fortran-only test case? This issue affects everyone using ARPACK on OSX. (Also, I guess Apple has some recommended Fortran compiler, so maybe check with that too?) I don't use OSX, so I'd be happy if someone else does this.

@mandli

This comment has been minimized.

Show comment
Hide comment
@mandli

mandli Jul 1, 2013

Just filed it.

mandli commented Jul 1, 2013

Just filed it.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jul 1, 2013

Member

Macports users can also verify that the issue exists on the macports-packaged arpack, and if yes, complain to macports, too.

Member

pv commented Jul 1, 2013

Macports users can also verify that the issue exists on the macports-packaged arpack, and if yes, complain to macports, too.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jul 22, 2013

Member

The ARPACK bug report is here: http://forge.scilab.org/index.php/p/arpack-ng/issues/1259/
There's some evidence appearing that it may also be a bug in ARPACK.

Member

pv commented Jul 22, 2013

The ARPACK bug report is here: http://forge.scilab.org/index.php/p/arpack-ng/issues/1259/
There's some evidence appearing that it may also be a bug in ARPACK.

@mandli

This comment has been minimized.

Show comment
Hide comment
@mandli

mandli Jul 22, 2013

I added a pointer to the Apple bug report to that discussion.

mandli commented Jul 22, 2013

I added a pointer to the Apple bug report to that discussion.

@mandli

This comment has been minimized.

Show comment
Hide comment
@mandli

mandli Jul 31, 2013

I heard back from Apple on this issue and they appear to have fixed the problem in a build of 10.9. I can confirm that the gist now provides the correct answer to the problem.

Running tests on numpy and scipy now leads to success (thankfully) on the build I am testing on as well.

mandli commented Jul 31, 2013

I heard back from Apple on this issue and they appear to have fixed the problem in a build of 10.9. I can confirm that the gist now provides the correct answer to the problem.

Running tests on numpy and scipy now leads to success (thankfully) on the build I am testing on as well.

@samueljohn

This comment has been minimized.

Show comment
Hide comment
@samueljohn

samueljohn Jul 31, 2013

That is good news, finally. So for 10.9, we can build scipy with Accelerate again and for 10.8 switching to openblas?

samueljohn commented Jul 31, 2013

That is good news, finally. So for 10.9, we can build scipy with Accelerate again and for 10.8 switching to openblas?

@joernhees

This comment has been minimized.

Show comment
Hide comment
@joernhees

joernhees Jul 31, 2013

Contributor

wow, that thanks so much for all the effort and even getting this fixed upstream 👍
I like the suggestion to use openblas on 10.8 and building with Accelerate on 10.9 again.

Contributor

joernhees commented Jul 31, 2013

wow, that thanks so much for all the effort and even getting this fixed upstream 👍
I like the suggestion to use openblas on 10.8 and building with Accelerate on 10.9 again.

@mandli

This comment has been minimized.

Show comment
Hide comment
@mandli

mandli Jul 31, 2013

I am hopeful, I will try to continue to test numpy and scipy as new builds come out.

mandli commented Jul 31, 2013

I am hopeful, I will try to continue to test numpy and scipy as new builds come out.

@pv

This comment has been minimized.

Show comment
Hide comment
@pv

pv Jul 31, 2013

Member

For 10.8, you can also apply the bug fix discussed in the Arpack bug report. This issue turned out to be a bug in Arpack, which however only manifested for certain Lapack versions (one of them being the one shipped with Accelerate).

Member

pv commented Jul 31, 2013

For 10.8, you can also apply the bug fix discussed in the Arpack bug report. This issue turned out to be a bug in Arpack, which however only manifested for certain Lapack versions (one of them being the one shipped with Accelerate).

pv added a commit to pv/scipy-work that referenced this issue Aug 4, 2013

BUG: arpack-ng: fix bug occurring on OSX or LAPACK 3.2.1
See discussion at http://forge.scilab.org/index.php/p/arpack-ng/issues/1259/

The Ritz vector purification step assumes workl(iq) still contains the
original Q matrix. This is however overwritten by the call to xGEQR2
earlier.

This patch fixes the issue by making a copy of the last row of the eigenvector
matrix, after it is recomputed after QR by xORM2R. The work space
WORKL(IW+NCV:IW+2*NCV) used for the TAU values of the QR decomposition is not
used later in the routine, and can be used to store a copy of the last row.

Fixes gh-2547 (provided you use -ff2c flag for compiling)

Thanks to Michael Wimmer for tracing the issue.
@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Aug 6, 2013

Member

gh-2684 merged. Thanks to @pv and @michaelwimmer for fixing this.

Member

rgommers commented Aug 6, 2013

gh-2684 merged. Thanks to @pv and @michaelwimmer for fixing this.

@samueljohn

This comment has been minimized.

Show comment
Hide comment
@samueljohn

samueljohn Sep 4, 2013

Update: I added 0.13.0b1 to by tap and I can confirm that these issues are no longer reported (though I get an PIL related failure where it cannot use zip). In case any one wants to try: brew update && brew reinstall scipy --devel

Though I notice that a lot of tests are skipped (S) or known (K), so I wonder if the arpack stuff has just been marked.

samueljohn commented Sep 4, 2013

Update: I added 0.13.0b1 to by tap and I can confirm that these issues are no longer reported (though I get an PIL related failure where it cannot use zip). In case any one wants to try: brew update && brew reinstall scipy --devel

Though I notice that a lot of tests are skipped (S) or known (K), so I wonder if the arpack stuff has just been marked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment