Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

easy_install of 0.9.0 fails on OS X 10.7 (Lion) (Trac #1476) #2001

Closed
scipy-gitbot opened this issue Apr 25, 2013 · 28 comments
Closed

easy_install of 0.9.0 fails on OS X 10.7 (Lion) (Trac #1476) #2001

scipy-gitbot opened this issue Apr 25, 2013 · 28 comments
Labels
Build issues Issues with building from source, including different choices of architecture, compilers and OS defect A clear bug or issue that prevents SciPy from being installed or used as expected Migrated from Trac
Milestone

Comments

@scipy-gitbot
Copy link

Original ticket http://projects.scipy.org/scipy/ticket/1476 on 2011-07-11 by trac user tacoe, assigned to @cournape.

This is a GM build of Lion, meaning no changes are expected before release.

From the first error to the end of the trace:

/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: warning: type defaults to ‘int’ in declaration of ‘complex’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:16: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:21: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: warning: type defaults to ‘int’ in declaration of ‘complex’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: warning: type defaults to ‘int’ in declaration of ‘complex’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:16: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:21: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
lipo: can't open input file: /var/tmp//cc7t1IIv.out (No such file or directory)
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: warning: type defaults to ‘int’ in declaration of ‘complex’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: warning: type defaults to ‘int’ in declaration of ‘complex’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:16: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:21: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: warning: type defaults to ‘int’ in declaration of ‘complex’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: warning: type defaults to ‘int’ in declaration of ‘complex’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:16: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/private/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:21: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
lipo: can't open input file: /var/tmp//cc7t1IIv.out (No such file or directory)
error: Setup script exited with error: Command "llvm-gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common
-fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE 

-DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE 
-arch i386 -arch x86_64 -pipe -I/private/tmp/easy_install-N6GrEl
/scipy-0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/SRC -I/System
/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
/numpy/core/include -c /private/tmp/easy_install-N6GrEl/scipy-
0.9.0/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS
/veclib_cabi_c.c -o build/temp.macosx-10.7-intel-2.7/private
/tmp/easy_install-N6GrEl/scipy-0.9.0/scipy/sparse/linalg/eigen
/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.o" failed with exit status 1
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras
/lib/python/numpy/distutils/misc_util.py:251: RuntimeWarning: 
Parent module 'numpy.distutils' not found while handling absolute import
  from numpy.distutils import log
@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-07-12

This could be due to easy_install or your setup in addition to the OS upgrade. Could you build on Snow Leopard? Can you install on Lion with standard "python setup.py install"? Can you use the standard installed gcc instead of llvm-gcc?

@scipy-gitbot
Copy link
Author

trac user tacoe wrote on 2011-07-12

Thanks a ton for picking up on this.

I did try python setup.py install as well.
I am not (anymore) able to build on SL as that would require me to format and reinstall my system.

I'll try and see if there's a way to use gcc instead of llvm (in 10.7, gcc is not shipped anymore; I seem to remember it's not easy getting gcc outside of xcode).

@scipy-gitbot
Copy link
Author

trac user cax wrote on 2011-07-13

I also have tried to build on Lion with no luck. In my case the problem is when trying to compile scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c.

Python is installed via homebrew.

Log:

C compiler: /usr/bin/cc -fno-strict-aliasing -O3 -march=core2 -msse4.1 -w -pipe -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

compile options: '-Iscipy/sparse/linalg/eigen/arpack/ARPACK/SRC -I/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/numpy/core/include -c'
cc: scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:16: error: expected ‘;’, ‘,’ or ‘)’ before ‘’ token
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:21: error: expected ‘;’, ‘,’ or ‘)’ before ‘
’ token
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:4: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:10: error: expected ‘;’, ‘,’ or ‘)’ before ‘float’
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:16: error: expected ‘;’, ‘,’ or ‘)’ before ‘’ token
scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c:21: error: expected ‘;’, ‘,’ or ‘)’ before ‘
’ token
error: Command "/usr/bin/cc -fno-strict-aliasing -O3 -march=core2 -msse4.1 -w -pipe -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Iscipy/sparse/linalg/eigen/arpack/ARPACK/SRC -I/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/numpy/core/include -c scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c -o build/temp.macosx-10.4-x86_64-2.7/scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.o" failed with exit status 1

@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-07-13

Never mind gcc then. Quite surprising that it's not shipped with OS X anymore.

It seems to be complaining about this:

#include <vecLib/vecLib.h>

#define WRAP_F77(a) a##_
void WRAP_F77(veclib_cdotc)(const int *N, const complex float *X, const int *incX,
const complex float *Y, const int *incY, complex float *dotc)
{
    cblas_cdotc_sub(*N, X, *incX, Y, *incY, dotc);
}

in particular the "const complex float" declarations. Those are non-standard, a quoted google search for them turns up the scipy arpack source as first hit which actually has this in it. The (C++) hits before all use "const complex<float>". Not sure what the right way in C is.

@scipy-gitbot
Copy link
Author

trac user cax wrote on 2011-07-14

It seems this is an "old" issue. See http://projects.scipy.org/scipy/ticket/510. It is funny because they closed the issue by saying Panther (10.3.9) was too old. By adding #include <complex.h> in scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c seems to fix the issue. However, I've got another one:
ld: file not found: crt3.o

@scipy-gitbot
Copy link
Author

trac user deadshort wrote on 2011-07-14

Section 7.3.1 of http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf suggests that adding #include <complex.h> is the correct thing to do for all C99 compilers.

@scipy-gitbot
Copy link
Author

trac user dloewenherz wrote on 2011-07-17

Adding #include <complex.h> to the top of the file doesn't fix the installation issue for me either. Traceback:

ld: file not found: crt3.o
error: Command "/usr/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.4-x86_64-2.7/build/src.macosx-10.4-x86_64-2.7/scipy/fftpack/_fftpackmodule.o build/temp.macosx-10.4-x86_64-2.7/scipy/fftpack/src/zfft.o build/temp.macosx-10.4-x86_64-2.7/scipy/fftpack/src/drfft.o build/temp.macosx-10.4-x86_64-2.7/scipy/fftpack/src/zrfft.o build/temp.macosx-10.4-x86_64-2.7/scipy/fftpack/src/zfftnd.o build/temp.macosx-10.4-x86_64-2.7/build/src.macosx-10.4-x86_64-2.7/scipy/fftpack/src/dct.o build/temp.macosx-10.4-x86_64-2.7/build/src.macosx-10.4-x86_64-2.7/fortranobject.o -Lbuild/temp.macosx-10.4-x86_64-2.7 -ldfftpack -lfftpack -lgfortran -o build/lib.macosx-10.4-x86_64-2.7/scipy/fftpack/_fftpack.so" failed with exit status 1

@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-07-18

@cax/dloewenherz: that error could be related to the 10.4 SDK, did you install it? Did you install numpy without problems and does ">>> numpy.test('full')" complete without failures? Can you post a full build log?

@deadshort: what about compilers without (full) C99 support? There are quite a few compilers we need to support.

@scipy-gitbot
Copy link
Author

trac user dloewenherz wrote on 2011-07-18

@rgommers

  1. The full test suite fails. - FAILED (KNOWNFAIL=3, SKIP=1, errors=17, failures=8)

  2. numpy appears to install correctly. [http://pastebin.com/pZB8bg82 Build log].

  3. What do you mean by install the 10.4 SDK. The following are all the choices I see:

[http://i.imgur.com/8CXQA.png]

@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-07-18

17 errors, those are probably all the Fortran related tests in numpy.distutils. Right? Is your gfortran version 4.2.3 from http://r.research.att.com/tools/? Did you install numpy 1.6.0, or something else?

Each OS X version has an SDK, it's on the same install dvd as XCode but in a separate folder, called optional installs or something like that. You need it if you are using the Python from python.org

@scipy-gitbot
Copy link
Author

trac user dloewenherz wrote on 2011-07-18

Actually, most of the errors I see are mostly related to mathematical operations, such as sqrt. I do see some fortran-related errors, such as the following:

error: Command "/usr/local/bin/gfortran -Wall -arch i686 -arch x86_64 -Wall -undefined dynamic_lookup -bu
ndle /var/folders/hd/9xvsx9192kj_t8dwsj176ht40000gn/T/tmp23BcD2/var/folders/hd/9xvsx9192kj_t8dwsj176ht400
00gn/T/tmp23BcD2/src.macosx-10.4-x86_64-2.7/_test_ext_module_5403module.o /var/folders/hd/9xvsx9192kj_t8d
wsj176ht40000gn/T/tmp23BcD2/var/folders/hd/9xvsx9192kj_t8dwsj176ht40000gn/T/tmp23BcD2/src.macosx-10.4-x86
_64-2.7/fortranobject.o /var/folders/hd/9xvsx9192kj_t8dwsj176ht40000gn/T/tmp23BcD2/var/folders/hd/9xvsx91
92kj_t8dwsj176ht40000gn/T/tmplygZ4G/tmpz_J3jv.o /var/folders/hd/9xvsx9192kj_t8dwsj176ht40000gn/T/tmp23BcD
2/var/folders/hd/9xvsx9192kj_t8dwsj176ht40000gn/T/tmp23BcD2/src.macosx-10.4-x86_64-2.7/_test_ext_module_5
403-f2pywrappers2.o -lgfortran -o ./_test_ext_module_5403.so" failed with exit status 1

I'm using gfortran version 4.2.4. I'm running these tests off of Numpy 2.0.0.

Apple must have stopped releasing the SDKs in an optional install folder. I only see one package (Xcode.mpkg), and "About Xcode.pdf".

I have a biting suspicion that the reason for these failures is the removal of gcc from the OS X toolchain. Everything is now being compiled through LLVM.

@scipy-gitbot
Copy link
Author

trac user deadshort wrote on 2011-07-18

@rgommers:

Messy. I'm not sure there's a simple correct answer. The C99 spec recommends wrapping something or other behind _ _STDC_IEC_559_COMPLEX_ _, but it hard to read in a comfortingly solid prescription for much of anything. Pre-C99 complex was essentially undefined (i.e. implementation-dependent), right?

@toolchain:

I have been using +gcc44 +no_atlas, and scipy builds and installs, but then fails 6 BLAS-related tests. I feel I should test a few variations.

======================================================================
FAIL: test_asum (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 58, in test_asum
    assert_almost_equal(f([3,-4,5]),12)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 12

======================================================================
FAIL: test_dot (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 67, in test_dot
    assert_almost_equal(f([3,-4,5],[2,5,1]),-9)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: -9

======================================================================
FAIL: test_nrm2 (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 78, in test_nrm2
    assert_almost_equal(f([3,-4,5]),math.sqrt(50))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 7.0710678118654755

======================================================================
FAIL: test_asum (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 57, in test_asum
    assert_almost_equal(f([3,-4,5]),12)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 12

======================================================================
FAIL: test_dot (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 66, in test_dot
    assert_almost_equal(f([3,-4,5],[2,5,1]),-9)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: -9

======================================================================
FAIL: test_nrm2 (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 83, in test_nrm2
    assert_almost_equal(f([3,-4,5]),math.sqrt(50))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 7.0710678118654755

======================================================================
FAIL: test_expon (test_morestats.TestAnderson)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/stats/tests/test_morestats.py", line 72, in test_expon
    assert_array_less(crit[:-1], A)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 869, in assert_array_less
    header='Arrays are not less-ordered')
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 613, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='inf')
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 588, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not less-ordered

x and y inf location mismatch:
 x: array([ 0.911,  1.065,  1.325,  1.587])
 y: array(inf)

----------------------------------------------------------------------
Ran 4722 tests in 45.268s

FAILED (KNOWNFAIL=18, SKIP=31, failures=7)
<nose.result.TextTestResult run=4722 errors=0 failures=7>
>>> 

@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-07-19

Why no_atlas, and how? With the default build options you should pick up Accelerate, does that fix your failures?

@scipy-gitbot
Copy link
Author

trac user dloewenherz wrote on 2011-07-19

Still don't have much of an idea of what's going on here...

P.S. Lion is being released tomorrow.

@scipy-gitbot
Copy link
Author

trac user chrisfonnesbeck wrote on 2011-07-21

I can confirm this bug. I have been trying all day to get SciPy built, but this failure gets me every time. Numpy builds without issue.

Using Xcode 4.1, gfortran 4.2.3 with Python 2.7.1.

@scipy-gitbot
Copy link
Author

trac user rsenk330 wrote on 2011-07-21

I also can confirm this.

I got past the error in the ticket description above by adding #include <complex.h> to the top of scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c, but then the installation fails with:

ld: library not found for -lgcc
ld: library not found for -lgcc
lipo: can't figure out the architecture type of: /var/folders/0_/1990lx5x25n3p75zl3n6mpqw0000gn/T//cctvJOYN.out
ld: library not found for -lgcc
ld: library not found for -lgcc
lipo: can't figure out the architecture type of: /var/folders/0_/1990lx5x25n3p75zl3n6mpqw0000gn/T//cctvJOYN.out
error: Command "/usr/local/bin/gfortran -Wall -arch i686 -arch x86_64 -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.7-intel-2.7/build/src.macosx-10.7-intel-2.7/scipy/fftpack/_fftpackmodule.o build/temp.macosx-10.7-intel-2.7/scipy/fftpack/src/zfft.o build/temp.macosx-10.7-intel-2.7/scipy/fftpack/src/drfft.o build/temp.macosx-10.7-intel-2.7/scipy/fftpack/src/zrfft.o build/temp.macosx-10.7-intel-2.7/scipy/fftpack/src/zfftnd.o build/temp.macosx-10.7-intel-2.7/build/src.macosx-10.7-intel-2.7/scipy/fftpack/src/dct.o build/temp.macosx-10.7-intel-2.7/build/src.macosx-10.7-intel-2.7/fortranobject.o -Lbuild/temp.macosx-10.7-intel-2.7 -ldfftpack -lfftpack -lgfortran -o build/lib.macosx-10.7-intel-2.7/scipy/fftpack/_fftpack.so" failed with exit status 1

@scipy-gitbot
Copy link
Author

trac user cshimmin wrote on 2011-07-22

Another confirmation here. I haven't bothered messing with veclib_cabi_c.c yet... it seems from the other reports that this isn't a complete fix.

Installing from pip.

@scipy-gitbot
Copy link
Author

trac user chrisfonnesbeck wrote on 2011-07-22

A bit more info: a full test of numpy results in the following errors. As they have to do with umath complex, are they relevant to the reported bug?

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, 1, inf, inf, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array((inf+infj))

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -1, inf, inf, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array((inf+infj))

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, 0.0, inf, inf, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array((inf+infj))

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -0.0, inf, inf, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array((inf+infj))

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, inf, inf, inf, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array((inf+infj))

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -inf, inf, inf, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array((inf+infj))

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, nan, inf, inf, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array((inf+infj))

======================================================================
FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -inf, 1, 0.0, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose-1.0.0-py2.7.egg/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/core/tests/test_umath_complex.py", line 568, in check_complex_value
    assert_equal(f(z1), z2)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 256, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 706, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 604, in assert_array_compare
    chk_same_position(x_id, y_id, hasval='nan')
  File "/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-x86_64.egg/numpy/testing/utils.py", line 587, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan+infj])
 y: array(infj)

----------------------------------------------------------------------
Ran 3631 tests in 77.206s

@scipy-gitbot
Copy link
Author

trac user rsenk330 wrote on 2011-07-23

I can get this to install using gfortran from homebrew. I had to use the gfortran formula from a testing repository until it gets pushed to the master. Instructions can be found on this [https://github.com/Homebrew/legacy-homebrew/issues/6500#issuecomment-1637840 homebrew ticket].

I also had to add #include <complex.h> to the top of the following files:

  • scipy/linalg/src/fblaswrap_veclib_c.c
  • scipy/lib/blas/fblaswrap_veclib_c.c.src
  • scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c

@scipy-gitbot
Copy link
Author

trac user cshimmin wrote on 2011-07-26

It looks like the updated gfortran binary from r.research.att.com has made its way into homebrew. With the new binary and adding complex.h to the files indicated by @rsenk330, scipy installs successfully.

However, when I run scipy.test() from ipython, I get the following sever error, causing python to abort:

In [1]: import scipy
In [2]: scipy.test('full')
Running unit tests for scipy
NumPy version 1.6.1
NumPy is installed in /usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/numpy
SciPy version 0.9.0
SciPy is installed in /usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/scipy
Python version 2.7.2 (default, Jun 29 2011, 14:53:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.0.0
................................................................................
................................................................................
....F.FFpython(33717) malloc: *** error for object 0x103b584b0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

I am seeing the exact numpy test failures as @chrisfonnesbeck above. They all appear to be related to complex square roots (TestCsqrt). Is there already an open ticket for this somewhere?

@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-07-27

There's no ticket for the complex square root failures yet, but it should be unrelated. They're corner cases of the C99 spec that are being tested, fail on several other platforms and have little practical relevance.

@scipy-gitbot
Copy link
Author

trac user mhansen wrote on 2011-07-28

I can confirm that adding #include <complex.h> to the top of the following files:

  • scipy/linalg/src/fblaswrap_veclib_c.c
  • scipy/lib/blas/fblaswrap_veclib_c.c.src
  • scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c

works for me when trying to build SciPy for Sage under Lion with XCode4.

@scipy-gitbot
Copy link
Author

@pv wrote on 2011-07-30

Adding complex.h is the right fix, as C99 IIRC specifies that the complex types are defined only after that. This fix went in effa6f68

@scipy-gitbot
Copy link
Author

Milestone changed to 0.10.0 by @pv on 2011-07-30

@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-08-10

Seems that this is now working for most people with the correct gfortran compiler.

@cshimmin: if you still see the same crash, can you run the tests with verbose=2 to see where it crashes?

@scipy-gitbot
Copy link
Author

trac user cshimmin wrote on 2011-08-10

@rgommers: I am seeing at least 4 different failure modes. I have just compiled a fresh copy of scipy 0.9.0 from PyPI (after adding patching in the #include <complex.h> lines discussed here.)

Here's the (top-truncated) output from a couple of runs of scipy.test(verbose=2):

test_djbfft (test_basic.TestDoubleFFT) ... ok
test_n_argument_real (test_basic.TestDoubleFFT) ... ok
test_definition (test_basic.TestDoubleIFFT) ... FAIL
test_definition_real (test_basic.TestDoubleIFFT) ... ok
test_djbfft (test_basic.TestDoubleIFFT) ... FAIL
test_random_complex (test_basic.TestDoubleIFFT) ... FAIL
python(80905,0x7fff77bdc960) malloc: *** error for object 0x1055a2320: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6





test_djbfft (test_basic.TestDoubleFFT) ... ok
test_n_argument_real (test_basic.TestDoubleFFT) ... ok
test_definition (test_basic.TestDoubleIFFT) ... FAIL
test_definition_real (test_basic.TestDoubleIFFT) ... ok
test_djbfft (test_basic.TestDoubleIFFT) ... FAIL
test_random_complex (test_basic.TestDoubleIFFT) ... FAIL
test_random_real (test_basic.TestDoubleIFFT) ... FAIL
test_size_accuracy (test_basic.TestDoubleIFFT) ... Segmentation fault: 11

Another failure mode I saw included a warning (sorry, didn't catch this one with verbosity):

........................................
........................................
........................................
............................./usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/scipy/cluster/vq.py:582: UserWarning: One of the clusters is empty. Re-run kmean with a different initialization.
  warnings.warn("One of the clusters is empty. "
...............F.FFpython(80891,0x7fff77bdc960) malloc: *** error for object 0x10590a820: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

Finally, in the last failure mode I have observed when running scipy.test(), the python interpreter hangs at 100% cpu, and doesn't respond to sigint. It died silently upon sigterm though.

I'm guessing at least some of this non-deterministic behavior is from the test_random_* functions, but I haven't spent time looking into this.

Let me know if any additional information can help.

@scipy-gitbot
Copy link
Author

trac user cshimmin wrote on 2011-08-10

Just to be clear: although the failure modes seem non-deterministic, I should mention that I have never run test() successfully; the farthest I have seen it get is to "test_definition ... Segmentation fault 11".

And as a bonus, here are 2 new failures that I just observed:

test_djbfft (test_basic.TestDoubleFFT) ... ok
test_n_argument_real (test_basic.TestDoubleFFT) ... ok
test_definition (test_basic.TestDoubleIFFT) ... FAIL
test_definition_real (test_basic.TestDoubleIFFT) ... ok
test_djbfft (test_basic.TestDoubleIFFT) ... FAIL
test_random_complex (test_basic.TestDoubleIFFT) ... Bus error: 10





test_djbfft (test_basic.TestDoubleFFT) ... ok
test_n_argument_real (test_basic.TestDoubleFFT) ... ok
test_definition (test_basic.TestDoubleIFFT) ... FAIL
test_definition_real (test_basic.TestDoubleIFFT) ... ok
test_djbfft (test_basic.TestDoubleIFFT) ... FAIL
test_random_complex (test_basic.TestDoubleIFFT) ... FAIL
test_random_real (test_basic.TestDoubleIFFT) ... FAIL
python(80972,0x7fff77bdc960) malloc: *** error for object 0x1054c7028: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

Perhaps this issue needs its own ticket?

@scipy-gitbot
Copy link
Author

@rgommers wrote on 2011-08-18

I opened gh-2021 for the issue of cshimmin. This seems fixed for everyone else so closing this ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build issues Issues with building from source, including different choices of architecture, compilers and OS defect A clear bug or issue that prevents SciPy from being installed or used as expected Migrated from Trac
Projects
None yet
Development

No branches or pull requests

1 participant