numpy.dot crash with numpy.float64 input

[EMCP]

Im just beginning to dive into python, so please pardon my ignorance .. I seem to have stumbled on a matrix that when attempting to perform a np.dot( on it .. along with it's transpose.. it seg faults.

I'm in OSX 10.12, Python 2.7.13 and reading some of the past things people ask. My guess is I need to do something with BLAS ?

>>> import numpy
>>> numpy.show_config()
lapack_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
openblas_lapack_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE
atlas_blas_threads_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
blas_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
blis_info:
  NOT AVAILABLE
atlas_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
>>> 

Additional info Edit :

>>> numpy.version.version
'1.12.1'

[njsmith]

Can you paste the exact code you're running and the output you get?

Also, how did you install numpy?

[EMCP]

Installed everything simply with pip install , I will bet the next thing to try may be compiling numpy and openBLAS on my mac.. yes? I am running an open source wrapper , see here :

https://github.com/lvdmaaten/bhtsne/blob/master/bhtsne.py#L108-L110

    if use_pca:
        print "Using PCA ..."
        samples = samples - np.mean(samples, axis=0)
        gettranspose = np.transpose(samples)
        cov_x = np.dot(gettranspose, samples)
        [eig_val, eig_vec] = np.linalg.eig(cov_x)

for debug purposes I pulled out the transpose step, and indeed it worked. It is when performing np.dot() it seg faults

output is very verbose.. the main crash is

    __TEXT                 000000010173e000-0000000101740000 [    8K] r-x/rwx SM=COW  /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python

Application Specific Information:
*** multi-threaded process forked ***
crashed on child side of fork pre-exec

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libdispatch.dylib             	0x00007fff8b024749 _dispatch_queue_push + 171
1   libBLAS.dylib                 	0x00007fff728dc63e dsyrk_r + 409
2   libBLAS.dylib                 	0x00007fff728dcdeb APL_dsyrk + 396
3   libBLAS.dylib                 	0x00007fff729c5eb6 cblas_dsyrk + 773
4   multiarray.so                 	0x00000001020b9a52 syrk + 306
5   multiarray.so                 	0x00000001020b9786 cblas_matrixproduct + 4950
6   multiarray.so                 	0x0000000102083230 PyArray_MatrixProduct2 + 240
7   multiarray.so                 	0x000000010208807d array_matrixproduct + 269
8   org.python.python             	0x00000001017d50b6 PyEval_EvalFrameEx + 29116
9   org.python.python             	0x00000001017cdced PyEval_EvalCodeEx + 1676
10  org.python.python             	0x00000001017d9200 fast_function + 117
11  org.python.python             	0x00000001017d4fd0 PyEval_EvalFrameEx + 28886
12  org.python.python             	0x00000001017cdced PyEval_EvalCodeEx + 1676
13  org.python.python             	0x00000001017d9200 fast_function + 117
14  org.python.python             	0x00000001017d4fd0 PyEval_EvalFrameEx + 28886
15  org.python.python             	0x00000001017cdced PyEval_EvalCodeEx + 1676
16  org.python.python             	0x00000001017d9200 fast_function + 117
17  org.python.python             	0x00000001017d4fd0 PyEval_EvalFrameEx + 28886
18  org.python.python             	0x00000001017cdced PyEval_EvalCodeEx + 1676
19  org.python.python             	0x00000001017cd65b PyEval_EvalCode + 48
20  org.python.python             	0x00000001017f28ef run_mod + 53
21  org.python.python             	0x00000001017f2999 PyRun_FileExFlags + 133
22  org.python.python             	0x00000001017f2490 PyRun_SimpleFileExFlags + 692
23  org.python.python             	0x0000000101803f18 Py_Main + 3120
24  libdyld.dylib                 	0x00007fff8b045235 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000100  rbx: 0x000000000000006f  rcx: 0x0000000000002000  rdx: 0x0000000000002000
  rdi: 0x00007fff93f2f340  rsi: 0x00000001040013c0  rbp: 0x00007fff5e4c0620  rsp: 0x00007fff5e4c0388
   r8: 0x00000000200020ff   r9: 0x00000000ffffffff  r10: 0x00000000ffff0000  r11: 0xffffffedf908af08
  r12: 0x0000000000000200  r13: 0x00007fed5931de00  r14: 0x0000000000000200  r15: 0x0000000000000004
  rip: 0x00007fff8b024749  rfl: 0x0000000000010206  cr2: 0x0000000000000110
  
Logical CPU:     4
Error Code:      0x00000006
Trap Number:     14

[juliantaylor]

is your application using fork or multiprocessing?
apples accelerate framework does not support forking on the child side, so you will have to use openblas (pthread or intel openmp version, not gnu openmp for the same reason)

[pv]

There seems to be a fork there: https://github.com/lvdmaaten/bhtsne/blob/master/bhtsne.py#L199 --- this use of Accelerate is no supported by Apple, and they refuse to fix it.

[njsmith]

Good catch!

Cc @matthew-brett as another data point in the discussion about whether accelerate or openblas would make a better default on MacOS.

[EMCP]

@juliantaylor thank you, regarding openBLAS installation.. I imagine I just follow their instructions and I will pick up the correct version? I'll keep an eye out for the pthread or intel version as I follow this along ..

Thank you @njsmith @pv as well.

Edit : Went ahead and installed from source ... retrying the calculation with the following. Unsure how I check which implementation I have of openBLAS though. Secondly, seems recommended I also reinstall Sci-py from source as well.. I'll wait on that unless people say otherwise

>>> np.show_config()
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/opt/openblas/lib']
    define_macros = [('HAVE_CBLAS', None)]
    language = c
    runtime_library_dirs = ['/usr/local/opt/openblas/lib']
blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/opt/openblas/lib']
    define_macros = [('HAVE_CBLAS', None)]
    language = c
    runtime_library_dirs = ['/usr/local/opt/openblas/lib']
openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/opt/openblas/lib']
    define_macros = [('HAVE_CBLAS', None)]
    language = c
    runtime_library_dirs = ['/usr/local/opt/openblas/lib']
blis_info:
  NOT AVAILABLE
openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/opt/openblas/lib']
    define_macros = [('HAVE_CBLAS', None)]
    language = c
    runtime_library_dirs = ['/usr/local/opt/openblas/lib']
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
>>> 

[EMCP]

Update, calculation is passing now! Thank you again everyone. Closing this ticket myself.. hope that's okay.