Bus error using 1.15.0rc2 openblas linked wheels (via non python threads) #11551
Comments
|
Certainly worth a try. How are you calling the function without going through Python? I note that NumPy doesn't do well as a subprocess. @matthew-brett Thoughts? |
|
@matthew-brett Are the Mac wheels actually built against openblas or are they using accelerate? |
|
Yes, looks like we're building against OpenBLAS : https://travis-ci.org/MacPython/numpy-wheels/jobs/401864648#L5256 |
The call does go through python. The difference is that the native thread is allocated and started by Qt's runtime instead of the python's. The python code is then invoked via the |
|
@matthew-brett Probably worth trying OpenBLAS 0.3.1. I'd like to release NumPy 1.15.0 out in about week, but if we update OpenBLAS I could do a rc3 sooner just to make sure nothing obvious breaks. |
|
I just merged OpenBLAS 0.3.1 builds into numpy-wheels/master - can you branch from there? |
Bah. Scratch that. Linking with OpenBLAS 0.3.0 homebrew build also worked. The differences seems to be in the OpenBLAS build parameter |
|
Hum - what NUM_THREADS does homebrew use? |
|
It does not specify the parameter (openblas.rb) so I assume it takes the current number or cores. When I build it with no parameter it builds with |
|
Can you try building the homebrew version of 0.3.0 with 64 threads, and verify it reproduces the crash? I'm worried it may be some other aspect of the build. |
|
Yes rebuilding and linking OpenBLAS 0.3.0 with NUM_THREADS=64 reproduces the crash. |
|
Sorry to ask, but how about rebuilding and linking with NUM_THREADS=4 |
|
Unfortunately, we can't really ship wheels with |
|
The problem actually seems to be insufficient stack size of the QThreadPool created threads (well actually Apple's default stack size for When OpenBLAS is build with NUM_THREADS=64 it allocates 512 KB on the stack just for the jobs array in Explicitly setting the the stack size to 8MB for the threads in the QThreadPool (QThreadPool::setStackSize) fixes the problem for both OpenBLAS 0.3.0 and 0.3.1 with NUM_THREADS=64. |
|
I guess you can close this as the problem lies somewhere else. Sorry for the trouble. |
No problem, I think we have all learned something worth knowing. |
|
Seconded - thanks for tracking this down. |
The default thread stack size for core worker io thread on macOS is insufficient to run some python code (e.g. import numpy numpy/numpy#11551). Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
…oject#41360) The default thread stack size for core worker io thread on macOS is insufficient to run some python code (e.g. import numpy numpy/numpy#11551). Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
Using numpy 1.15.0rc2 wheels,
numpy.dotfaults with aBus error: 10if the call is made from a non main thread which is not a 'Python native' thread; in particular a Qt5QThreadPool's managed thread (via PyQt5).Code
This faults in:
(the full report)
Unfortunately I cannot duplicate this with Python's
threadingmodule. I am not sure why that is - openblas, python and Qt5 use the same posix threading model underneath.Building numpy 1.15.0rc2 against OpenBLAS 3.1.0 (from homebrew) seems to fix this. OpenBLAS 3.1.0 release notes do feature a note: 'rewritten thread initialization code with significantly reduced overhead'
Is it too late in the 1.15.0 release plan to rebuild and retest with OpenBLAS 3.1.0?
Python: 3.6.5
Platform: macOS (10.11.6)
Numpy: 1.15.0rc2 with openblas 3.0.0 (pypi wheel)
The text was updated successfully, but these errors were encountered: