Segmentation fault when accesing void object of numpy module

[danieltomasz]

Segmentation fault and Spyder crash when accessing void object of numpy module (loaded from matlab file) in spyder variable explorer

import numpy as np

B = np.load('example.npy', allow_pickle=True )[()]

python 3.6.8
numpy 1.17.3
IPython 7.9.0
I am including example file here
https://www7.zippyshare.com/v/ZvMh1rkg/file.html

[ccordoba12]

@impact27, could you take a look at this one? It looks like a problem in our comms.

[impact27]

faulthandler:

Fatal Python error: Segmentation fault

Current thread 0x000000011c26adc0 (most recent call first):
  File "/Users/quentinpeter/Library/Python/3.7/lib/python/site-packages/numpy/core/arrayprint.py", line 1337 in _void_scalar_repr
  File "/Users/quentinpeter/pyscripts/spyder/spyder/plugins/variableexplorer/widgets/collectionsdelegate.py", line 292 in setEditorData
  File "/Users/quentinpeter/pyscripts/spyder/spyder/plugins/variableexplorer/widgets/collectionseditor.py", line 769 in mouseDoubleClickEvent
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/mainwindow.py", line 3562 in run_spyder
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/mainwindow.py", line 3663 in main
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/start.py", line 218 in main
  File "/Applications/Spyder.app/Contents/MacOS/Spyder", line 4 in <module>

Top of c-stack

Thread 0 Crashed:: CrBrowserMain  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff735ab49a __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff736686cb pthread_kill + 384
2   libsystem_c.dylib             	0x00007fff734c33a2 raise + 26
3   libsystem_platform.dylib      	0x00007fff7365db1d _sigtramp + 29
4   ???                           	000000000000000000 0 + 0
5   _multiarray_umath.cpython-37m-darwin.so	0x0000000104dfb9de PyArray_Item_INCREF + 222
6   _multiarray_umath.cpython-37m-darwin.so	0x0000000104dff4ce PyArray_FromScalar + 206
7   _multiarray_umath.cpython-37m-darwin.so	0x0000000104d34680 PyArray_FromAny + 752
8   _multiarray_umath.cpython-37m-darwin.so	0x0000000104d34c92 PyArray_CheckFromAny + 242
9   _multiarray_umath.cpython-37m-darwin.so	0x0000000104dd74d5 _array_fromobject + 901
10  org.python.python3            	0x0000000103c10d52 _PyMethodDef_RawFastCallKeywords + 722
11  org.python.python3            	0x0000000103c100c9 _PyCFunction_FastCallKeywords + 41
12  org.python.python3            	0x0000000103ce5100 call_function + 656
13  org.python.python3            	0x0000000103ce1c68 _PyEval_EvalFrameDefault + 27416
14  org.python.python3            	0x0000000103c105ad function_code_fastcall + 237
15  org.python.python3            	0x0000000103c11270 _PyObject_CallFunctionVa + 272
16  org.python.python3            	0x0000000103c1142e _PyObject_CallFunction_SizeT + 142
17  org.python.python3            	0x0000000103c4cf0f PyObject_Repr + 127
18  org.python.python3            	0x0000000103c10d87 _PyMethodDef_RawFastCallKeywords + 775
19  org.python.python3            	0x0000000103c100c9 _PyCFunction_FastCallKeywords + 41
20  org.python.python3            	0x0000000103ce5100 call_function + 656
21  org.python.python3            	0x0000000103ce1c68 _PyEval_EvalFrameDefault + 27416
22  org.python.python3            	0x0000000103c105ad function_code_fastcall + 237
23  org.python.python3            	0x0000000103c11053 _PyObject_Call_Prepend + 131
24  org.python.python3            	0x0000000103c101e8 PyObject_Call + 136
25  sip.cpython-37m-darwin.so     	0x000000011395b4c0 sip_api_call_procedure_method + 208
26  QtWidgets.so                  	0x000000011666a26e sipQItemDelegate::setEditorData(QWidget*, QModelIndex const&) const + 206

[impact27]

Doesn't look like it. Maybe it might be a numpy bug?
It looks like numpy tries to PyArray_Item_INCREF the NULL pointer.

[impact27]

This is a numpy error. To reproduce:

1. Open a python interpreter

>>> import numpy as np
>>> import cloudpickle
>>> B = np.load('example.npy', allow_pickle=True )[()]
>>> cloudpickle.dumps(B)

3. Open a new python interperter
4. set bin_str to the output of the first interpeter

>>> import cloudpickle
>>> cloudpickle.loads(bin_str)

The process then crashes

The same happens with pickle

[ccordoba12]

Ok, thanks for digging into this one @impact27! Is there any way to avoid de-serializing this, so we can avoid the crash in our side?

[ccordoba12]

This is a numpy error

What's the error exactly?

[impact27]

I can get the same crash with just one interpreter:

import pickle
import numpy as np
B = np.load('example.npy', allow_pickle=True )[()]
data = pickle.dumps(B)
del B
print(pickle.loads(data))

Probably pickle saves a pointer but not the underlying data. The code doesn't crash when de-serializing this, only when trying to access the data (repr(value)).
We could:

• Disallow the numpy.void datatype ()
• Do it in another process and expect that this process might crash?

[impact27]

#10617 should fix this

[danieltomasz]

@impact27 I am including (one of) original source mat file, I also included it in the numpy issue;

import scipy.io  as sio
B = sio.loadmat('example.mat')['atlas']
temp = B[0][0]

spyder still crashes when I am trying to access temp variable but I have problem to reproduce pickle behavior

np.save('example', temp)
B = np.load('example.npy', allow_pickle=True )[()]
data = pickle.dumps(B)
del B
print(pickle.loads(data))

gives normal output, but when running as a script in Spyder it gives

An error ocurred while starting the kernel
[IPKernelApp] WARNING | No such comm: 6d37879000a711ea86f2d4258b451ef1
[IPKernelApp] WARNING | No such comm: 6d37879000a711ea86f2d4258b451ef1
[IPKernelApp] WARNING | No such comm: 6d37879000a711ea86f2d4258b451ef1

in the console