memoview[0] creates an invalid view if ndim != 1

[vstinner]

BPO	8305
Nosy	@gvanrossum, @birkenfeld, @mdickinson, @pitrou, @scoder, @vstinner, @benjaminp, @skrah
Superseder	bpo-10181: Problems with Py_buffer management in memoryobject.c (and elsewhere?)
Files	memoryview_crash.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-02-09.22:33:54.293>
created_at = <Date 2010-04-03.15:24:41.446>
labels = ['interpreter-core', 'type-crash']
title = 'memoview[0] creates an invalid view if ndim != 1'
updated_at = <Date 2012-03-02.07:49:15.344>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2012-03-02.07:49:15.344>
actor = 'skrah'
assignee = 'teoliphant'
closed = True
closed_date = <Date 2012-02-09.22:33:54.293>
closer = 'skrah'
components = ['Interpreter Core']
creation = <Date 2010-04-03.15:24:41.446>
creator = 'vstinner'
dependencies = []
files = ['16749']
hgrepos = []
issue_num = 8305
keywords = []
message_count = 8.0
messages = ['102270', '102271', '102278', '103956', '141860', '152993', '154242', '154747']
nosy_count = 10.0
nosy_names = ['gvanrossum', 'georg.brandl', 'teoliphant', 'mark.dickinson', 'pitrou', 'scoder', 'vstinner', 'benjamin.peterson', 'skrah', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '10181'
type = 'crash'
url = 'https://bugs.python.org/issue8305'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2', 'Python 3.3']

[vstinner]

memory_item() function creates a new memoryview object if ndim is different than 1. Example with numpy:

   from numpy import array
   y=array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
   y.shape = 3,4
   view=memoryview(y)
   view2 = view[0]
   print type(view2)

Result: <type 'memoryview'>. (Without the shape, ndim equals 1, and view2 is a standard str object.)

The problem is that view attribute of the view2 (PyMemoryViewObject) is not initialized. Extract of memory_item():

        /* Return a new memory-view object */
        Py_buffer newview;
        memset(&newview, 0, sizeof(newview));
        /* XXX:  This needs to be fixed so it actually returns a sub-view */
        return PyMemoryView_FromBuffer(&newview);

"This needs to be fixed" :-/

--

view2 is not initialized and so view2.tolist() does crash.

[vstinner]

Full example (using numpy) crashing Python: memoryview_crash.py

[vstinner]

According to bpo-2394, the implementation of the new buffer API is not complete.

[pitrou]

I'll leave this to our numpy experts. I don't even know how to unit test this without installing numpy.

[skrah]

The crash is fixed in the features/pep-3118 repo:

>>> from numpy import array
>>> y=array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
>>> y.shape = 3,4
>>> view=memoryview(y)
>>> view2 = view[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NotImplementedError: multi-dimensional sub-views are not implemented
[182251 refs]

[skrah]

Fixed in bpo-10181.

[python-dev]

New changeset 3f9b3b6f7ff0 by Stefan Krah in branch 'default':

• Issue bpo-10181: New memoryview implementation fixes multiple ownership
  http://hg.python.org/cpython/rev/3f9b3b6f7ff0

[skrah]

Since this issue targeted 2.7 and 3.2:

In a brief discussion on python-dev it was decided that the 3.3 fixes
from bpo-10181 won't be backported for a number of reasons, see:

http://mail.python.org/pipermail/python-dev/2012-February/116872.html