PyArg_ParseTuple("s*") does not always incref object

[kristjanvalur]

BPO	10538
Nosy	@pitrou, @kristjanvalur, @vstinner, @skrah
Files	#10538.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-03-22.16:38:31.045>
created_at = <Date 2010-11-26.08:07:31.722>
labels = ['type-bug']
title = 'PyArg_ParseTuple("s*") does not always incref object'
updated_at = <Date 2012-03-22.20:34:01.313>
user = 'https://github.com/kristjanvalur'

bugs.python.org fields:

activity = <Date 2012-03-22.20:34:01.313>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2012-03-22.16:38:31.045>
closer = 'kristjan.jonsson'
components = []
creation = <Date 2010-11-26.08:07:31.722>
creator = 'kristjan.jonsson'
dependencies = []
files = ['24995']
hgrepos = []
issue_num = 10538
keywords = ['patch']
message_count = 8.0
messages = ['122445', '124098', '156582', '156587', '156591', '156597', '156614', '156617']
nosy_count = 5.0
nosy_names = ['pitrou', 'kristjan.jonsson', 'vstinner', 'skrah', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10538'
versions = ['Python 2.7']

[kristjanvalur]

The new "s*" code for PyArg_ParseTuple is used to fill a Py_buffer object from the arguments. This object must be relased using PyBuffer_Release() after use.

However, if the object in the tuple does not support the new buffer interface, the old buffer interface is queried and the Py_buffer object is manually filled in. For this case, the source object is _not_ increfed and buffer.obj remains set to 0.

This causes different semantics in the function for objects that are passed in: If the Py_buffer interface is supported directly, then it is safe for the function to store this and release this at a later time. If it isn't supported, then no extra reference to the object is got and the function cannot safely keep the Py_buffer object around.

The Fix is as follows:  Change line 1402 of getargs.c from:
PyBuffer_FillInfo(view, NULL, buf, count, 1, 0);
to
PyBuffer_FillInfo(view, arg, buf, count, 1, 0);

[kristjanvalur]

Well, I can submit a patch if anyone is interested.
I came across this when writing asynchronous network code. By hanging onto the Py_buffer, I should have access to the data during the network call. But it only worked for "true" Py_buffer objects and not the others.

[kristjanvalur]

Adding a patch here.

[pitrou]

Looks good to me.

[python-dev]

New changeset 17c671529f7e by Kristján Valur Jónsson in branch '2.7':
Issue bpo-10538. Put a reference to the source object in the Py_buffer when
http://hg.python.org/cpython/rev/17c671529f7e

[vstinner]

You should mention your change in Misc/NEWS.

[python-dev]

New changeset 8efe88c0f14e by krisvale in branch '2.7':
Issue bpo-10538 - Update Misc/NEWS
http://hg.python.org/cpython/rev/8efe88c0f14e

[vstinner]

Thanks :-)