PyBufferObject fixes

[xscott]

BPO	552438
Nosy	@gvanrossum, @tim-one, @loewis
Files	bufferfix.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2002-12-05.16:58:33.000>
created_at = <Date 2002-05-05.04:26:05.000>
labels = ['interpreter-core']
title = 'PyBufferObject fixes'
updated_at = <Date 2002-12-05.16:58:33.000>
user = 'https://bugs.python.org/xscott'

bugs.python.org fields:

activity = <Date 2002-12-05.16:58:33.000>
actor = 'loewis'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2002-05-05.04:26:05.000>
creator = 'xscott'
dependencies = []
files = ['4234']
hgrepos = []
issue_num = 552438
keywords = ['patch']
message_count = 8.0
messages = ['39873', '39874', '39875', '39876', '39877', '39878', '39879', '39880']
nosy_count = 4.0
nosy_names = ['gvanrossum', 'tim.peters', 'loewis', 'xscott']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue552438'
versions = []

[xscott]

This patch fixes these problems:

1. Dangling pointer problem
2. buffer allocated by PyBuffer_New not aligned

The PyBufferObject acts differently depending on
whether it allocated the memory or if it's borrowing
the memory from a PyBufferProcs supporting object.

In the case of allocating it's own memory, I made a
slight addition that adds some padding so that the ptr
is on a sizeof(double) boundary.

In the case of borrowing another objects PyBufferProcs
memory, PyBufferObject no longer caches the pointer.
This might slow things down (probably not by much),
but it keeps PyBufferObject from working with a stale
pointer.

Normally I wouldn't do this, but since this patch
touches pretty much every function anyway, I fixed
many deviations from the Python coding style.

[xscott]

Logged In: YES
user_id=38318

Can I assign this to you or does it take admin privs?

[xscott]

Logged In: YES
user_id=38318

Apparently assigning a patch is poor form. My bad.

[tim-one]

Logged In: YES
user_id=31435

Na, assigning a bug is fine by me -- it helps to have
*someone* feel guilty <wink>. Assigning it doesn't mean it
goes to the top of the assignee's heap, though. I can't
make time to look at it this week, so it's just as well
that it got unassigned.

[gvanrossum]

Logged In: YES
user_id=6380

Note, the patch is out of date since somebody fixed some
nits with slicing, so I'm marking this as Out Of Date.

You might as well upload the new version of the file. :-)

Why do you think you need to fix the allocation? Since
allocation is done via malloc(), and malloc() guarantees
allocation for a double ("for all types"), shouldn't that be
enough??? (If it's obmalloc that you're worried about, it's
easy to force this to use the real malloc() and free().)

I hope Tim will make some time to review this (the "not this
week" comment is several months old now). Superficially it
looks like a big improvement.

[xscott]

Logged In: YES
user_id=38318

On top of the current patch being out of data, in private email,
Guido indicated that Tim thinks the code needs more
refactoring to simplify it.

I'd like to hold off on resubmitting a current patch to see how
the bytes object fairs (PEP-296). If the bytes object makes it
into the Python core, then probably the best way to simplify
and fix the implementation of the buffer object is to reduce it
nothing but a "Buffer Inspector" for other objects. (Tearing out
the b_ptr field and a lot of if statements at least.) The bytes
object could be used to implement the following calls:

    PyBuffer_FromMemory(...)
    PyBuffer_FromReadWriteMemory(...)
    PyBuffer_New(...)

In these cases, the bytes object would hold the actual
memory, and the buffer object would just be inspecting the
bytes object. I'd still stick to the strategy of having the buffer
object re-request the pointer before every use (since typically
the pointer is only valid while the GIL is held). I haven't
figured out how to handle the case when the size specified for
the buffer object gets out of whack when the inspected object
resizes. Raise an exception?

Even with these changes, there would still be some problems
in here. For instance, the hash value is easy to invalidate.

[tim-one]

Logged In: YES
user_id=31435

Since Scott is on to something else, marked this Postponed
and unassigned it.

[loewis]

Logged In: YES
user_id=21627

It appears that whatever the final outcome is, it will have little
in common with the current patch.

So I'm closing it as rejected now; Scott, if you ever see the
need to provide an updated version, feel free to either reopen
it, or submit a new patch.