Query regarding shared buffer

[time4tea]

Just wondering about using shared buffers. According to the docs - Image.frombuffer() is a way to share an image buffer. All the code seems to do the "right thing" - with PyImaging_MapBuffer setting up all the pointers etc. https://github.com/python-pillow/Pillow/blob/main/src/map.c#L52

However, in Image.frombuffer() it sets im.readonly to 1, ( https://github.com/python-pillow/Pillow/blob/main/src/PIL/Image.py#L2937 ) which means that any subsequent draw operations will clone the image - making it non-shared again.( https://github.com/python-pillow/Pillow/blob/main/src/PIL/ImageDraw.py#L63 )

It does seem possible to set im.readonly to 0 again after running Image.frombuffer() - and drawing works fine - but it looks like this was done on purpose - https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst - for version 1.1.2c1 -

The "load" method now sets the "readonly" attribute for memory-mapped images. Operations that modifies an image in place (such as "paste" and drawing operations) creates an in-memory copy of the image, if necessary. (before this change, any attempt to modify a memory-mapped image resulted in a core dump...)

This comment is very old! - I was wondering if it was considered safe to set im.readonly to 0 and share image data via a buffer.

something along the lines of (very approximately!)

[...]
        self._buffer_size = (dimensions.x * dimensions.y * 4)
        self._buffer = ctypes.create_string_buffer(self._buffer_size)
        self._image = Image.frombuffer("RGBA", (dimensions.x, dimensions.y), self._buffer, "raw", "RGBA", 0, 1)
        self._image.readonly = 0

Is this a very bad idea?

Thanks!

James

[wiredfool]

Some years ago I did this: #2980 (still open, and potentially still useful).

It appeared to work, but I was never certain that it was stable. I think that some of the concerns over it are now gone, as python2 is long past us.

There are of course some caveats with shared memory and concurrent access, it's very possible that bad things can happen if both programs are actually trying to modify the memory at once.

[time4tea]

Thanks for that - setting im.readonly = 0 like in the above example does appear to work today.. and I can successfully create an ImageDraw based on the image and draw things into it. I guess I was just wondering if it was recommended or conversely, "here be dragons". I'm guessing its not something that gets used much, but maybe should work.

The reason I'd like to use a shared buffer is to implement some kind of double buffer thing - where I can draw to one frame while writing another frame to ffmpeg.

Not quite sure how to do do this exactly at the moment, while negotiating the GIL. Possibly threads + Py_BEGIN_ALLOW_THREADS ? Possibly sub-process + shm... maybe it is a terrible idea!

Thanks!!

[radarhere]

@time4tea is this resolved, or are there unanswered questions here?

[time4tea]

hiya - thanks - no i guess not really. it does seem to work using the method above. but i guess its not really supported behaviour. cheers!