SharedMemory object implement non-uniform size behaviors.

[chaddnn9]

Bug report

Bug description:

Here's the python behavior:

The object which created SharedMemory does enforce the size specified.
As such, myblock read/writes work within the bounds of size.

>>> myblock = shared_memory.SharedMemory(create=True, name='MYBLOCK', size=8)
>>> myblock.size
8
>>> myblock.buf[7]=7
>>> myblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x07'

Expectedly, errors are thrown beyond the bounds of size.

>>> myblock.buf[4095]=3
Traceback (most recent call last):
  File "<python-input-37>", line 1, in <module>
    myblock.buf[4095]=3
    ~~~~~~~~~~~^^^^^^
IndexError: index out of bounds on dimension 1
>>> myblock.buf[4094:4095]=b'k'
Traceback (most recent call last):
  File "<python-input-39>", line 1, in <module>
    myblock.buf[4094:4095]=b'k'
    ~~~~~~~~~~~^^^^^^^^^^^
ValueError: memoryview assignment: lvalue and rvalue have different structures
>>>

However, objects attaching to the same block may not have the created size bounds.

>>> from multiprocessing.shared_memory import SharedMemory
>>> attachblock = SharedMemory(name='MYBLOCK')
>>> attachblock.size
4096
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\x00\--excluded--'

Attached objects may read/write up to the nearest system page size without errors thrown.

>>> attachblock.buf[4095]=3
>>> attachblock.buf[4090:4091]=b'k'
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\--excluded--\x00k\x00\x00\x00\x00\x03'
>>>

With the stark contrast of object behavior between create=True and create=False, I would strongly argue that this behavior should be considered a bug until there is sufficient documentation describing the use case for a single named SharedMemory to both employ and present non-uniform size.

As a bugfix, I would suggest any of a few mutually exclusive solutions to provide uniform size and function.

Solution 1 [create]:

When create=True & size is specified: During instantiation, the memory size allocated by the OS is queried, and reflected in the python object's size. Subsequently, the python object allows read/writes up to the allocated size.

• Brings the implementation closer towards current documentation, needing less rework in the doc.
• Likely little work necessary on the code as it closely matches the current attachment behavior.

Solution 2 [attach]:

When create=False: allow read/writes up to only the size specified when the named SharedMemory was created.

• Python intuitive behavior i.e. you get what you expect.
• In essence, repeats name implementation pattern on size.

Solution 3: [match_system or nearest_size parameter]

Alternatively, I'd suggest a new default parameter match_system=False or nearest_size=False be created for SharedMemory which controls whether the memory size allocated by the OS is queried, and reflected in the python object's size.

---

Additional historical context: There is an old thread specific to documentation. #101623 (comment)
However, in light of the demonstrated behaviors, the doc is actually not accurate nor insightful for what is going on & also should be revised to reflect the behaviors that python developers should expect when interacting with the callable. That may be a separate task from a bug.

CPython versions tested on:

3.14

Operating systems tested on:

Windows

[picnixz]

It's the documented behavior. size doesn't guarantee the exact size that will be allocated. See:

The requested number of bytes when creating a new shared memory block. Because some platforms choose to allocate chunks of memory based upon that platform’s memory page size, the exact size of the shared memory block may be larger or equal to the size requested. When attaching to an existing shared memory block, the size parameter is ignored

I don't understand the problem in your examples. Your first example uses size=8 while your second has an unbounded size which defaults to 4096 on your system and as you can see you can correctly read/write on those 4096 bytes.

[chaddnn9]

It's the documented behavior.

Not quite. What's the use case for that?

[picnixz]

Again, what is the problem in your examples? I still don't understand what you actually want. What is the behavior you expect (show an example)

[chaddnn9]

In a block of named SharedMemory, the same size bounds should always be enforced for the same name.

For multiple examples, I would be happy with any of these 3 as expected behaviors:

>>> myblock = shared_memory.SharedMemory(create=True, name='MYBLOCK', nearest_size=True, size=8)
>>> myblock.size
4096

>>> attachblock = SharedMemory(name='MYBLOCK)
>>> attachblock.size
4096

---

>>> myblock = shared_memory.SharedMemory(create=True, name='MYBLOCK', size=8)
>>> myblock.size
8

>>> attachblock = SharedMemory(name='MYBLOCK)
>>> attachblock.size
8

---

>>> myblock = shared_memory.SharedMemory(create=True, name='MYBLOCK', size=8)
>>> myblock.size
4096

>>> attachblock = SharedMemory(name='MYBLOCK)
>>> attachblock.size
4096

[picnixz]

cc @gpshead

[gpshead]

I get the desire for consistency. It'd be reasonable to honor the size= parameter on attaching to a named segment (erroring if a size greater than the actual allocation were asked for) to allow applications to do their own handshaking for a consistent behavior from all who attach. There is no good way to communicate the application level odd size within multiprocessing though, that always requires a side channel. By definition they're going to be pages as that is how the OS and hardware MMUs for most platforms work.

Were I designing this API, I'd actually have had SharedMemory(create=True, name='MYBLOCK', size=8).size == 4096 from the start as it does not make sense to think of shared memory as anything less granular than what the platform allows for. But at this point we must assume people write code that depends on the current behavior in the create=True process.