Allocating extra data after an object instance

[jbradaric]

Feature or enhancement

Add an unstable (in PEP 689 sense) function that allocates memory for a GC object and allows the caller to specify the size of additional memory that should be allocated after the object. Similar to what could be done using _PyObject_GC_Malloc function in Python <= 3.10.

Pitch

Currently there is no way of allocating extra data after an object and still being able to have a standard Python class that supports GC, allows inheriting and doesn't come with limitations of PyVarObject (see #103740).

See the linked discussion for more details, but in essence the use case is the following. We have a base class and metaclass that support defining "slots" on classes (similar to Python __slots__, but with a different behavior, more on that later). The data for the slots is stored after the object, in the same block of memory. When inheriting from a class that defines such "slots", the subclass gets all the slots from its base classes + slots defined on it. The data for all those slots is always stored after the object, so the slot storage size doesn't really come into calculation for tp_basicsize.

A nice effect of this behavior is that a subclass can inherit from multiple classes with each of the bases defining its own slots. The subclass then gets all those slots, as if they were directly defined on it (in contrast with Python __slots__, which prevents such inheritance).

CC: @encukou

Previous discussion

See this discussion for more details.

Linked PRs

• gh-103743: Add PyUnstable_Object_GC_NewWithExtraData #103744