gh-103740: Improve support of __slots__ for variable-length classes

[serhiy-storchaka]

• Allow defining the dict and weakref slots for any class.
• Allow defining any slots for a class derived from the type class or other "variable-length" built-in type with the Py_TPFLAGS_ITEMS_AT_END flag.

• Issue: Allow __slots__ on classes with Py_TPFLAGS_ITEMS_AT_END #103740

---

📚 Documentation preview 📚: https://cpython-previews--141636.org.readthedocs.build/

[picnixz]

Well.. that's weird. The docs still fail?

[serhiy-storchaka]

Thank you. I noticed this error yesterday (this is is the second time I make such error in a month), but it was already late night.

[picnixz]

No worry! don't hesitate to ping in the future if you encounter other Sphinx errors!

[encukou]

Thank you!
I need a bit more time to review the code; sending docs suggestions in the meantime.

[serhiy-storchaka]

Maybe use "they"?

Suggested change

	For example, *__slots__* other than *__dict__* and *__weakref__* cannot
	be defined on subclasses of :class:`int`, :class:`bytes`, or :class:`tuple`.
	For example, they cannot be defined on subclasses of
	:class:`int`, :class:`bytes`, or :class:`tuple`.

[encukou]

The only Py_TPFLAGS_ITEMS_AT_END type in stdlib is type itself.
So, this PR allows slotted types. Adding data descriptors to types leads to some rather hard-to-explain behaviour, for example:

>>> class Meta(type):
...     __slots__ = ['foo']  # now possible
...     
>>> class X(metaclass=Meta):
...     def foo(self): return 'f'
...     
>>> X().foo
<bound method X.foo of <__main__.X object at 0x7f7c579c2900>>
>>> X.foo
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    X.foo
AttributeError: 'Meta' object has no attribute 'foo'
>>> X.foo = slice
>>> X.foo(X())
slice(None, <__main__.X object at 0x7f7c5788c050>, None)
>>> X().foo()
'f'

Do we want to make this easier?

[serhiy-storchaka]

I did not have any use case for this feature, but this looks interesting. We already have data descriptors (__module__, __qualname__, etc). And we can get the same with properties:

class Meta(type):
    @property
    def foo(self):
        return slice

class X(metaclass=Meta):
    def foo(self): return 'f'
 
print(X.foo(X()))
print(X().foo())

[encukou]

Yes, descriptors are already weird. But given stdlib's non-usage of Py_TPFLAGS_ITEMS_AT_END, this feature looks to be specifically about allowing descriptors for type, so I'm not sure we want it.

What about explicitly disabling arbitrary __slots__ on type subclasses, given how many optimizations assume that the __dict__ is authoritative?

---

The __dict__ & __weakref__ change looks uncontroversial -- and easier to explain, too.

[serhiy-storchaka]

Was not it the same with PEP 697? It allowed extending type in C. This PR allows to extend it in Python.

Anyway, I am only interesting in extending tuples. Adding support for special slots would be a side effect. Supporting __slots__ in the type subclasses is not necessary, it would only look inconsistent to not support it after PEP 697.

[serhiy-storchaka]

See #141755.

[encukou]

PEP 697's “extending” was about adding a C-level chunk of memory, not about Python attributes. You can already extend type in Python. It has a __dict__.

Having a __dict__ also means the memory savings from __slots__ will be tiny, unless you have many slots. (Not to mention that type itself is huge.)

[serhiy-storchaka]

The other half of this PR (very simple) has been moved to #141764.

[serhiy-storchaka]

Lets discuss this not in PRs, but in the issue #103740.