Make abstract/dunder methods positional-only.

[randolf-scholz]

Feature or enhancement

It would be nice if certain abstract methods / dunder methods would be defined with positional-only arguments, so that when one subclasses them an appropriate name can be chosen, or they can be defined positional-only in the subclass.¹

Code sample in pyright playground¹

from typing import overload
from collections.abc import Sequence

class MySequence[T](Sequence[T]):
    def __len__(self): return 0
    @overload
    def __getitem__(self, idx: int, /) -> T: ...
    @overload
    def __getitem__(self, idx: slice, /) -> Sequence[T]: ...
    def __getitem__(self, idx: int | slice, /) -> T | Sequence[T]:  # ❌ override
        return MySequence[T]()

In some cases, the stubs already fake that the argument is positional-only, such as for Container.__contains__:

• def __contains__(self, x: object, /) -> bool: ... in typeshed/stdlib/typing.pyi
• def __contains__(self, x): in cpython/Lib/_collections_abc.py

Personally, I think only changing the stubs would be a pragmatic choice, but I was told by the maintainers that they believe this change should be done in Cpython first.

Backward Compatibility

1. Changing non-abstract methods can potentially break user code. For example, this is currently legal code at runtime

   from collections.abc import Container.
   Container.__subclasshook__(C=list)  # True

2. Changing the signature of abstract methods can introduce type-checker errors Code sample in pyright playground¹

   from collections.abc import Sequence
   def get(x: Sequence[float], index: int) -> float:
       return x.__getitem__(index=index)

With this in mind, I believe changing abstrac dunder methods to use positional only arguments should be relatively safe, as calling them directly with keyword is very uncommon, and at worst it will introduce type errors.

Personally, I'd also like so see methods like Container.__subclasshook__ use positional-only arguments, but I also understand if this is not feasible due to bc-conerns.

Has this already been discussed elsewhere?

python/typeshed#14071

Footnotes

1. Note that mypy special cases certain dunder methods, and yields different results than pyright on these examples ↩ ↩² ↩³

[randolf-scholz]

I did a scan of the collections.abc module to see how often certain methods are called with keyword arguments, here are the results.

• Container:
  • __contains__(x=...): abstract, yields 0 results on github search
  • __subclasshook__(C=...): non-abstract, yields 0 results on github search
• Sequence:
  • __getitem__(index=...): abstract, yields 165 results on github search
  • note that some registered subclasses like list do not even accept keyword index.
  • __contains__(value=...): non-abstract, yields 0 results on github search
  • index(value=...): non-abstract, yields 9 results on github search
  • count(value=...): non-abstract, yields 9 results on github search
• MutableSequence:
  • __setitem__(index=..., value=...): abstract, yields 0 results on github search
  • __delitem__(index=...): abstract, yields 2 results on github search
  • insert(index=..., value=...): abstract, yields 22 results on github search
  • append(value=...): non-abstract, yields 56 results on github search
  • extend(values=...): non-abstract, yields 3 results on github search
  • pop(index=...): non-abstract, yields 520 results on github search
  • These seem to be coming mostly from the blender API
  • remove(value=...): non-abstract, yields 17 results on github search
  • __iadd__(values=...): non-abstract, yields 0 results on github search
• Mapping:
  • __contains__(key=...): non-abstract, yields 36 results on github search
  • __getitem__(key=...): abstract, yields 50 results on github search
  • __eq__(other=...): non-abstract, yields 139 results on github search
  • obj.get(key=...,): non-abstract, yields 7.9k results on github search
  • obj.get(..., default=...) yields 1.3k results on github search
  • obj.get(key=..., default=...) yields 164 results on github search
  • Potentially a lot of non-Mapping hits, such as google.cloud.datastore.client.Client.get.
• MutableMapping:
  • __setitem__(key=..., value=...): non-abstract, yields 47 results on github search
  • __delitem__(key=...): non-abstract, yields 15 results on github search
  • pop(key=...): non-abstract, yields 41 results on github search
  • pop(..., default=...): non-abstract, yields 65 results on github search
  • pop(key=..., default=...): non-abstract, yields 12 results on github search
  • Includes some real hits like os.environ, which subclasses MutableMapping.
  • setdefault(key=...): non-abstract, yields 0 results on github search
  • setdefault(..., default=...): non-abstract, yields 82 results on github search
  • setdefault(key=..., default=...): non-abstract, yields 10 results on github search
• Set
  • __le__(other=..): non-abstract, yields 2 results on github search
  • __lt__(other=...): non-abstract, yields 5 results on github search
  • __ge__(other=...): non-abstract, yields 2 results on github search
  • __gt__(other=...): non-abstract, yields 2 results on github search
  • __eq__(other=...): non-abstract, yields 139 results on github search
  • __and__(other=...): non-abstract, yields 0 results on github search
  • __rand__(other=...): non-abstract, yields 0 results on github search
  • __or__(other=...): non-abstract, yields 3 results on github search
  • __sub__(other=...): non-abstract, yields 39 results on github search
  • __rsub__(other=...): non-abstract, yields 1 results on github search
  • __xor__(other=...): non-abstract, yields 0 results on github search
  • __rxor__(other=...): non-abstract, yields 0 results on github search
  • isdisjoint(other=...): non-abstract, yields 0 results on github search
  • _from_iterable(it=...): non-abstract, yields 0 results on github search
• MutableSet:
  • add(value=...): abstract, yields 117 results on github search
  • discard(value=...): non-abstract, yields 0 results on github search
  • remove(value=...): non-abstract, yields 17 results on github search
• MappingView:
  • (mapping=...): non-abstract, yields 2.4k results on github search
• KeysView
  • __contains__(key=...): non-abstract, yields 36 results on github search
• ItemsView
  • __contains__(item=...): non-abstract, yields 5 results on github search
• ValuesView
  • __contains__(value=...): non-abstract, yields 0 results on github search
• Coroutine:
  • send(value=...): abstract, yields 297 results on github search
  • throw(typ=...): abstract, yields 0 results on github search
• Generator:
  • send(value=...): abstract, yields 297 results on github search
  • throw(typ=...): abstract, yields 0 results on github search
• AsyncGenerator:
  • asend(value=...): abstract, yields 0 results on github search
  • athrow(typ=...): abstract, yields 0 results on github search

EDIT: index/count had some false positive matches due to github search not defaulting to case-sensistive.
EDIT-2: Added non-abstract Mapping/MutableMapping methods with different signatures.

[JelleZijlstra]

This is probably fine for the dunder methods in 3.15; it technically breaks compatibility but any affected code (if any exists) is extremely questionable.

The non-dunder methods should probably also use positional-only args if we add them today, but making them positional-only now seems a bit riskier.

[serhiy-storchaka]

Abstract methods define protocol, but accepting keyword arguments was not intentional. For example, list.index() does not accept the keyword argument value. Either list support of the Sequence protocol is incorrect, or the Sequence protocol is defined incorrectly. The same is true for all other builtin and extension collection types.

If the abstract class provides non-abstract implementation of the method, then such change can break user code. But if it is an abstract method, making its arguments positional-only should be safe.