Conflict between using annotations in singledispatch() and MyPy

[serhiy-storchaka]

BPO	46191
Nosy	@gvanrossum, @rhettinger, @ambv, @serhiy-storchaka, @Fidget-Spinner, @AlexWaygood

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-12-30.07:36:01.393>
created_at = <Date 2021-12-29.08:28:12.035>
labels = ['library', '3.9', '3.10', '3.11']
title = 'Conflict between using annotations in singledispatch() and MyPy'
updated_at = <Date 2021-12-30.07:36:01.388>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2021-12-30.07:36:01.388>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-12-30.07:36:01.393>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2021-12-29.08:28:12.035>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 46191
keywords = []
message_count = 7.0
messages = ['409295', '409310', '409313', '409314', '409336', '409339', '409351']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'rhettinger', 'lukasz.langa', 'serhiy.storchaka', 'kj', 'AlexWaygood']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue46191'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[serhiy-storchaka]

You can dispatch on collections.abc.Sequence.

@functools.singledispatch
def f(a: collections.abc.Sequence) -> None:
    pass

But MyPy complains about this:

error: Missing type parameters for generic type "Sequence"

MyPy requires parametrized generic (like collections.abc.Sequence[int]), but singledispatch() does not work with it.

[gvanrossum]

That mypy error is only reported with a certain mypy setting (--strict, or probably one of the specific settings that that turns on).

We won't be supporting Sequence[Any] at runtime (because we only support actual types), but we should support Sequence, since that *is* an actual type (at least collections.abc.Sequence is).

So IMO this is either a problem in mypy (though in that case probably in typeshed) or a situation for which the user should just silence mypy (it's not perfect and never claimed to be).

But I think there's nothing for us to do in Python itself, so this bug should be closed.

[serhiy-storchaka]

I do not think there is a problem in MyPy. What if use __origin__ for dispatching? Registering with parametrized generics with the same __origin__ will be error.

@sigledispatch
def f(a: int) -> None:
    pass

@f.register # ok
def _(a: list[int]) -> None:
pass

@f.register # runtime error
def _(a: list[str]) -> None:
pass

@f.register # runtime error
def _(a: list) -> None:
pass

f(1)  # ok
f([1])  # ok
f([])  # ok
f(['abc'])  # static type checking error

I think that it will have advantage of stronger static type checking.

[AlexWaygood]

I like Serhiy's idea from a type-checking perspective. It would mean we could continue to have sophisticated type inference from mypy/pyright/etc. inside singledispatch function definitions. It would also mean people could use type annotations in singledispatch functions in much the same way as in the rest of their code, instead of having to remember that *this specific stdlib function* works differently. Lastly, it avoids having to have each type-checker separately special-case singledispatch so that they do not raise an error when an unparameterised generic is used as a type annotation.

My concern, however, is that the runtime semantics are slightly unintuitive. Registering an implementation to the "type" list[str] (and having it succeed without an error being raised) might give the false implication that there will be runtime checking of whether all the elements in a list are strings.

I also think that GenericAlias objects should probably only be accepted in the form of singledispatch.register() that parses type annotations. There's no use case for allowing GenericAlias objects in the other forms of registering implementations.

[gvanrossum]

If we allow registering list[int] but give it the same meaning as registering plain list (at runtime), that would violate user expectations pretty strongly -- for the same reason why we don't allow isinstance(x, list[int]).

If you want stronger checking inside the function you should probably do something like

@foo.register
def _(_a: list) -> ...:
    a: list[int] = _a
    ...

That said I don't care enough about singledispatch to argue strongly.

[AlexWaygood]

Yeah, I think I agree with Guido. Mypy only has partial support for singledispatch anyway, and only achieves that through a plugin, so special-casing by type-checkers is inevitable.

[serhiy-storchaka]

Okay. As a workaround we can explicitly specify the dispatching type:

@f.register(list)
def _(a: list[int]) -> None:
    pass