singledispatch register should typecheck its argument

[amogorkon]

BPO	27984
Nosy	@rhettinger, @ncoghlan, @ethanfurman, @ambv, @eryksun, @zhangyangyu, @csabella
PRs	bpo-27984: explictly check type of singledispatch.dispatch() argument #6114 [3.6] bpo-27984: Limit the cls argument of singledispatch.register and #6115
Superseder	bpo-32227: singledispatch support for type annotations
Files	issue27984.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-03-14.22:23:48.241>
created_at = <Date 2016-09-06.23:18:51.783>
labels = ['3.8', 'type-bug', '3.7']
title = 'singledispatch register should typecheck its argument'
updated_at = <Date 2018-03-15.10:33:30.900>
user = 'https://bugs.python.org/amogorkon'

bugs.python.org fields:

activity = <Date 2018-03-15.10:33:30.900>
actor = 'xiang.zhang'
assignee = 'none'
closed = True
closed_date = <Date 2018-03-14.22:23:48.241>
closer = 'lukasz.langa'
components = []
creation = <Date 2016-09-06.23:18:51.783>
creator = 'amogorkon'
dependencies = []
files = ['44428']
hgrepos = []
issue_num = 27984
keywords = ['patch']
message_count = 12.0
messages = ['274661', '274666', '274669', '274704', '274784', '313770', '313800', '313818', '313842', '313845', '313865', '313869']
nosy_count = 8.0
nosy_names = ['rhettinger', 'ncoghlan', 'ethan.furman', 'lukasz.langa', 'eryksun', 'xiang.zhang', 'amogorkon', 'cheryl.sabella']
pr_nums = ['6114', '6115']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '32227'
type = 'behavior'
url = 'https://bugs.python.org/issue27984'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[amogorkon]

from functools import singledispatch
from enum import Enum

IS = Enum("IS", "a, b")

@singledispatch
def foo(x):
    print(foo.dispatch(x))
    print("foo")

@foo.register(IS.a)
def bar(x):
    print(foo.dispatch(x))
    print("bar")
    
@foo.register(int)
def baz(x):
    print("baz")
    
>>> foo(IS.a)
<function bar at 0x7fa92c319d90>
foo

I think the result is self-explaining. The foo.dispatch() correctly says function bar should be handling the call when IS.a is passed, but foo is handling the call anyway. If an int is passed, baz works as expected.

[ethanfurman]

IS = Enum("IS", "a, b")

functools is meant to work with types, but the enum member "IS.a" is not a type -- however, the enum "IS" is a type.

Use "foo.register(IS)" instead.

[eryksun]

The register() method should raise a TypeError if it's called with an object that's not a type.

Consider the following:

    from functools import singledispatch

    class C:
        pass

    obj = C()

    @singledispatch
    def foo(x):
        print(foo.dispatch(x))
        print("foo")

    @foo.register(obj)
    def bar(x):
        print(foo.dispatch(x))
        print("bar")

    >>> foo(obj)
    <function bar at 0x7f16f2adc9d8>
    foo

[ncoghlan]

Catching the erroneous registration rather than silently ignoring it sounds like the right thing to do here to me as well.

I'm actually surprised that code isn't already throwing an exception later on, as "isinstance" itself does fail with non-types:

>>> from enum import Enum
>>> 
>>> IS = Enum("IS", "a, b")
>>> isinstance(IS.a, IS)
True
>>> isinstance(IS.a, IS.a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: isinstance() arg 2 must be a type or tuple of types

[zhangyangyu]

It's also better to add the typecheck to dispatch. Otherwise sometimes it can generate not obvious exception message.

>>> from enum import IntEnum
>>> from functools import singledispatch
>>> IS = IntEnum("IS", "a, b")
>>> @singledispatch
... def foo(x):
...     pass
... 
>>> foo.dispatch(IS.a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/angwer/cpython/Lib/functools.py", line 718, in dispatch
    impl = dispatch_cache[cls]
  File "/home/angwer/cpython/Lib/weakref.py", line 365, in __getitem__
    return self.data[ref(key)]
TypeError: cannot create weak reference to 'IS' object
>>>

[csabella]

@xiang.zhang, would you be able to make a PR for your patch? Thanks!

[zhangyangyu]

Thanks for the reminding, I already forget this. I'll do it night.

[zhangyangyu]

Revising the patch, register() is fixed by feature in bpo-32227. So I will only fix dispatch() in 3.7 and 3.8, the whole patch to 3.6.

[ambv]

I'm sorry that you wasted your time on those pull requests, Xiang but:

• we cannot accept any new features for 3.6 and 3.7 anymore;
• the problem described in this issue is fixed, as you noticed yourself, by bpo-32227.

The wrapper returned by the @register decorator is calling dispatch() with the first argument's __class__. It can only ever be invalid if somebody deliberately wrote something invalid to the object's __class__. It's extremely unlikely.

We should not slow down calling of all generic functions on the basis that somebody might pass a non-type straigh to dispatch().

Cheryl, thanks for your help triaging the bug tracker! It might be a good idea to confirm with people on old issues whether moving the patch to a pull request is indeed the missing part of it getting merged. In this case it wasn't, I hope Xiang won't feel bad about me rejecting his pull requests after you pinged him for them.

[zhangyangyu]

It's all right.

[csabella]

Łukasz,

Sorry about that. Since Xiang is a core developer, I thought he would be in the position to make the decision about moving forward with the patch. I apologize for causing unnecessary work.

[zhangyangyu]

Cheryl, It's a good remind for me. I totally forget about this issue.

Since Xiang is a core developer, I thought he would be in the position to make the decision about moving forward with the patch.

Yes, I made the decision. But seems not a wise one, sorry. :-( But fortunately Łukasz noticed it. :-)

I am happy to see finally this issue is solved, instead of hanging with no reply. :-) Thanks for Łukasz and Cheryl.

I am sorry for the PRs making noise here. Again, it's quite all right for me to close them. :-)