Exception handling should match subclasses, not subtypes

[mbmccoy]

BPO	33271
Nosy	@serhiy-storchaka, @mbmccoy
PRs	bpo-12029: Exception handling should match subclasses #6461
Superseder	bpo-12029: [doc] clarify that except does not match virtual subclasses of the specified exception type

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-04-13.20:12:26.849>
created_at = <Date 2018-04-13.07:39:34.468>
labels = ['interpreter-core', '3.7', '3.8']
title = 'Exception handling should match subclasses, not subtypes'
updated_at = <Date 2018-04-13.20:12:26.848>
user = 'https://github.com/mbmccoy'

bugs.python.org fields:

activity = <Date 2018-04-13.20:12:26.848>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2018-04-13.20:12:26.849>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2018-04-13.07:39:34.468>
creator = 'Michael McCoy'
dependencies = []
files = []
hgrepos = []
issue_num = 33271
keywords = ['patch']
message_count = 5.0
messages = ['315241', '315252', '315255', '315258', '315262']
nosy_count = 2.0
nosy_names = ['serhiy.storchaka', 'Michael McCoy']
pr_nums = ['6461']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '12029'
type = None
url = 'https://bugs.python.org/issue33271'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[mbmccoy]

Exception handling matches subtypes, not subclasses

# Example

    from abc import ABC
    
    class MyException(Exception, ABC):
        pass
    
    class OtherException(Exception):
        pass
    
    MyException.register(OtherException)
    
    try:
        raise OtherException
    except MyException:
        print("Correct: Caught MyException")
    except Exception:
        print("Wrong: Caught something else")
        
    # "Wrong: Caught something else"

# Background and evidence of bug-ness

bpo-2534 [1] (10 years ago!) introduced the behavior, but only in the Python 3 patch [2]. During code review, the correct function call was used [3], but the function's name got switched in the final python3 patch without any comment.

The current Python 2 code uses PyObject_IsSubclass, and produces the correct behavior in the example above (using __metaclass__ = ABCMeta, of course). This leads me to strongly suspect that this is a bug, not a feature. The note below regarding unittest for further evidence that this code has eight legs.

Given the ancient nature of this bug, it affects all versions of python3.

[1] https://bugs.python.org/issue2534
[2] https://bugs.python.org/file11257/isinstance3k-2.patch
[3] https://codereview.appspot.com/483/diff/1/21#newcode114
[4] https://github.com/python/cpython/blob/2.7/Python/errors.c#L119

# Solution

Coming very soon in a PR on Github, but in short, we do the following:

1. Switch PyType_IsSubtype to PyObject_IsSubclass.
2. Revert the changes made to remove “dead code” in https://bugs.python.org/issue31091. The code was dead because the wrong function was used—the PR left only the bug.
3. Add tests. Note that unittest’s self.assertRaises function uses issubclass and does not alert to this bug. (Different symptom, same cause.)

# Note

This bug has nothing to do with the abc package, beyond being a simple way to generate the error.

-Mike

Gitub: mbmccoy

[mbmccoy]

Adding Serhiy because this relates to bpo-31091, and specifically claims that there _was_ a bug in the old code (msg299585).

[serhiy-storchaka]

Isn't this a duplicate of bpo-12029?

[mbmccoy]

Serhiy, it sure is. I'll note that the issue is open. Moreover, reading through the history, Guido says it's a bug (msg160418), and there was agreement that it should be fixed. It's unclear why it was dropped.

Can you help me get this in? I'm not sure the correct protocol, since this is my first contrib.

Note that if this is not a bug, then there are probably bugs anywhere issubclass(...) is used to emulate exception handling. For example, unittest's self.assertRaises is inconsistent with the current behavior:

    from abc import ABC
    import unittest
    
    class TestExceptionABC(unittest.TestCase):
    
        def test_exception(self):
            class A(Exception, ABC):
                pass
            class B(Exception):
                pass
            A.register(B)
            with self.assertRaises(A, msg="Wrong exception raised"):
                raise B

Test passes in python3, but not in python2.

[serhiy-storchaka]

Let to keep the discussion in a single place -- in bpo-12029.