bpo-36829: Add test.support.catch_unraisable_exception()

[vstinner]

• Copy test_exceptions.test_unraisable() to
  test_sys.UnraisableHookTest().
• test_exceptions.test_unraisable() uses catch_unraisable_exception();
  simplify the test. test_sys now checks the exact output.
• Use catch_unraisable_exception() in test_coroutines,
  test_exceptions, test_generators.

https://bugs.python.org/issue36829

[vstinner]

I chose to leave test_io unchanged on purpose: I have a local branch which fix also https://bugs.python.org/issue36918 in Lib/_pyio.py and use this new catch_unraisable_exception() function. Once this PR will be merged, I will write a second PR based on it.

[vstinner]

It's a little bit strange to have to catch half of the error message from stderr, and the other half from sys.unraisablehook. My PR #13488 will allow to inject the error message in the unraisable exception to catch both at the same time.

[graingert]

this try/with/finally is somewhat tricky, how about:

try:
    with support.throw_unraisable_exceptions():
        ...
except Exception as e:
    ... # the exception is now here

eg:

@contextlib.contextmanager
def throw_unraisable_exceptions():
    unraisable = None
    old_hook = sys.unraisablehook

    def hook(exc):
        nonlocal unraisable
        unraisable = exc

    sys.unraisablehook = hook
    try:
        yield
        if unraisable is not None:
            raise unraisable
    finally:
        unraisable = None
        sys.unraisablehook = old_hook

[graingert]

or recommend usage like this:

    with support.catch_unraisable_exceptions() as cm:
        ...
        cm.unraisable  # only available here
    assert cm.unraisable is None  # now it's gone

by updating __exit__:

   def __exit__(self, *exc_info):
        self.unraisable = None  # clear unraisable here
        sys.unraisablehook = self._old_hook

[vstinner]

Good idea :-) I implemented your __exit__ idea to avoid the need for try/finally.

throw_unraisable_exceptions() might be useful, but modified tests needs to access the 'obj' attribute of the unraisable hook. Later, they might also want to get access to the 'err_msg' attribute: #13488

[graingert]

There's still a sharp edge here:

how about making cm.unraisable throw if accessed outside the context instead of being None, eg:

def __exit__(self, *exc_info):
    del self.unraisable
    sys.unraisablehook = self._old_hook

[graingert]

and if you want to access additional unraisablehook info using throw_unraisable_exceptions:

class UnraisableException(Exception):
    def __init__(self, unraisable):
        self.unraisable = unraisable
        super().__init__(self, unraisable)


@contextlib.contextmanager
def throw_unraisable_exceptions():
    unraisable = None
    old_hook = sys.unraisablehook

    def hook(unraisable_):
        nonlocal unraisable
        unraisable = unraisable_

    sys.unraisablehook = hook
    try:
        yield
        if unraisable is not None:
            raise UnraisableException(unraisable) from unraisable.exc_value
    finally:
        sys.unraisablehook = old_hook

then you can use it with:

try:
    with throw_unraisable_exceptions():
        ...
except UnraisableException as e:
    print(repr(e.__cause__))
    err_msg = e.unraisable.err_msg

[vstinner]

I wrote PR #13554 to implement your "del self.unraisable" idea.