shielded task exception never retrieved when outer task cancelled

[natano]

BPO	42636
Nosy	@asvetlov, @1st1, @natano
Files	trigger-warning.py: Short script for showing the issue regression-test.patch: Regression test

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-12-14.10:36:21.291>
labels = ['3.7', '3.8', '3.9', '3.10', 'expert-asyncio']
title = 'shielded task exception never retrieved when outer task cancelled'
updated_at = <Date 2020-12-15.09:44:41.561>
user = 'https://github.com/natano'

bugs.python.org fields:

activity = <Date 2020-12-15.09:44:41.561>
actor = 'natano'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2020-12-14.10:36:21.291>
creator = 'natano'
dependencies = []
files = ['49679', '49680']
hgrepos = []
issue_num = 42636
keywords = ['patch']
message_count = 4.0
messages = ['382975', '382976', '382979', '383039']
nosy_count = 3.0
nosy_names = ['asvetlov', 'yselivanov', 'natano']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue42636'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[natano]

A task created with asyncio.shield() never retrieves the task exception, which results in a log message being generated. See the attached script for a minimal example. Output looks something like this:

<snip>
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<some_coroutine() done, defined at /Users/natano/python-bug/trigger-warning.py:4> exception=Exception('foo')>
Traceback (most recent call last):
  File "/Users/natano/python-bug/trigger-warning.py", line 6, in some_coroutine
    raise Exception('foo')
Exception: foo
</snip>

I believe this behaviour is not intended and is an unintended side-effect of commit b35acc5 (b35acc5). The _inner_done_callback has has code to retrieve the exception when the outer task was cancelled, but since that commit the _inner_done_callback is not called anymore when the outer task was cancelled.

(Tested with 3.8, 3.9 and 3.10. I don't have a python 3.7 version available for testing, but I think this issue also affects 3.7.)

[natano]

I'm not sure how to fix this, but here is a regression test for the issue. (failing on the current master branch)

[asvetlov]

Thanks for the report.

I didn't test but looks like outer.exception() call can suppress the message about not retrieved exception.

[natano]

How would calling outer.exception() suppress the message? This happens when the outer task is cancelled while the inner task is still running. In that case the exception is never set on outer.