@asynccontextmanager doesn't work well with async generators

[ValentinLavrinenko]

BPO	33786
Nosy	@ncoghlan, @ned-deily, @asvetlov, @ambv, @1st1, @miss-islington, @oremanj
PRs	bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() #7467 [3.7] bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) #7506 [3.6] bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() #7507 bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() #7514 [3.7] bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) #21878

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-08-14.09:53:24.162>
created_at = <Date 2018-06-06.17:24:35.200>
labels = ['interpreter-core', '3.8', 'type-bug', '3.7']
title = "@asynccontextmanager doesn't work well with async generators"
updated_at = <Date 2020-08-14.09:54:49.933>
user = 'https://bugs.python.org/ValentinLavrinenko'

bugs.python.org fields:

activity = <Date 2020-08-14.09:54:49.933>
actor = 'asvetlov'
assignee = 'none'
closed = True
closed_date = <Date 2020-08-14.09:53:24.162>
closer = 'ned.deily'
components = ['Interpreter Core']
creation = <Date 2018-06-06.17:24:35.200>
creator = 'Valentin Lavrinenko'
dependencies = []
files = []
hgrepos = []
issue_num = 33786
keywords = ['patch']
message_count = 12.0
messages = ['318853', '318854', '318882', '318967', '318998', '319005', '319012', '375189', '375219', '375388', '375389', '375390']
nosy_count = 8.0
nosy_names = ['ncoghlan', 'ned.deily', 'asvetlov', 'lukasz.langa', 'yselivanov', 'miss-islington', 'Valentin Lavrinenko', 'Joshua Oreman']
pr_nums = ['7467', '7506', '7507', '7514', '21878']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33786'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[ValentinLavrinenko]

@asynccontextmanager
async def ctx():
    yield


async def gen():
    async with ctx():
        yield 'hello'
        yield 'world'


async def main():
    async with ctx():
        async for value in gen():
            print(value)
            raise RuntimeError()

Running main() leads to RuntimeError: generator didn't stop after throw(). This happens because async gernerator's .athrow() method doesn't re-throw GeneratorExit exception; probably, this is wrong.

[ValentinLavrinenko]

Sorry, async with ctx() in main() is not needed.

[1st1]

Would be nice to fix this in 3.7.0

[ned-deily]

Would be nice to fix this in 3.7.0

Please do!

[1st1]

New changeset 52698c7 by Yury Selivanov in branch 'master':
bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467)
52698c7

[1st1]

New changeset 8de73d5 by Yury Selivanov in branch '3.6':
bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) (GH-7507)
8de73d5

[1st1]

New changeset b0bb9a8 by Yury Selivanov in branch '3.6':
bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) (bpo-7514)
b0bb9a8

[oremanj]

This doesn't appear to have been backported to 3.7, even though it's in 3.6.6 and 3.8.0a0.

[ned-deily]

Indeed, the backport to 3.7 slipped through the cracks somehow; we should fix that. Thanks for bringing this up!

[ned-deily]

New changeset cf79cbf by Miss Islington (bot) in branch '3.7':
bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) (GH-21878)
cf79cbf

[ned-deily]

I'm still not sure exactly what happened here but it looks like the backport to 3.7 (PR 7506) from the original fix in master (pre-3.8) (PR 7467) failed but the backport to 3.6 (PR 7507) succeeded. And then it was backported a second time to 3.6 (PR 7514) which also succeeded but had no effect since there were no intervening changes to those files. So it may be that PR 7514 was intended to be for 3.7 instead of 3.6. In any case, a fresh backport from master to 3.7 (PR 21878) succeeded and tests pass, so into 3.7 it goes for release in 3.7.9.

[asvetlov]

Thank you very much, Ned!