Regression: it should be possible to close async iterators multiple times

[njsmith]

BPO	39606
Nosy	@ned-deily, @njsmith, @asvetlov, @ambv, @1st1, @miss-islington
PRs	bpo-39606: allow closing async generators that are already closed #18475 [3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) #18501 [3.7] bpo-39606: allow closing async generators that are already closed (GH-18475) #18502

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-02-13.09:44:45.676>
created_at = <Date 2020-02-11.06:35:23.034>
labels = ['3.7', '3.8', '3.9', 'release-blocker']
title = 'Regression: it should be possible to close async iterators multiple times'
updated_at = <Date 2020-02-13.09:44:45.676>
user = 'https://github.com/njsmith'

bugs.python.org fields:

activity = <Date 2020-02-13.09:44:45.676>
actor = 'njs'
assignee = 'none'
closed = True
closed_date = <Date 2020-02-13.09:44:45.676>
closer = 'njs'
components = []
creation = <Date 2020-02-11.06:35:23.034>
creator = 'njs'
dependencies = []
files = []
hgrepos = []
issue_num = 39606
keywords = ['patch', '3.7regression', '3.8regression', '3.9regression']
message_count = 4.0
messages = ['361783', '361943', '361948', '361952']
nosy_count = 6.0
nosy_names = ['ned.deily', 'njs', 'asvetlov', 'lukasz.langa', 'yselivanov', 'miss-islington']
pr_nums = ['18475', '18501', '18502']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue39606'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[njsmith]

In bpo-39386, the 'aclose' method for async generators was fixed so that the following broken code would correctly raise an error:

# -- bad code --
async def agen_fn():
    yield

async def do_bad_thing():
    agen = agen_fn()
    aclose_coro = agen.aclose()
    await aclose_coro
    # Should raise RuntimeError:
    await aclose_coro

asyncio.run(do_bad_thing())

# -----------------

However, the merged fix also broke the following correct code, which should *not* raise an error:

# -- good code --
async def agen_fn():
    yield

async def do_good_thing():
    agen = agen_fn()
    await agen.aclose()
    # Should *not* raise an error, but currently does in Python dev branches:
    await agen.aclose()

asyncio.run(do_good_thing())

# ----------------

It's not supported to iterate the same coroutine object twice. However, making two *independent* calls to aclose should return two independent coroutine objects, and it should be legal to iterate over each object once.

This can also occur even if there's only a single call to 'aclose'. For example, this is the recommended idiom for making sure that an async generator correctly closes all its resources:

# -- good code --
async def agen_fn():
    yield

async def careful_loop():
    agen = agen_fn()
    try:
        async for _ in agen:
            pass
    finally:
        await agen.aclose()

asyncio.run(careful_loop())

# -------------------

On released Python, the code above runs correctly. On the latest Python dev branches, it raises a RuntimeError.

So basically the problem is that the fix for bpo-39386 is storing the "was aclose iterated?" state on the async generator object, but in fact it needs to be stored on the aclose coroutine object.

[miss-islington]

New changeset 925dc7f by Nathaniel J. Smith in branch 'master':
bpo-39606: allow closing async generators that are already closed (GH-18475)
925dc7f

[miss-islington]

New changeset 8dbdf5f by Miss Islington (bot) in branch '3.8':
[3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18501)
8dbdf5f

[njsmith]

New changeset f464edf by Nathaniel J. Smith in branch '3.7':
bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18502)
f464edf