Prevent double awaiting of async iterator

[jmg]

BPO	39386
Nosy	@njsmith, @asvetlov, @1st1, @miss-islington
PRs	bpo-39386: Prevent double awaiting of async iterator #18081 [3.8] bpo-39386: Prevent double awaiting of async iterator (GH-18081) #18086 [3.7] bpo-39386: Prevent double awaiting of async iterator (GH-18081) #18087 bpo-39606: allow closing async generators that are already closed #18475 [3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) #18501 [3.7] bpo-39606: allow closing async generators that are already closed (GH-18475) #18502
Files	asyncitertc.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-01-20.23:45:40.207>
created_at = <Date 2020-01-19.06:43:40.732>
labels = ['3.7', '3.8', '3.9', 'expert-asyncio']
title = 'Prevent double awaiting of async iterator'
updated_at = <Date 2020-02-13.09:33:42.106>
user = 'https://bugs.python.org/jmg'

bugs.python.org fields:

activity = <Date 2020-02-13.09:33:42.106>
actor = 'njs'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-20.23:45:40.207>
closer = 'asvetlov'
components = ['asyncio']
creation = <Date 2020-01-19.06:43:40.732>
creator = 'jmg'
dependencies = []
files = ['48852']
hgrepos = []
issue_num = 39386
keywords = ['patch']
message_count = 8.0
messages = ['360254', '360326', '360340', '360341', '360342', '361942', '361947', '361951']
nosy_count = 5.0
nosy_names = ['jmg', 'njs', 'asvetlov', 'yselivanov', 'miss-islington']
pr_nums = ['18081', '18086', '18087', '18475', '18501', '18502']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue39386'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[jmg]

If I create a coro from an async iterator, then wait_for it w/ a timeout, but shielded, so it won't get canceled, and then await upon it, it returns invalid data.

See the attached test case.

The reason I do the following is to make sure that an async iterator that I have written doesn't return data early, and needs to wait till later. If I didn't shield it, then the async iterator would get cancelled, and I don't want this.

I'd expect either correct results to be returned, or an exception to be raised, but in this case, and the docs for wait_for ( https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for ), I'd expect the correct results to be returned.

In the attached case, this is the results that I get:
$python3.7 asyncitertc.py
3.7.5 (default, Oct 18 2019, 23:59:39)
[Clang 7.0.2 (clang-700.1.81)]
timed out
yielding 1
results: None
getting 2: 2

I do not have python 3.8 to test with.

[asvetlov]

Thanks for the report.

Your example could be boiled down to the following:

coro = asynciter()
await coro
await coro

The second call incorrectly returns None but should raise an exception, double awaiting is a programming error.

For regular async functions it raises "RuntimeError: cannot reuse already awaited coroutine"

[asvetlov]

New changeset a96e06d by Andrew Svetlov in branch 'master':
bpo-39386: Prevent double awaiting of async iterator (GH-18081)
a96e06d

[miss-islington]

New changeset 5cadd3f by Miss Islington (bot) in branch '3.8':
bpo-39386: Prevent double awaiting of async iterator (GH-18081)
5cadd3f

[miss-islington]

New changeset b76d5e9 by Miss Islington (bot) in branch '3.7':
bpo-39386: Prevent double awaiting of async iterator (GH-18081)
b76d5e9

[miss-islington]

New changeset 925dc7f by Nathaniel J. Smith in branch 'master':
bpo-39606: allow closing async generators that are already closed (GH-18475)
925dc7f

[miss-islington]

New changeset 8dbdf5f by Miss Islington (bot) in branch '3.8':
[3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18501)
8dbdf5f

[njsmith]

New changeset f464edf by Nathaniel J. Smith in branch '3.7':
bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18502)
f464edf