asyncio.run() error related to finalizing async generators

[samfrances]

BPO	37455
Nosy	@asvetlov, @1st1, @tirkarthi, @samfrances
Superseder	bpo-38013: AsyncGenerator breaks when not iterated fully with RuntimeError("can't send non-None value to a just-started coroutine")

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-09-12.21:35:01.509>
created_at = <Date 2019-06-30.23:53:08.254>
labels = ['3.8', 'type-bug', '3.7', 'expert-asyncio']
title = 'asyncio.run() error related to finalizing async generators'
updated_at = <Date 2019-09-12.21:35:01.509>
user = 'https://github.com/samfrances'

bugs.python.org fields:

activity = <Date 2019-09-12.21:35:01.509>
actor = 'xtreak'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-12.21:35:01.509>
closer = 'xtreak'
components = ['asyncio']
creation = <Date 2019-06-30.23:53:08.254>
creator = 'samfrances'
dependencies = []
files = []
hgrepos = []
issue_num = 37455
keywords = []
message_count = 5.0
messages = ['346943', '346944', '346946', '352215', '352222']
nosy_count = 4.0
nosy_names = ['asvetlov', 'yselivanov', 'xtreak', 'samfrances']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '38013'
type = 'behavior'
url = 'https://bugs.python.org/issue37455'
versions = ['Python 3.7', 'Python 3.8']

[samfrances]

The documentation for asyncio.run() states:

"This function runs the passed coroutine, taking care of managing the asyncio event loop and finalizing asynchronous generators."

However, the following example seems to indicate that async generators are not being cleared up correctly.

import asyncio


async def count():
    try:
        i = 0
        while True:
            yield i
            i += 1
    finally:
        print("count() cleanup")


async def double(source):
    try:
        async for n in source:
            yield n * 2
    finally:
        print("double() cleanup")


async def anext(aiter):
    return await aiter.__anext__()


async def main():
    async for i in double(count()):
        if i > 10:
            return
        print(i)

asyncio.run(main())

The result is:

$ python example.py 
0
2
4
6
8
10
unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-2' coro=<<async_generator_athrow without __name__>()> exception=RuntimeError("can't send non-None value to a just-started coroutine")>
RuntimeError: can't send non-None value to a just-started coroutine
count() cleanup

The above error is from Python 3.8.0b1+, but the exact same error occurs in Python 3.7.

I'm not sure if this is a bug or if I am misunderstanding the intended behaviour of asyncio.run(), but the behaviour was certainly surprising to me.

[samfrances]

Apologies for the stray unused function in the example. It should have read:

import asyncio


async def count():
    try:
        i = 0
        while True:
            yield i
            i += 1
    finally:
        print("count() cleanup")


async def double(source):
    try:
        async for n in source:
            yield n * 2
    finally:
        print("double() cleanup")


async def main():
    async for i in double(count()):
        if i > 10:
            return
        print(i)

asyncio.run(main())

[samfrances]

One final note: changing the main() function from the example to add a zero-sleep seems to fix it, but this seems like a rather ad-hoc solution.

async def main():
    async for i in double(count()):
        if i > 10:
            break
        print(i)
    await asyncio.sleep(0)

result:

$ python example.py 
0
2
4
6
8
10
double() cleanup
count() cleanup

[tirkarthi]

I think this would be fixed by bpo-38013. Andrew I feel these are the same issues here and can be closed.

[asvetlov]

Yes, please close as a duplicate.