anext_awaitable is not a collections.abc.Generator

[lordmauve]

BPO	44963
Nosy	@jab, @asvetlov, @ambv, @1st1, @graingert, @lordmauve, @pablogsal, @miss-islington
PRs	bpo-44963: Implement send() and throw() methods for anext_awaitable objects #27955 [3.10] bpo-44963: Implement send() and throw() methods for anext_awaitable objects (GH-27955) #28198

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-07.11:07:01.418>
created_at = <Date 2021-08-20.17:24:00.924>
labels = ['type-bug', 'library', 'release-blocker', '3.10', '3.11']
title = 'anext_awaitable is not a collections.abc.Generator'
updated_at = <Date 2021-09-07.11:07:01.418>
user = 'https://github.com/lordmauve'

bugs.python.org fields:

activity = <Date 2021-09-07.11:07:01.418>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-07.11:07:01.418>
closer = 'pablogsal'
components = ['Library (Lib)']
creation = <Date 2021-08-20.17:24:00.924>
creator = 'lordmauve'
dependencies = []
files = []
hgrepos = []
issue_num = 44963
keywords = ['patch']
message_count = 5.0
messages = ['399982', '399993', '400039', '401223', '401233']
nosy_count = 8.0
nosy_names = ['jab', 'asvetlov', 'lukasz.langa', 'yselivanov', 'graingert', 'lordmauve', 'pablogsal', 'miss-islington']
pr_nums = ['27955', '28198']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44963'
versions = ['Python 3.10', 'Python 3.11']

[lordmauve]

The anext_awaitable object returned by anext(..., default) does not support .send()/.throw(). It only supports __next__().

So we can pass messages from the suspending coroutine to the event loop but not from the event loop to the suspending coroutine.

trio and curio rely on both directions working. (I don't know about asyncio.)

For example, this trio code fails:

    import trio

    async def produce():
       for v in range(3):
           await trio.sleep(1)
           yield v

    async def consume():
       p = produce()
       while True:
            print(await anext(p, 'finished'))

    trio.run(consume)

raising AttributeError: 'anext_awaitable' object has no attribute 'send'.

I realise that any awaitable that wants to await another awaitable must return not an iterator from __await__() but something that implements the full PEP-342 generator protocol. Should PEP-492 section on __await__()[1] say something about that?

[1] https://www.python.org/dev/peps/pep-0492/#await-expression

[graingert]

it also fails with asyncio.create_task

import asyncio

async def agen():
    yield

async def main():
    p = agen()
    await asyncio.create_task(anext(p, 'finished'))

asyncio.run(main())

Traceback (most recent call last):
  File "/home/graingert/projects/datapraxis/analysis/foo.py", line 10, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/home/graingert/projects/asyncio-demo/foo.py", line 8, in main
    await asyncio.create_task(anext(p, 'finished'))
  File "/usr/lib/python3.10/asyncio/tasks.py", line 337, in create_task
    task = loop.create_task(coro)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 433, in create_task
    task = tasks.Task(coro, loop=self, name=name)
TypeError: a coroutine was expected, got <anext_awaitable object at 0x7f67f5587550>

there's also a problem with cancelling and aclosing an async gen this way:

import asyncio
import contextlib

async def agen():
    await asyncio.sleep(1)
    yield

async def main():
    async with contextlib.aclosing(agen()) as p:
        asyncio.current_task().cancel()
        try:
            await anext(p, 'finished')
        except asyncio.CancelledError:
            pass

asyncio.run(main())

an error occurred during closing of asynchronous generator <async_generator object agen at 0x7f2061a61dc0>
asyncgen: <async_generator object agen at 0x7f2061a61dc0>
Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/home/graingert/projects/asyncio-demo/foo.py", line 9, in main
    async with contextlib.aclosing(agen()) as p:
  File "/usr/lib/python3.10/contextlib.py", line 366, in __aexit__
    await self.thing.aclose()
RuntimeError: aclose(): asynchronous generator is already running

During handling of the above exception, another exception occurred:

RuntimeError: aclose(): asynchronous generator is already running
Traceback (most recent call last):
  File "/home/graingert/projects/datapraxis/analysis/foo.py", line 16, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/home/graingert/projects/asyncio-demo/foo.py", line 9, in main
    async with contextlib.aclosing(agen()) as p:
  File "/usr/lib/python3.10/contextlib.py", line 366, in __aexit__
    await self.thing.aclose()
RuntimeError: aclose(): asynchronous generator is already running

[pablogsal]

I am marking this as a release blocker, given that this is probably going to trip a lot of users.

[pablogsal]

New changeset 533e725 by Pablo Galindo Salgado in branch 'main':
bpo-44963: Implement send() and throw() methods for anext_awaitable objects (GH-27955)
533e725

[miss-islington]

New changeset adc80a5 by Miss Islington (bot) in branch '3.10':
bpo-44963: Implement send() and throw() methods for anext_awaitable objects (GH-27955)
adc80a5