Async generators

[EliseAv]

Scenarios like these are not currently supported:

@unsync
def my_gen(block_size):
    while block := slow_read(block_size):
        yield from block

async def my_consumer():
    async for item in my_gen(1000):
         print(f"Found item: {item}")

They will error out by not finding __aiter__. At first sight this is easy to add, and I was planning to just drop a PR and walk off, but there are a few design decisions here.

Should we just call next() on whatever thread in the thread pool as __anext__ is called? Or should we trap a thread for the lifetime of the iterator, detecting iterator abandonment using the weakref module? Should that be exposed as an option in the decorator?

[alex-sherman]

I would be interested I think to first see how best to handle "standard" async iterators, like:

@unsync
async def my_gen():
  for item in some_thing:
    await slow_thing()
    yield whatever

My main first concern is the blocking API calls will have to be made more complicated to handle this async iterator case. With that ground work I think it will be easier to consider this use case of converting conventional iterators to async ones. (Which is a super a cool idea by the way!)

[EliseAv]

Hm, I hadn't considered that. Part of the problem is that async iterators are rich objects that receive calls with arguments and whatnot. There's even more thought involved than I initially thought! :)

Convenient PEP link: https://www.python.org/dev/peps/pep-0525/