Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generator with await not considered an AsyncGenerator #10534

Closed
audoh-tickitto opened this issue May 26, 2021 · 2 comments · Fixed by #17452
Closed

Generator with await not considered an AsyncGenerator #10534

audoh-tickitto opened this issue May 26, 2021 · 2 comments · Fixed by #17452
Labels
bug mypy got something wrong topic-async async, await, asyncio

Comments

@audoh-tickitto
Copy link

audoh-tickitto commented May 26, 2021

Bug Report

  • Generators containing the await keyword are considered Generator, rather than AsyncGenerator
  • Generator syntax does not permit await in a non-async function

To Reproduce

test.py:

import asyncio
from typing import AsyncGenerator


async def get_value(v: int) -> int:
    await asyncio.sleep(1)
    return v + 1


def get_generator() -> AsyncGenerator[int, None]:
    return (await get_value(v) for v in [1, 2, 3])


async def test() -> None:
    print(type(get_generator()))
    async for x in get_generator():
        print(x)


asyncio.get_event_loop().run_until_complete(test())

python:
> py test.py

<class 'async_generator'>
2
3
4

mypy:
mypy test.py

test.py:11: error: Incompatible return value type (got "Generator[int, None, None]", expected "AsyncGenerator[int, None]")

Expected Behavior

No errors should be reported.

Actual Behavior

test.py:11: error: Incompatible return value type (got "Generator[int, None, None]", expected "AsyncGenerator[int, None]")
test.py:11: error: 'await' outside coroutine ('async def')

Your Environment

  • Mypy version used: 0.812
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files):

[mypy]
plugins = pydantic.mypy

strict_optional = True
warn_redundant_casts = True
warn_unused_ignores = True
disallow_any_generics = True
check_untyped_defs = True
disallow_untyped_defs = True

[pydantic-mypy]
init_forbid_extra = True
warn_required_dynamic_aliases = True
warn_untyped_fields = True

[mypy-bson.*]
ignore_missing_imports = True

[mypy-cachetools.*]
ignore_missing_imports = True

[mypy-asyncache.*]
ignore_missing_imports = True

[mypy-loguru]
ignore_missing_imports = True

[mypy-aiohttp_xmlrpc.*]
ignore_missing_imports = True

[mypy-asyncpg.*]
ignore_missing_imports = True

[mypy-bcrypt]
ignore_missing_imports = True

[mypy-passlib.*]
ignore_missing_imports = True

[mypy-slugify.*]
ignore_missing_imports = True

[mypy-barcode.*]
ignore_missing_imports = True

[mypy-qrcode.*]
ignore_missing_imports = True

[mypy-furl]
ignore_missing_imports = True
although 
[mypy-defusedxml.*]
ignore_missing_imports = True

[mypy-pyticketswitch.*]
ignore_missing_imports = True

[mypy-pymongo.*]
ignore_missing_imports = True

[mypy-motor.*]
ignore_missing_imports = True

[mypy-stackprinter]
ignore_missing_imports = True

[mypy-PIL.*]
ignore_missing_imports = True

[mypy-aztec_code_generator.*]
ignore_missing_imports = True

[mypy-pdf417.*]
ignore_missing_imports = True

[mypy-pylibdmtx.*]
ignore_missing_imports = True


[mypy-toolz]
ignore_missing_imports = True

[mypy-stripe]
ignore_missing_imports = True

[mypy-google.*]
ignore_missing_imports = True

[mypy-coinoxr]
ignore_missing_imports = True

[mypy-sendgrid.*]
ignore_missing_imports = True

[mypy-opencensus.*]
ignore_missing_imports = True
  • Python version used: 3.9.2, acquired via pyenv
  • Operating system and version: Ubuntu 20.04.2
@joaoe
Copy link

joaoe commented Jul 29, 2022

Seems you are missing async def here def get_generator() -> AsyncGenerator[int, None]:

@tamird
Copy link
Contributor

tamird commented Jun 19, 2024

Simple repro with mypy latest: https://mypy-play.net/?mypy=master&python=3.12&gist=71cdbfcc61179de3290480733083180c&flags=warn-unreachable

async def something(i: int) -> tuple[None, ...]:
    return (None,)

async def main() -> None:
    async_or_nah = (
        thing
        for i in range(5)
        for thing in await something(i)
    )
    
    
    async for blah in async_or_nah: # error: "Generator[None, None, None]" has no attribute "__aiter__" (not async iterable)  [attr-defined]
        print(blah)

hauntsaninja pushed a commit that referenced this issue Jun 30, 2024
Fixes #10534

This PR fixes a bug in typechecking asynchronous generators.

Mypy currently typechecks a generator/comprehension as `AsyncGenerator`
if the leftmost expression contains `await`, or if it contains an `async
for`.

However, there are other situations where we should get async generator:
If there is an `await` expression in any of the conditions or in any
sequence except for the leftmost one, the generator/comprehension should
also be typechecked as `AsyncGenerator`.

I've implemented this change in Mypy and added a test case to assert
this behavior. If I enter the test cases into a regular repl, I can
confirm that the runtime representation is generator/async_generator as
the test case expects.

According to the [language
reference](https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-comp_for):

> If a comprehension contains either async for clauses or await
expressions or other asynchronous comprehensions it is called an
asynchronous comprehension.

Confusingly, the documentation itself is actually not quite correct
either, as pointed out in
python/cpython#114104

Alongside this change, I've made a PR to update the docs to be more
precise: python/cpython#121175 has more details.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong topic-async async, await, asyncio
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants