Improve various signatures that shouldn't be async def, but currently are

[AlexWaygood]

As discussed in #7475, the following methods are all "async def" functions at runtime, but should not be "async def" functions in the stub:

• AsyncIterator.__anext__
• AyncGenerator.__anext__
• AsyncGenerator.aclose
• AsyncGenerator.asend
• AsyncGenerator.athrow

typing.AsyncIterator and typing.AsyncGenerator represent abstract interfaces rather than concrete classes, and PEP 525, which introduced the asynchronous iteration protocol, explicitly states that it is fine for these methods to return any awaitable. The current annotations, which state that these methods have to return coroutines, are therefore too restrictive, and cause false positives.

In addition to these changes:

• AsyncGenerator.__anext__ is not abstract at runtime, and I don't see any reason why it should be in the stub either.
• AsyncGenerator.__aiter__ is also not abstract at runtime, and not even defined on AsyncGenerator at runtime; it is simply inherited from AsyncIterator. I think we should do the same in the stub.
• AsyncGenerator.aclose is not abstract at runtime, and I don't see any reason why it should be in the stub.
• _typeshed.SupportsAnext.__anext__ and contextlib._SupportsAclose.aclose were both erroneously made async def functions in Fix several methods that should be async def, but aren't #7107; this PR also reverts the changes that PR made to those classes.

This PR reverts the changes #7105 made to typing.pyi, but not the changes #7105 made to types.pyi. I think that makes sense, as the classes in types are concrete classes, whereas the classes in typing are abstract interfaces.

This PR adds several entries to the allowlist, but we can take them off again if python/mypy#12343 is merged.

[AlexWaygood]

The new boostedblob error is related to this line here, and is because asyncio.create_task is annotated as requiring a coroutine following #6779, and this PR would change the definition of AsyncIterator.__anext__ such that it wouldn't necessarily be guaranteed to return a coroutine anymore.

[AlexWaygood]

Cc. @sobolevn, since this partially reverts your PR 🙂

[graingert]

Would SupportsAnext need fixing? https://github.com/python/typeshed/blob/master/stdlib/_typeshed/__init__.pyi#L36

currently it requires a coroutine - but any Awaitable is supported (it gets wrapped with anext_awaitable with a default)

[AlexWaygood]

Would SupportsAnext need fixing? https://github.com/python/typeshed/blob/master/stdlib/_typeshed/__init__.pyi#L36

Good spot. contextlib._SupportsAclose as well.

[graingert]

anext is actually slightly odd as it returns whatever __anext__ returns - eg an Awaitable or even not an awaitable!

Python 3.10.2 (main, Jan 15 2022, 18:02:07) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo:
...     def __anext__(self):
...         return "hello"
... 
>>> anext(Foo())
'hello'
>>> 

[graingert]

so the type annotation for builtins.anext should look more like this:

class SupportsAnyAnext(Protocol[_T_co]):
    def __anext__(self) -> _T_co: ...

class SupportsAwaitableAnext(Protcol[_T_co]):
    def __anext__(self) -> Awaitable[_T_co]: ...

@overload
def anext(__i: SupportsAnyAnext[_T]) -> _T: ...
@overload
async def anext(__i: SupportsAwaitableAnext[_T], default: _VT) -> _T | _VT: ...

so that anext doesn't incorrectly promote an AsyncIterable.__next__ Awaitable into a Coroutine

[AlexWaygood]

anext is actually slightly odd as it returns whatever __anext__ returns - eg an Awaitable or even not an awaitable!

Sure, but I'd imagine that if __anext__ returns something that isn't awaitable, it's almost certainly indicative of a programming error. So surely we'd be doing the developer a favour by raising an error if they try to call anext on an object that has an __anext__ method that doesn't return an awaitable?

[graingert]

my point was more about preserving the return type of the .__anext__ call:

_Awaitable_T_co = TypeVar("_Awaitable_T_co", bound=Awaitable, covariant=True)  # bound isn't strictly true - anext supports any return value

class SupportsAnext(Protocol[_Awaitable_T_co]):
    def __anext__(self) -> _Awaitable_T_co: ...

class SupportsAwaitableAnext(Protocol[_T_co]):
    def __anext__(self) -> Awaitable[_T_co]: ...

_Awaitable_T = TypeVar("_Awaitable_T", bound=Awaitable)

@overload
def anext(__i: SupportsAnext[_Awaitable_T]) -> _Awaitable_T: ...
@overload
async def anext(__i: SupportsAwaitableAnext[_T], default: _VT) -> _T | _VT: ...

[AlexWaygood]

my point was more about preserving the return type of the .__anext__ call:

It's frying my brain somewhat, but I think I understand. Thanks for the explanation!

[AlexWaygood]

Better now @graingert?

[AlexWaygood]

Thanks for the review @graingert! (I'll leave this open for a while to give other maintainers a chance to review.)

[AlexWaygood]

(Planning to merge this in a day or so, unless any other maintainers want to review, since it's sort of blocking python/mypy#12321)

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

tornado (https://github.com/tornadoweb/tornado)
- tornado/gen.py:429: error: Incompatible return value type (got "WaitIterator", expected "AsyncIterator[Any]")
- tornado/gen.py:429: note: Following member(s) of "WaitIterator" have conflicts:
- tornado/gen.py:429: note:     Expected:
- tornado/gen.py:429: note:         def __anext__(self) -> Coroutine[Any, Any, Any]
- tornado/gen.py:429: note:     Got:
- tornado/gen.py:429: note:         def __anext__(self) -> Future[Any]

boostedblob (https://github.com/hauntsaninja/boostedblob)
+ boostedblob/boost.py:547: error: Need type annotation for "task"
+ boostedblob/boost.py:547: error: Argument 1 to "create_task" has incompatible type "Awaitable[T]"; expected "Union[Generator[Any, None, <nothing>], Coroutine[Any, Any, <nothing>]]"