return type cannot be awaitable | int?

[pbelskiy]

Hello! I maintain Python package which supports both sync and async syntax with same code base, depending on selected adapter (requests or aiohttp) -- https://github.com/pbelskiy/ujenkins

I've faced with problem with Pyright, it ignores return type, but mypy works here ok. Why?

Here Pyright says:

"int" is not awaitablePylance[reportGeneralTypeIssues]

But it in fact Awaitable | int

import asyncio
from typing import Awaitable

ASYNC_MODE = True


async def echo_async(arg: int) -> int:
    return arg


def echo_sync(arg: int) -> int:
    return arg


def echo(arg: int) -> Awaitable | int:  # <--- PROBLEM HERE
    if ASYNC_MODE:
        return echo_async(arg)
    else:
        return echo_sync(arg)


async def example():
    print(await echo(777))


asyncio.run(example())

[erictraut]

Pyright's behavior here is correct. Your function echo declares that its return type is Awaitable | int. That means any use of the returned value must work for both Awaitable and int values, which means that it's not possible to await the value.

Not surprisingly, mypy generates the same error in this case. (Actually, mypy doesn't generate an error by default here because mypy skips type checking for any function that is unannotated. If you add a -> None return type annotation to the function example, then you'll see that mypy generates the same error as pyright.)

Most libraries that support both sync and async operations offer distinct functions for both cases. It looks like you're trying to handle both cases with the same function (echo). That won't work well for static type checking.

If you change your example as follows, it works fine:

async def example() -> None:
    print(await echo_async(777))

[pbelskiy]

You are right, agree, thanks 🙏🙏

[hmc-cs-mdrissi]

I took a light look at ujenkins library and if your approach is having separate adapter classes then generics + overloads could do this. Simplified idea,

AsyncMode = TypeVar('AsyncT', Literal[True, False])
class MainJob(Generic[AsyncMode]):
  @overload
  def __init__(self: MainJob[Literal[True]], adapter: AsyncAdapter):
    ...

  @overload
  def __init__(self: MainJob[Literal[False]], adapter: SyncAdapter):
    ...

  @overload
  def echo(self: MainJob[Literal[True]], message: str) -> Awaitable:
    ...

  @overload
  def echo(self: MainJob[False], message: str) -> int:
    ...

Key trick is adapter determines where MainJob type variable for asyncness is true/false and overloads can be used from there to select Awaitable/int as needed. If you want same function to handle both async/sync you'll need overloads and either make class generic or add an argument to function to pick the mode.

An alternate variation is __new__ and returning a different type (AsyncJob vs SyncJob). Something like,

class MainJob:
  @overload
  def __new__(cls, adapter: AsyncAdapter) -> AsyncJob:
    ...

  @overload
  def __new__(cls, adapter: SyncAdapter) -> SyncJob:
    ...

class AsyncJob(MainJob):
  ...

class SyncJob(MainJob):
  ...

python is rather flexible although whether complexity is worth it is your decision.

[pbelskiy]

Thank you so much!🙏