Improve healthcheck message when @given() wraps an async def function (which returns a coroutine object)

[MapleCCC]

The document states that besides running the test with pytest discovery, "we could also have run it explicitly ourselves". This is true for regular hypothesis-decorated tests. But it's not true for hypothesis-decorated coroutine tests.

import asyncio
import pytest
from hypothesis import given
from hypothesis.strategies import integers

def inc(x: int) -> int:
    return x + 1

@given(integers())
def test1(i: int) -> None:
    assert inc(i) == i + 1

@given(integers())
@pytest.mark.asyncio
async def test2(i: int) -> None:
    await asyncio.sleep(0.001)
    assert inc(i) == i + 1

Running these tests with pytest is fine. But if we run them explicitly,

if __name__ == "__main__":
    test1()
    test2()

test1() runs fine, but test2() yields hypothesis.errors.FailedHealthCheck: Tests run under @given should return None, but test2 returned <coroutine object test2 at 0x000001972E473F40> instead.

Should we consider this a bug? Because this directly contradicts the instructions from the document.

[Zac-HD]

Hypothesis doesn't actually support async def coroutine functions; it's just that pytest-asyncio transparently handles that for you using our hook to insert an async-to-sync wrapper.

We could use a more specific error message for coroutine objects though.

[MapleCCC]

@Zac-HD Thank you for your kind help and informative reply. I understand what happened now.

Another problem is that the type checker I used, Pyright, detects a type error in the above code I presented.

The type annotation of @pytest.mark.asyncio is MarkDecorator, which basically is an identity function in the eyes of type checker, and just left the type annotation of its decorated function intact. From the source code, the type annotation of @given() is Callable[[Callable[..., None]], Callable[..., None]], and it's not compatible with an async function as input. Hence the type error.

Do you have any advice in how to fix this type error? Anything that we can do in hypothesis or pytest-asyncio? Or maybe we should just use # type: ignore to disable type checking for this specific use case?

[Zac-HD]

You'll need a # type: ignore somewhere, but I'd probably put it into a shim that replaces the @pytest.mark.asyncio decorator:

def mark_asyncio(f: Coroutine[None, None, None]) -> Callable[..., None]:
    # The actual replacement is handled by magic inside pytest-asyncio
    # which understands Hypothesis, so we just ignore the types here.
    return pytest.mark.asyncio(f)  # type: ignore

@given(integers())
@mark_asyncio
async def test2(i: int) -> None:
    await asyncio.sleep(0.001)
    assert inc(i) == i + 1

[MapleCCC]

@Zac-HD Thank you for your kind help. I tried your suggested workaround, and it worked perfectly. That's a very creative solution.