Incompatibility with asgiref, pypy and coverage

[LilyAcorn]

I'm not sure which project is at fault, but asgiref 3.7.0 and 3.7.1 both fail. 3.6.0 does not.

If I put this in a test.py:

from asgiref.sync import iscoroutinefunction, markcoroutinefunction, sync_to_async

And run:

$ coverage run test.py 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    from asgiref.sync import iscoroutinefunction, markcoroutinefunction, sync_to_async
  File "/home/ian/work/asgiref/asgiref/sync.py", line 130, in <module>
    class AsyncToSync(Generic[_P, _R]):
  File "/home/ian/.pyenv/versions/pypy3.8-7.3.11/lib/pypy3.8/typing.py", line 261, in inner
    return func(*args, **kwds)
  File "/home/ian/.pyenv/versions/pypy3.8-7.3.11/lib/pypy3.8/typing.py", line 890, in __class_getitem__
    raise TypeError(
TypeError: Parameters to Generic[...] must all be type variables

These are my installed packages:

$ pip freeze
asgiref==3.7.1
cffi==1.15.1
coverage==7.2.6
greenlet==0.4.13
hpy==0.0.4.dev179+g9b5d200
readline==6.2.4.1
typing_extensions==4.6.1

Bisected to 72f45c5 locally.

[LilyAcorn]

Actually this might be a coverage bug, since this only appears when I run under coverage.

[andrewgodwin]

This bug looks very like #392 - are you sure that coverage is not running in a separate virtualenv or similar with an old typing_extensions package?

[LilyAcorn]

It does look very similar, but I'm using a fresh virtualenv with typing_extensions 4.6.1:

lily@lily:~/work/test-asgiref$ mkvirtualenv test-asgiref -a . -p pypy3.8
created virtual environment PyPy3.8.16.final.0-64 in 162ms
  creator PyPy3Posix(dest=/home/lily/.virtualenvs/test-asgiref, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/lily/.local/share/virtualenv)
    added seed packages: pip==23.1.2, setuptools==67.7.2, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
virtualenvwrapper.user_scripts creating /home/lily/.virtualenvs/test-asgiref/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/lily/.virtualenvs/test-asgiref/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/lily/.virtualenvs/test-asgiref/bin/preactivate
virtualenvwrapper.user_scripts creating /home/lily/.virtualenvs/test-asgiref/bin/postactivate
virtualenvwrapper.user_scripts creating /home/lily/.virtualenvs/test-asgiref/bin/get_env_details
Setting project for test-asgiref to /home/lily/work/test-asgiref

(test-asgiref) lily@lily:~/work/test-asgiref$ pip install coverage
Collecting coverage
  Using cached coverage-7.2.6-pp37.pp38.pp39-none-any.whl (193 kB)
Installing collected packages: coverage
Successfully installed coverage-7.2.6

(test-asgiref) lily@lily:~/work/test-asgiref$ pip install asgiref
Collecting asgiref
  Using cached asgiref-3.7.1-py3-none-any.whl (24 kB)
Collecting typing-extensions>=4 (from asgiref)
  Using cached typing_extensions-4.6.1-py3-none-any.whl (31 kB)
Installing collected packages: typing-extensions, asgiref
Successfully installed asgiref-3.7.1 typing-extensions-4.6.1

(test-asgiref) lily@lily:~/work/test-asgiref$ coverage run test.py 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    from asgiref.sync import iscoroutinefunction, markcoroutinefunction, sync_to_async
  File "/home/lily/.virtualenvs/test-asgiref/lib/pypy3.8/site-packages/asgiref/sync.py", line 130, in <module>
    class AsyncToSync(Generic[_P, _R]):
  File "/home/lily/.pyenv/versions/pypy3.8-7.3.11/lib/pypy3.8/typing.py", line 261, in inner
    return func(*args, **kwds)
  File "/home/lily/.pyenv/versions/pypy3.8-7.3.11/lib/pypy3.8/typing.py", line 890, in __class_getitem__
    raise TypeError(
TypeError: Parameters to Generic[...] must all be type variables

(test-asgiref) lily@lily:~/work/test-asgiref$ which coverage
/home/lily/.virtualenvs/test-asgiref/bin/coverage

[andrewgodwin]

Hmm, this is perplexing. It's possible it's pypy instead - are you able to try on cpython?

[LilyAcorn]

I can confirm I don't see the error on cpython 3.9.13 or 3.8.16.

[andrewgodwin]

OK, so this must be some sort of pypy typing incompatability. Have you tried pypy3.9?

I'm not sure if we have a sensible way to fix this other than saying "stick to asgiref 3.6 if you're on pypy" - the only real improvements in 3.7 were some speed optimisations and typing changes. I've just also relased 3.7.2, which changes the type signatures again, but I don't think pypy will be happy with it still, as it's still some weird ParamSpec stuff.

[jpmckinney]

I've tried pypy3.9 and it's the same error (TypeError: Parameters to Generic[...] must all be type variables).

Using asgiref 3.6 instead of 3.7 works.

[andrewgodwin]

I suspect we'll just have to say that pypy is incompatible with the typing in asgiref 3.7 for now then, which is annoying. I'll investigate and see if we can find some way to replace Generic with something that ignores errors if pypy is detected, but I suspect the only real fix here will be pypy hitting 3.10 and proper ParamSpec support.

[jpmckinney]

I suspect the only real fix here will be pypy hitting 3.10 and proper ParamSpec support.

pypy 3.10 was released June 16, 2023 https://doc.pypy.org/en/latest/release-v7.3.12.html