Error building from sources.

[luckydenis]

🐞 Describe the bug

When building according to the instructions, an error occurs.

💡 To Reproduce

Correct code build.

📋 Logs/tracebacks

...
...
...
Requirement already satisfied, skipping upgrade: PyJWT[crypto] in ./venv/lib/python3.8/site-packages (from gidgethub->cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (1.7.1)
Installing collected packages: aiohttp
  Attempting uninstall: aiohttp
    Found existing installation: aiohttp 3.6.2
    Uninstalling aiohttp-3.6.2:
      Successfully uninstalled aiohttp-3.6.2
  Running setup.py develop for aiohttp
    ERROR: Command errored out with exit status 1:
     command: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/denis/Project/aiohttp/
    Complete output (29 lines):
    **********************
    * Accellerated build *
    **********************
    running develop
    running egg_info
    writing aiohttp.egg-info/PKG-INFO
    writing dependency_links to aiohttp.egg-info/dependency_links.txt
    writing requirements to aiohttp.egg-info/requires.txt
    writing top-level names to aiohttp.egg-info/top_level.txt
    reading manifest file 'aiohttp.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'aiohttp' anywhere in distribution
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*.so' found anywhere in distribution
    warning: no previously-included files matching '*.lib' found anywhere in distribution
    warning: no previously-included files matching '*.dll' found anywhere in distribution
    warning: no previously-included files matching '*.a' found anywhere in distribution
    warning: no previously-included files matching '*.obj' found anywhere in distribution
    warning: no previously-included files found matching 'aiohttp/*.html'
    no previously-included directories found matching 'docs/_build'
    writing manifest file 'aiohttp.egg-info/SOURCES.txt'
    running build_ext
    building 'aiohttp._websocket' extension
    x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/denis/Project/aiohttp/venv/include -I/usr/include/python3.8 -c aiohttp/_websocket.c -o build/temp.linux-x86_64-3.8/aiohttp/_websocket.o
    x86_64-linux-gnu-gcc: error: aiohttp/_websocket.c: Нет такого файла или каталога
    x86_64-linux-gnu-gcc: fatal error: no input files
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
  Rolling back uninstall of aiohttp
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp-3.6.2.dist-info/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp-3.6.2.dist-info
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp
ERROR: Command errored out with exit status 1: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

📋 Your version of the Python

$ python --version
Python 3.8.2

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.6.2
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: fafhrd91@gmail.com
License: Apache 2
Location: /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages
Requires: attrs, async-timeout, chardet, multidict, yarl
Required-by: pytest-aiohttp

$ python -m pip show multidict
Name: multidict
Version: 4.7.6
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp

$ python -m pip show yarl
Name: yarl
Version: 1.4.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp

📋 Additional context

OS: Ubuntu 20.04 LTS

[webknjaz]

I guess you missed something and didn't cythonize the sources. FWIW there's no "build" instructions on that page so I don't even know what exactly you ran.

[luckydenis]

I guess you missed something and didn't cythonize the sources. FWIW there's no "build" instructions on that page so I don't even know what exactly you ran.

pip install -r requirements/dev.txt

Full trace:

pip install -r requirements/dev.txt
Ignoring uvloop: markers 'platform_system != "Windows" and implementation_name == "cpython" and python_version < "3.7"' don't match your environment
Ignoring idna-ssl: markers 'python_version < "3.7"' don't match your environment
Obtaining file:///home/denis/Project/aiohttp (from -r requirements/ci.txt (line 8))
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: setuptools-git==1.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 1)) (1.2)
Requirement already satisfied: mypy==0.770 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 2)) (0.770)
Requirement already satisfied: mypy-extensions==0.4.3 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 3)) (0.4.3)
Requirement already satisfied: freezegun==0.3.15 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 4)) (0.3.15)
Requirement already satisfied: flake8==3.7.9 in ./venv/lib/python3.8/site-packages (from -r requirements/flake.txt (line 1)) (3.7.9)
Requirement already satisfied: isort==4.3.21 in ./venv/lib/python3.8/site-packages (from -r requirements/flake.txt (line 2)) (4.3.21)
Requirement already satisfied: attrs==19.3.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 2)) (19.3.0)
Requirement already satisfied: async-timeout==4.0.0a2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 3)) (4.0.0a2)
Requirement already satisfied: Brotli==1.0.7 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 4)) (1.0.7)
Requirement already satisfied: cchardet==2.1.6 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 5)) (2.1.6)
Requirement already satisfied: chardet==3.0.4 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 6)) (3.0.4)
Requirement already satisfied: coverage==5.1 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 7)) (5.1)
Requirement already satisfied: gunicorn==20.0.4 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 8)) (20.0.4)
Requirement already satisfied: multidict==4.7.6 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 9)) (4.7.6)
Requirement already satisfied: pytest==5.4.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 10)) (5.4.2)
Requirement already satisfied: pytest-cov==2.8.1 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 11)) (2.8.1)
Requirement already satisfied: pytest-mock==3.1.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 12)) (3.1.0)
Requirement already satisfied: typing_extensions==3.7.4.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 13)) (3.7.4.2)
Requirement already satisfied: yarl==1.4.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 14)) (1.4.2)
Requirement already satisfied: aiodns==2.0.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 19)) (2.0.0)
Requirement already satisfied: cryptography==2.9.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 20)) (2.9.2)
Requirement already satisfied: trustme==0.6.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 21)) (0.6.0)
Requirement already satisfied: codecov==2.1.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 22)) (2.1.0)
Requirement already satisfied: sphinx==2.4.4 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 1)) (2.4.4)
Requirement already satisfied: sphinxcontrib-asyncio==0.2.0 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 2)) (0.2.0)
Requirement already satisfied: pygments==2.6.1 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 3)) (2.6.1)
Requirement already satisfied: aiohttp-theme==0.1.6 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 4)) (0.1.6)
Requirement already satisfied: sphinxcontrib-blockdiag==2.0.0 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 5)) (2.0.0)
Requirement already satisfied: towncrier==19.2.0 in ./venv/lib/python3.8/site-packages (from -r requirements/towncrier.txt (line 1)) (19.2.0)
Requirement already satisfied: cherry_picker==1.3.2 in ./venv/lib/python3.8/site-packages (from -r requirements/dev.txt (line 3)) (1.3.2)
Requirement already satisfied: typed-ast<1.5.0,>=1.4.0 in ./venv/lib/python3.8/site-packages (from mypy==0.770->-r requirements/ci.txt (line 2)) (1.4.1)
Requirement already satisfied: python-dateutil!=2.0,>=1.0 in ./venv/lib/python3.8/site-packages (from freezegun==0.3.15->-r requirements/ci.txt (line 4)) (2.8.1)
Requirement already satisfied: six in ./venv/lib/python3.8/site-packages (from freezegun==0.3.15->-r requirements/ci.txt (line 4)) (1.15.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (0.6.1)
Requirement already satisfied: pycodestyle<2.6.0,>=2.5.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (2.5.0)
Requirement already satisfied: pyflakes<2.2.0,>=2.1.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (2.1.1)
Requirement already satisfied: entrypoints<0.4.0,>=0.3.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (0.3)
Requirement already satisfied: setuptools>=3.0 in ./venv/lib/python3.8/site-packages (from gunicorn==20.0.4->-r requirements/ci-wheel.txt (line 8)) (47.1.1)
Requirement already satisfied: wcwidth in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (0.2.4)
Requirement already satisfied: packaging in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (20.4)
Requirement already satisfied: py>=1.5.0 in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (1.8.1)
Requirement already satisfied: pluggy<1.0,>=0.12 in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (8.3.0)
Requirement already satisfied: idna>=2.0 in ./venv/lib/python3.8/site-packages (from yarl==1.4.2->-r requirements/ci-wheel.txt (line 14)) (2.9)
Requirement already satisfied: pycares>=3.0.0 in ./venv/lib/python3.8/site-packages (from aiodns==2.0.0->-r requirements/ci-wheel.txt (line 19)) (3.1.1)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in ./venv/lib/python3.8/site-packages (from cryptography==2.9.2->-r requirements/ci-wheel.txt (line 20)) (1.14.0)
Requirement already satisfied: requests>=2.7.9 in ./venv/lib/python3.8/site-packages (from codecov==2.1.0->-r requirements/ci-wheel.txt (line 22)) (2.23.0)
Requirement already satisfied: alabaster<0.8,>=0.7 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (0.7.12)
Requirement already satisfied: babel!=2.0,>=1.3 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2.8.0)
Requirement already satisfied: sphinxcontrib-htmlhelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.3)
Requirement already satisfied: sphinxcontrib-qthelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.3)
Requirement already satisfied: imagesize in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.2.0)
Requirement already satisfied: sphinxcontrib-jsmath in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.1)
Requirement already satisfied: snowballstemmer>=1.1 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2.0.0)
Requirement already satisfied: sphinxcontrib-serializinghtml in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.1.4)
Requirement already satisfied: Jinja2>=2.3 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2.11.2)
Requirement already satisfied: docutils>=0.12 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (0.16)
Requirement already satisfied: sphinxcontrib-devhelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.2)
Requirement already satisfied: sphinxcontrib-applehelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.2)
Requirement already satisfied: blockdiag>=1.5.0 in ./venv/lib/python3.8/site-packages (from sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (2.0.1)
Requirement already satisfied: incremental in ./venv/lib/python3.8/site-packages (from towncrier==19.2.0->-r requirements/towncrier.txt (line 1)) (17.5.0)
Requirement already satisfied: toml in ./venv/lib/python3.8/site-packages (from towncrier==19.2.0->-r requirements/towncrier.txt (line 1)) (0.10.1)
Requirement already satisfied: Click in ./venv/lib/python3.8/site-packages (from towncrier==19.2.0->-r requirements/towncrier.txt (line 1)) (7.1.2)
Requirement already satisfied: gidgethub in ./venv/lib/python3.8/site-packages (from cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (4.1.1)
Requirement already satisfied: pyparsing>=2.0.2 in ./venv/lib/python3.8/site-packages (from packaging->pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (2.4.7)
Requirement already satisfied: pycparser in ./venv/lib/python3.8/site-packages (from cffi!=1.11.3,>=1.8->cryptography==2.9.2->-r requirements/ci-wheel.txt (line 20)) (2.20)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./venv/lib/python3.8/site-packages (from requests>=2.7.9->codecov==2.1.0->-r requirements/ci-wheel.txt (line 22)) (1.25.9)
Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.8/site-packages (from requests>=2.7.9->codecov==2.1.0->-r requirements/ci-wheel.txt (line 22)) (2020.4.5.2)
Requirement already satisfied: pytz>=2015.7 in ./venv/lib/python3.8/site-packages (from babel!=2.0,>=1.3->sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2020.1)
Requirement already satisfied: MarkupSafe>=0.23 in ./venv/lib/python3.8/site-packages (from Jinja2>=2.3->sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.1.1)
Requirement already satisfied: funcparserlib in ./venv/lib/python3.8/site-packages (from blockdiag>=1.5.0->sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (0.3.6)
Requirement already satisfied: Pillow>3.0 in ./venv/lib/python3.8/site-packages (from blockdiag>=1.5.0->sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (7.1.2)
Requirement already satisfied: webcolors in ./venv/lib/python3.8/site-packages (from blockdiag>=1.5.0->sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (1.11.1)
Requirement already satisfied: PyJWT[crypto] in ./venv/lib/python3.8/site-packages (from gidgethub->cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (1.7.1)
Requirement already satisfied: uritemplate>=3.0.1 in ./venv/lib/python3.8/site-packages (from gidgethub->cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (3.0.1)
Installing collected packages: aiohttp
  Attempting uninstall: aiohttp
    Found existing installation: aiohttp 3.6.2
    Uninstalling aiohttp-3.6.2:
      Successfully uninstalled aiohttp-3.6.2
  Running setup.py develop for aiohttp
    ERROR: Command errored out with exit status 1:
     command: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/denis/Project/aiohttp/
    Complete output (29 lines):
    **********************
    * Accellerated build *
    **********************
    running develop
    running egg_info
    writing aiohttp.egg-info/PKG-INFO
    writing dependency_links to aiohttp.egg-info/dependency_links.txt
    writing requirements to aiohttp.egg-info/requires.txt
    writing top-level names to aiohttp.egg-info/top_level.txt
    reading manifest file 'aiohttp.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'aiohttp' anywhere in distribution
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*.so' found anywhere in distribution
    warning: no previously-included files matching '*.lib' found anywhere in distribution
    warning: no previously-included files matching '*.dll' found anywhere in distribution
    warning: no previously-included files matching '*.a' found anywhere in distribution
    warning: no previously-included files matching '*.obj' found anywhere in distribution
    warning: no previously-included files found matching 'aiohttp/*.html'
    no previously-included directories found matching 'docs/_build'
    writing manifest file 'aiohttp.egg-info/SOURCES.txt'
    running build_ext
    building 'aiohttp._websocket' extension
    x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/denis/Project/aiohttp/venv/include -I/usr/include/python3.8 -c aiohttp/_websocket.c -o build/temp.linux-x86_64-3.8/aiohttp/_websocket.o
    x86_64-linux-gnu-gcc: error: aiohttp/_websocket.c: Нет такого файла или каталога
    x86_64-linux-gnu-gcc: fatal error: no input files
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
  Rolling back uninstall of aiohttp
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp-3.6.2.dist-info/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp-3.6.2.dist-info
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp
ERROR: Command errored out with exit status 1: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

[guusbertens]

I get the same. I'm on Debian Testing (bullseye). In particular these steps give what LuckyDenis also sees:

git clone $REPO $DIR --recurse-submodules
cd $DIR
python3 -m venv venv
. venv/bin/activate
pip install -r requirements/dev.txt

These are the suggested steps from the contributing section. However I need to add the following steps before the last step:

pip install wheel
pip install cython
make cythonize

With this everything works as expected.

[guusbertens]

Indeed, setup.py states that the makefile runs cythonize. The setuptools documentation confirms that setuptools doesn't run cythonize; it only compiles .c files, and hence the cythonized code should be included in the git repo. The cython documentation suggests using the cythonize function from within setup.py, so that pip can cythonize all that is needed.

[luckydenis]

It seems that this point is not obvious and it would be good to reflect this in the documentation or add the necessary commands to the installation scripts.
@webknjaz, @guusbertens

[webknjaz]

We moved cythonize out of setup.py in order to have Cython an optional build dependency.

[webknjaz]

I don't see you running make test. What exactly did you do?

[guusbertens]

@webknjaz I just ran this:

git clone $REPO $DIR --recurse-submodules
cd $DIR
python3 -m venv venv
. venv/bin/activate
pip install wheel
pip install cython
make cythonize
pip install -r requirements/dev.txt
make test

This works like a charm.

Do you agree that, given what is said in the setuptools docs and the Cython docs, the current breakage is expected, and that this issue is indeed a bug, and isn't an invalid issue?

You mention that you keeping Cython optional is important. As far as setup.py is concerned, Cython could be in there while being optional, but optional dependencies aren't supported by pyproject.toml, and this causes the problem. Am I right about this? If this is the case, then I think there are two options:

• include cythonize'd C-files in the git repo; or
• update the documentation to add the cythonize step; or
• maybe using two different pyproject.toml files, but I think this overcomplicates matters a lot...

Am I missing something?

[webknjaz]

Do you agree that, given what is said in the setuptools docs and the Cython docs, the current breakage is expected, and that this issue is indeed a bug, and isn't an invalid issue?

This is controlled by AIOHTTP_NO_EXTENSIONS=1 env var. If it's set, setup.py won't attempt to build the C-extensions. sdists on PyPI are shipped with pre-cythonized sources (C-files) and during development you really have to be explicit about what you want and also the default is to build the extensions (so that you accidentally don't test pure-python while thinking that you also test Cython).

You mention that you keeping Cython optional is important. As far as setup.py is concerned, Cython could be in there while being optional, but optional dependencies aren't supported by pyproject.toml, and this causes the problem. Am I right about this? If this is the case, then I think there are two options:

• include cythonize'd C-files in the git repo; or
• update the documentation to add the cythonize step; or
• maybe using two different pyproject.toml files, but I think this overcomplicates matters a lot...

Am I missing something?

Before pyproject.toml was a thing and before pip started doing mandatory build isolation we could ask users to pip install Cython into the same env and that would do the trick. Now we cannot do this because the build env is always different. And if we put Cython into setup_requires, it'd be mandatory. We cannot do this because even just installing Cython often requires GCC (at least it used to).
Oh, and most of the tools don't have support for having a different pyproject.toml source.

Now, if you look at the instructions, there's make test which, if you inspect Makefile has a chain of dependencies test -> .develop -> .install-deps -> cythonize -> "running cython over all sources".

So just running make test should also cythonize stuff for development. If you know how to improve the docs, though, I'm always open to reasonable improvements.

[guusbertens]

@webknjaz Thanks for your extensive answer. I agree, there is make test, but if one follows the instructions on the Contributing page, one will try to take these steps:

1. git clone
2. create and activate venv
3. pip install -r requirements/dev.txt
4. make test

Step 3 fails - you wouldn't make it to step 4. However, as you mention, due to the way the Makefile is set up, it will do the right thing even without step 3 ever being done.

In short, I think it's best to just remove pip install -r requirements/dev.txt from the contributing page, and have people go directly to make test.

Also, on that same page: it seems like make cov doesn't work. If I install tox it works, but it doesn't give the expected result. The command make cov-dev works immediately, and gives the expected result. Should I open a new issue for that?

[webknjaz]

1. Yes, I guess that could be removed.
2. tox is not integrated and is likely out of date. You may send a PR right away.
   If you look at the CI, it doesn't even use make for that anymore. IMHO there's
   a lot of things that could be refactored, I'd even replace make with tox but
   some other maintainers don't share this opinion so it's stuck in this half-
   broken state now.

[guusbertens]

Okay, I'll prepare a PR for both things later today. Thanks for your feedback!

[guusbertens]

Hah, #4161 already solves most of this. However, that documentation didn't go to docs.aiohttp.org yet, which is why I missed it.