Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error building from sources. #4810

Closed
luckydenis opened this issue Jun 10, 2020 · 15 comments
Closed

Error building from sources. #4810

luckydenis opened this issue Jun 10, 2020 · 15 comments
Labels
bug invalid This doesn't seem right

Comments

@luckydenis
Copy link

馃悶 Describe the bug

When building according to the instructions, an error occurs.

馃挕 To Reproduce

Correct code build.

馃搵 Logs/tracebacks

...
...
...
Requirement already satisfied, skipping upgrade: PyJWT[crypto] in ./venv/lib/python3.8/site-packages (from gidgethub->cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (1.7.1)
Installing collected packages: aiohttp
  Attempting uninstall: aiohttp
    Found existing installation: aiohttp 3.6.2
    Uninstalling aiohttp-3.6.2:
      Successfully uninstalled aiohttp-3.6.2
  Running setup.py develop for aiohttp
    ERROR: Command errored out with exit status 1:
     command: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/denis/Project/aiohttp/
    Complete output (29 lines):
    **********************
    * Accellerated build *
    **********************
    running develop
    running egg_info
    writing aiohttp.egg-info/PKG-INFO
    writing dependency_links to aiohttp.egg-info/dependency_links.txt
    writing requirements to aiohttp.egg-info/requires.txt
    writing top-level names to aiohttp.egg-info/top_level.txt
    reading manifest file 'aiohttp.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'aiohttp' anywhere in distribution
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*.so' found anywhere in distribution
    warning: no previously-included files matching '*.lib' found anywhere in distribution
    warning: no previously-included files matching '*.dll' found anywhere in distribution
    warning: no previously-included files matching '*.a' found anywhere in distribution
    warning: no previously-included files matching '*.obj' found anywhere in distribution
    warning: no previously-included files found matching 'aiohttp/*.html'
    no previously-included directories found matching 'docs/_build'
    writing manifest file 'aiohttp.egg-info/SOURCES.txt'
    running build_ext
    building 'aiohttp._websocket' extension
    x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/denis/Project/aiohttp/venv/include -I/usr/include/python3.8 -c aiohttp/_websocket.c -o build/temp.linux-x86_64-3.8/aiohttp/_websocket.o
    x86_64-linux-gnu-gcc: error: aiohttp/_websocket.c: 袧械褌 褌邪泻芯谐芯 褎邪泄谢邪 懈谢懈 泻邪褌邪谢芯谐邪
    x86_64-linux-gnu-gcc: fatal error: no input files
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
  Rolling back uninstall of aiohttp
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp-3.6.2.dist-info/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp-3.6.2.dist-info
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp
ERROR: Command errored out with exit status 1: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

馃搵 Your version of the Python

$ python --version
Python 3.8.2

馃搵 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.6.2
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: fafhrd91@gmail.com
License: Apache 2
Location: /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages
Requires: attrs, async-timeout, chardet, multidict, yarl
Required-by: pytest-aiohttp
$ python -m pip show multidict
Name: multidict
Version: 4.7.6
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp
$ python -m pip show yarl
Name: yarl
Version: 1.4.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp

馃搵 Additional context

OS: Ubuntu 20.04 LTS

@luckydenis luckydenis added the bug label Jun 10, 2020
@webknjaz
Copy link
Member

webknjaz commented Jun 10, 2020

I guess you missed something and didn't cythonize the sources. FWIW there's no "build" instructions on that page so I don't even know what exactly you ran.

@webknjaz webknjaz added the invalid This doesn't seem right label Jun 10, 2020
@luckydenis
Copy link
Author

I guess you missed something and didn't cythonize the sources. FWIW there's no "build" instructions on that page so I don't even know what exactly you ran.

pip install -r requirements/dev.txt

Full trace:

pip install -r requirements/dev.txt
Ignoring uvloop: markers 'platform_system != "Windows" and implementation_name == "cpython" and python_version < "3.7"' don't match your environment
Ignoring idna-ssl: markers 'python_version < "3.7"' don't match your environment
Obtaining file:///home/denis/Project/aiohttp (from -r requirements/ci.txt (line 8))
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: setuptools-git==1.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 1)) (1.2)
Requirement already satisfied: mypy==0.770 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 2)) (0.770)
Requirement already satisfied: mypy-extensions==0.4.3 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 3)) (0.4.3)
Requirement already satisfied: freezegun==0.3.15 in ./venv/lib/python3.8/site-packages (from -r requirements/ci.txt (line 4)) (0.3.15)
Requirement already satisfied: flake8==3.7.9 in ./venv/lib/python3.8/site-packages (from -r requirements/flake.txt (line 1)) (3.7.9)
Requirement already satisfied: isort==4.3.21 in ./venv/lib/python3.8/site-packages (from -r requirements/flake.txt (line 2)) (4.3.21)
Requirement already satisfied: attrs==19.3.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 2)) (19.3.0)
Requirement already satisfied: async-timeout==4.0.0a2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 3)) (4.0.0a2)
Requirement already satisfied: Brotli==1.0.7 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 4)) (1.0.7)
Requirement already satisfied: cchardet==2.1.6 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 5)) (2.1.6)
Requirement already satisfied: chardet==3.0.4 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 6)) (3.0.4)
Requirement already satisfied: coverage==5.1 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 7)) (5.1)
Requirement already satisfied: gunicorn==20.0.4 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 8)) (20.0.4)
Requirement already satisfied: multidict==4.7.6 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 9)) (4.7.6)
Requirement already satisfied: pytest==5.4.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 10)) (5.4.2)
Requirement already satisfied: pytest-cov==2.8.1 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 11)) (2.8.1)
Requirement already satisfied: pytest-mock==3.1.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 12)) (3.1.0)
Requirement already satisfied: typing_extensions==3.7.4.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 13)) (3.7.4.2)
Requirement already satisfied: yarl==1.4.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 14)) (1.4.2)
Requirement already satisfied: aiodns==2.0.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 19)) (2.0.0)
Requirement already satisfied: cryptography==2.9.2 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 20)) (2.9.2)
Requirement already satisfied: trustme==0.6.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 21)) (0.6.0)
Requirement already satisfied: codecov==2.1.0 in ./venv/lib/python3.8/site-packages (from -r requirements/ci-wheel.txt (line 22)) (2.1.0)
Requirement already satisfied: sphinx==2.4.4 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 1)) (2.4.4)
Requirement already satisfied: sphinxcontrib-asyncio==0.2.0 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 2)) (0.2.0)
Requirement already satisfied: pygments==2.6.1 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 3)) (2.6.1)
Requirement already satisfied: aiohttp-theme==0.1.6 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 4)) (0.1.6)
Requirement already satisfied: sphinxcontrib-blockdiag==2.0.0 in ./venv/lib/python3.8/site-packages (from -r requirements/doc.txt (line 5)) (2.0.0)
Requirement already satisfied: towncrier==19.2.0 in ./venv/lib/python3.8/site-packages (from -r requirements/towncrier.txt (line 1)) (19.2.0)
Requirement already satisfied: cherry_picker==1.3.2 in ./venv/lib/python3.8/site-packages (from -r requirements/dev.txt (line 3)) (1.3.2)
Requirement already satisfied: typed-ast<1.5.0,>=1.4.0 in ./venv/lib/python3.8/site-packages (from mypy==0.770->-r requirements/ci.txt (line 2)) (1.4.1)
Requirement already satisfied: python-dateutil!=2.0,>=1.0 in ./venv/lib/python3.8/site-packages (from freezegun==0.3.15->-r requirements/ci.txt (line 4)) (2.8.1)
Requirement already satisfied: six in ./venv/lib/python3.8/site-packages (from freezegun==0.3.15->-r requirements/ci.txt (line 4)) (1.15.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (0.6.1)
Requirement already satisfied: pycodestyle<2.6.0,>=2.5.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (2.5.0)
Requirement already satisfied: pyflakes<2.2.0,>=2.1.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (2.1.1)
Requirement already satisfied: entrypoints<0.4.0,>=0.3.0 in ./venv/lib/python3.8/site-packages (from flake8==3.7.9->-r requirements/flake.txt (line 1)) (0.3)
Requirement already satisfied: setuptools>=3.0 in ./venv/lib/python3.8/site-packages (from gunicorn==20.0.4->-r requirements/ci-wheel.txt (line 8)) (47.1.1)
Requirement already satisfied: wcwidth in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (0.2.4)
Requirement already satisfied: packaging in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (20.4)
Requirement already satisfied: py>=1.5.0 in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (1.8.1)
Requirement already satisfied: pluggy<1.0,>=0.12 in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in ./venv/lib/python3.8/site-packages (from pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (8.3.0)
Requirement already satisfied: idna>=2.0 in ./venv/lib/python3.8/site-packages (from yarl==1.4.2->-r requirements/ci-wheel.txt (line 14)) (2.9)
Requirement already satisfied: pycares>=3.0.0 in ./venv/lib/python3.8/site-packages (from aiodns==2.0.0->-r requirements/ci-wheel.txt (line 19)) (3.1.1)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in ./venv/lib/python3.8/site-packages (from cryptography==2.9.2->-r requirements/ci-wheel.txt (line 20)) (1.14.0)
Requirement already satisfied: requests>=2.7.9 in ./venv/lib/python3.8/site-packages (from codecov==2.1.0->-r requirements/ci-wheel.txt (line 22)) (2.23.0)
Requirement already satisfied: alabaster<0.8,>=0.7 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (0.7.12)
Requirement already satisfied: babel!=2.0,>=1.3 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2.8.0)
Requirement already satisfied: sphinxcontrib-htmlhelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.3)
Requirement already satisfied: sphinxcontrib-qthelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.3)
Requirement already satisfied: imagesize in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.2.0)
Requirement already satisfied: sphinxcontrib-jsmath in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.1)
Requirement already satisfied: snowballstemmer>=1.1 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2.0.0)
Requirement already satisfied: sphinxcontrib-serializinghtml in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.1.4)
Requirement already satisfied: Jinja2>=2.3 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2.11.2)
Requirement already satisfied: docutils>=0.12 in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (0.16)
Requirement already satisfied: sphinxcontrib-devhelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.2)
Requirement already satisfied: sphinxcontrib-applehelp in ./venv/lib/python3.8/site-packages (from sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.0.2)
Requirement already satisfied: blockdiag>=1.5.0 in ./venv/lib/python3.8/site-packages (from sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (2.0.1)
Requirement already satisfied: incremental in ./venv/lib/python3.8/site-packages (from towncrier==19.2.0->-r requirements/towncrier.txt (line 1)) (17.5.0)
Requirement already satisfied: toml in ./venv/lib/python3.8/site-packages (from towncrier==19.2.0->-r requirements/towncrier.txt (line 1)) (0.10.1)
Requirement already satisfied: Click in ./venv/lib/python3.8/site-packages (from towncrier==19.2.0->-r requirements/towncrier.txt (line 1)) (7.1.2)
Requirement already satisfied: gidgethub in ./venv/lib/python3.8/site-packages (from cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (4.1.1)
Requirement already satisfied: pyparsing>=2.0.2 in ./venv/lib/python3.8/site-packages (from packaging->pytest==5.4.2->-r requirements/ci-wheel.txt (line 10)) (2.4.7)
Requirement already satisfied: pycparser in ./venv/lib/python3.8/site-packages (from cffi!=1.11.3,>=1.8->cryptography==2.9.2->-r requirements/ci-wheel.txt (line 20)) (2.20)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./venv/lib/python3.8/site-packages (from requests>=2.7.9->codecov==2.1.0->-r requirements/ci-wheel.txt (line 22)) (1.25.9)
Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.8/site-packages (from requests>=2.7.9->codecov==2.1.0->-r requirements/ci-wheel.txt (line 22)) (2020.4.5.2)
Requirement already satisfied: pytz>=2015.7 in ./venv/lib/python3.8/site-packages (from babel!=2.0,>=1.3->sphinx==2.4.4->-r requirements/doc.txt (line 1)) (2020.1)
Requirement already satisfied: MarkupSafe>=0.23 in ./venv/lib/python3.8/site-packages (from Jinja2>=2.3->sphinx==2.4.4->-r requirements/doc.txt (line 1)) (1.1.1)
Requirement already satisfied: funcparserlib in ./venv/lib/python3.8/site-packages (from blockdiag>=1.5.0->sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (0.3.6)
Requirement already satisfied: Pillow>3.0 in ./venv/lib/python3.8/site-packages (from blockdiag>=1.5.0->sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (7.1.2)
Requirement already satisfied: webcolors in ./venv/lib/python3.8/site-packages (from blockdiag>=1.5.0->sphinxcontrib-blockdiag==2.0.0->-r requirements/doc.txt (line 5)) (1.11.1)
Requirement already satisfied: PyJWT[crypto] in ./venv/lib/python3.8/site-packages (from gidgethub->cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (1.7.1)
Requirement already satisfied: uritemplate>=3.0.1 in ./venv/lib/python3.8/site-packages (from gidgethub->cherry_picker==1.3.2->-r requirements/dev.txt (line 3)) (3.0.1)
Installing collected packages: aiohttp
  Attempting uninstall: aiohttp
    Found existing installation: aiohttp 3.6.2
    Uninstalling aiohttp-3.6.2:
      Successfully uninstalled aiohttp-3.6.2
  Running setup.py develop for aiohttp
    ERROR: Command errored out with exit status 1:
     command: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/denis/Project/aiohttp/
    Complete output (29 lines):
    **********************
    * Accellerated build *
    **********************
    running develop
    running egg_info
    writing aiohttp.egg-info/PKG-INFO
    writing dependency_links to aiohttp.egg-info/dependency_links.txt
    writing requirements to aiohttp.egg-info/requires.txt
    writing top-level names to aiohttp.egg-info/top_level.txt
    reading manifest file 'aiohttp.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'aiohttp' anywhere in distribution
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*.so' found anywhere in distribution
    warning: no previously-included files matching '*.lib' found anywhere in distribution
    warning: no previously-included files matching '*.dll' found anywhere in distribution
    warning: no previously-included files matching '*.a' found anywhere in distribution
    warning: no previously-included files matching '*.obj' found anywhere in distribution
    warning: no previously-included files found matching 'aiohttp/*.html'
    no previously-included directories found matching 'docs/_build'
    writing manifest file 'aiohttp.egg-info/SOURCES.txt'
    running build_ext
    building 'aiohttp._websocket' extension
    x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/denis/Project/aiohttp/venv/include -I/usr/include/python3.8 -c aiohttp/_websocket.c -o build/temp.linux-x86_64-3.8/aiohttp/_websocket.o
    x86_64-linux-gnu-gcc: error: aiohttp/_websocket.c: 袧械褌 褌邪泻芯谐芯 褎邪泄谢邪 懈谢懈 泻邪褌邪谢芯谐邪
    x86_64-linux-gnu-gcc: fatal error: no input files
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
  Rolling back uninstall of aiohttp
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp-3.6.2.dist-info/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp-3.6.2.dist-info
  Moving to /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/aiohttp/
   from /home/denis/Project/aiohttp/venv/lib/python3.8/site-packages/~iohttp
ERROR: Command errored out with exit status 1: /home/denis/Project/aiohttp/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/denis/Project/aiohttp/setup.py'"'"'; __file__='"'"'/home/denis/Project/aiohttp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

@guusbertens
Copy link
Contributor

I get the same. I'm on Debian Testing (bullseye). In particular these steps give what LuckyDenis also sees:

git clone $REPO $DIR --recurse-submodules
cd $DIR
python3 -m venv venv
. venv/bin/activate
pip install -r requirements/dev.txt

These are the suggested steps from the contributing section. However I need to add the following steps before the last step:

pip install wheel
pip install cython
make cythonize

With this everything works as expected.

@guusbertens
Copy link
Contributor

guusbertens commented Jun 11, 2020

Indeed, setup.py states that the makefile runs cythonize. The setuptools documentation confirms that setuptools doesn't run cythonize; it only compiles .c files, and hence the cythonized code should be included in the git repo. The cython documentation suggests using the cythonize function from within setup.py, so that pip can cythonize all that is needed.

@luckydenis
Copy link
Author

luckydenis commented Jun 12, 2020

It seems that this point is not obvious and it would be good to reflect this in the documentation or add the necessary commands to the installation scripts.
@webknjaz, @guusbertens

@webknjaz
Copy link
Member

We moved cythonize out of setup.py in order to have Cython an optional build dependency.

@webknjaz
Copy link
Member

I don't see you running make test. What exactly did you do?

@guusbertens
Copy link
Contributor

@webknjaz I just ran this:

git clone $REPO $DIR --recurse-submodules
cd $DIR
python3 -m venv venv
. venv/bin/activate
pip install wheel
pip install cython
make cythonize
pip install -r requirements/dev.txt
make test

This works like a charm.

Do you agree that, given what is said in the setuptools docs and the Cython docs, the current breakage is expected, and that this issue is indeed a bug, and isn't an invalid issue?

You mention that you keeping Cython optional is important. As far as setup.py is concerned, Cython could be in there while being optional, but optional dependencies aren't supported by pyproject.toml, and this causes the problem. Am I right about this? If this is the case, then I think there are two options:

  • include cythonize'd C-files in the git repo; or
  • update the documentation to add the cythonize step; or
  • maybe using two different pyproject.toml files, but I think this overcomplicates matters a lot...

Am I missing something?

@webknjaz
Copy link
Member

Do you agree that, given what is said in the setuptools docs and the Cython docs, the current breakage is expected, and that this issue is indeed a bug, and isn't an invalid issue?

This is controlled by AIOHTTP_NO_EXTENSIONS=1 env var. If it's set, setup.py won't attempt to build the C-extensions. sdists on PyPI are shipped with pre-cythonized sources (C-files) and during development you really have to be explicit about what you want and also the default is to build the extensions (so that you accidentally don't test pure-python while thinking that you also test Cython).

You mention that you keeping Cython optional is important. As far as setup.py is concerned, Cython could be in there while being optional, but optional dependencies aren't supported by pyproject.toml, and this causes the problem. Am I right about this? If this is the case, then I think there are two options:

  • include cythonize'd C-files in the git repo; or
  • update the documentation to add the cythonize step; or
  • maybe using two different pyproject.toml files, but I think this overcomplicates matters a lot...

Am I missing something?

Before pyproject.toml was a thing and before pip started doing mandatory build isolation we could ask users to pip install Cython into the same env and that would do the trick. Now we cannot do this because the build env is always different. And if we put Cython into setup_requires, it'd be mandatory. We cannot do this because even just installing Cython often requires GCC (at least it used to).
Oh, and most of the tools don't have support for having a different pyproject.toml source.

Now, if you look at the instructions, there's make test which, if you inspect Makefile has a chain of dependencies test -> .develop -> .install-deps -> cythonize -> "running cython over all sources".

So just running make test should also cythonize stuff for development. If you know how to improve the docs, though, I'm always open to reasonable improvements.

@guusbertens
Copy link
Contributor

@webknjaz Thanks for your extensive answer. I agree, there is make test, but if one follows the instructions on the Contributing page, one will try to take these steps:

  1. git clone
  2. create and activate venv
  3. pip install -r requirements/dev.txt
  4. make test

Step 3 fails - you wouldn't make it to step 4. However, as you mention, due to the way the Makefile is set up, it will do the right thing even without step 3 ever being done.

In short, I think it's best to just remove pip install -r requirements/dev.txt from the contributing page, and have people go directly to make test.

Also, on that same page: it seems like make cov doesn't work. If I install tox it works, but it doesn't give the expected result. The command make cov-dev works immediately, and gives the expected result. Should I open a new issue for that?

@webknjaz
Copy link
Member

  1. Yes, I guess that could be removed.
  2. tox is not integrated and is likely out of date. You may send a PR right away.
    If you look at the CI, it doesn't even use make for that anymore. IMHO there's
    a lot of things that could be refactored, I'd even replace make with tox but
    some other maintainers don't share this opinion so it's stuck in this half-
    broken state now.

@guusbertens
Copy link
Contributor

Okay, I'll prepare a PR for both things later today. Thanks for your feedback!

@guusbertens
Copy link
Contributor

Hah, #4161 already solves most of this. However, that documentation didn't go to docs.aiohttp.org yet, which is why I missed it.

This was referenced Jun 12, 2020
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Oct 24, 2020
This fixes py-yarl in pkgsrc being too new for py-aiohttp.


3.7.0 (2020-10-24)
==================

Features
--------

- Response headers are now prepared prior to running ``on_response_prepare`` hooks, directly before headers are sent to the client.
  `#1958 <https://github.com/aio-libs/aiohttp/issues/1958>`_
- Add a ``quote_cookie`` option to ``CookieJar``, a way to skip quotation wrapping of cookies containing special characters.
  `#2571 <https://github.com/aio-libs/aiohttp/issues/2571>`_
- Call ``AccessLogger.log`` with the current exception available from ``sys.exc_info()``.
  `#3557 <https://github.com/aio-libs/aiohttp/issues/3557>`_
- `web.UrlDispatcher.add_routes` and `web.Application.add_routes` return a list
  of registered `AbstractRoute` instances. `AbstractRouteDef.register` (and all
  subclasses) return a list of registered resources registered resource.
  `#3866 <https://github.com/aio-libs/aiohttp/issues/3866>`_
- Added properties of default ClientSession params to ClientSession class so it is available for introspection
  `#3882 <https://github.com/aio-libs/aiohttp/issues/3882>`_
- Don't cancel web handler on peer disconnection, raise `OSError` on reading/writing instead.
  `#4080 <https://github.com/aio-libs/aiohttp/issues/4080>`_
- Implement BaseRequest.get_extra_info() to access a protocol transports' extra info.
  `#4189 <https://github.com/aio-libs/aiohttp/issues/4189>`_
- Added `ClientSession.timeout` property.
  `#4191 <https://github.com/aio-libs/aiohttp/issues/4191>`_
- allow use of SameSite in cookies.
  `#4224 <https://github.com/aio-libs/aiohttp/issues/4224>`_
- Use ``loop.sendfile()`` instead of custom implementation if available.
  `#4269 <https://github.com/aio-libs/aiohttp/issues/4269>`_
- Apply SO_REUSEADDR to test server's socket.
  `#4393 <https://github.com/aio-libs/aiohttp/issues/4393>`_
- Use .raw_host instead of slower .host in client API
  `#4402 <https://github.com/aio-libs/aiohttp/issues/4402>`_
- Allow configuring the buffer size of input stream by passing ``read_bufsize`` argument.
  `#4453 <https://github.com/aio-libs/aiohttp/issues/4453>`_
- Pass tests on Python 3.8 for Windows.
  `#4513 <https://github.com/aio-libs/aiohttp/issues/4513>`_
- Add `method` and `url` attributes to `TraceRequestChunkSentParams` and `TraceResponseChunkReceivedParams`.
  `#4674 <https://github.com/aio-libs/aiohttp/issues/4674>`_
- Add ClientResponse.ok property for checking status code under 400.
  `#4711 <https://github.com/aio-libs/aiohttp/issues/4711>`_
- Don't ceil timeouts that are smaller than 5 seconds.
  `#4850 <https://github.com/aio-libs/aiohttp/issues/4850>`_
- TCPSite now listens by default on all interfaces instead of just IPv4 when `None` is passed in as the host.
  `#4894 <https://github.com/aio-libs/aiohttp/issues/4894>`_
- Bump ``http_parser`` to 2.9.4
  `#5070 <https://github.com/aio-libs/aiohttp/issues/5070>`_


Bugfixes
--------

- Fix keepalive connections not being closed in time
  `#3296 <https://github.com/aio-libs/aiohttp/issues/3296>`_
- Fix failed websocket handshake leaving connection hanging.
  `#3380 <https://github.com/aio-libs/aiohttp/issues/3380>`_
- Fix tasks cancellation order on exit. The run_app task needs to be cancelled first for cleanup hooks to run with all tasks intact.
  `#3805 <https://github.com/aio-libs/aiohttp/issues/3805>`_
- Don't start heartbeat until _writer is set
  `#4062 <https://github.com/aio-libs/aiohttp/issues/4062>`_
- Fix handling of multipart file uploads without a content type.
  `#4089 <https://github.com/aio-libs/aiohttp/issues/4089>`_
- Preserve view handler function attributes across middlewares
  `#4174 <https://github.com/aio-libs/aiohttp/issues/4174>`_
- Fix the string representation of ``ServerDisconnectedError``.
  `#4175 <https://github.com/aio-libs/aiohttp/issues/4175>`_
- Raising RuntimeError when trying to get encoding from not read body
  `#4214 <https://github.com/aio-libs/aiohttp/issues/4214>`_
- Remove warning messages from noop.
  `#4282 <https://github.com/aio-libs/aiohttp/issues/4282>`_
- Raise ClientPayloadError if FormData re-processed.
  `#4345 <https://github.com/aio-libs/aiohttp/issues/4345>`_
- Fix a warning about unfinished task in ``web_protocol.py``
  `#4408 <https://github.com/aio-libs/aiohttp/issues/4408>`_
- Fixed 'deflate' compression. According to RFC 2616 now.
  `#4506 <https://github.com/aio-libs/aiohttp/issues/4506>`_
- Fixed OverflowError on platforms with 32-bit time_t
  `#4515 <https://github.com/aio-libs/aiohttp/issues/4515>`_
- Fixed request.body_exists returns wrong value for methods without body.
  `#4528 <https://github.com/aio-libs/aiohttp/issues/4528>`_
- Fix connecting to link-local IPv6 addresses.
  `#4554 <https://github.com/aio-libs/aiohttp/issues/4554>`_
- Fix a problem with connection waiters that are never awaited.
  `#4562 <https://github.com/aio-libs/aiohttp/issues/4562>`_
- Always make sure transport is not closing before reuse a connection.

  Reuse a protocol based on keepalive in headers is unreliable.
  For example, uWSGI will not support keepalive even it serves a
  HTTP 1.1 request, except explicitly configure uWSGI with a
  ``--http-keepalive`` option.

  Servers designed like uWSGI could cause aiohttp intermittently
  raise a ConnectionResetException when the protocol poll runs
  out and some protocol is reused.
  `#4587 <https://github.com/aio-libs/aiohttp/issues/4587>`_
- Handle the last CRLF correctly even if it is received via separate TCP segment.
  `#4630 <https://github.com/aio-libs/aiohttp/issues/4630>`_
- Fix the register_resource function to validate route name before splitting it so that route name can include python keywords.
  `#4691 <https://github.com/aio-libs/aiohttp/issues/4691>`_
- Improve typing annotations for ``web.Request``, ``aiohttp.ClientResponse`` and
  ``multipart`` module.
  `#4736 <https://github.com/aio-libs/aiohttp/issues/4736>`_
- Fix resolver task is not awaited when connector is cancelled
  `#4795 <https://github.com/aio-libs/aiohttp/issues/4795>`_
- Fix a bug "Aiohttp doesn't return any error on invalid request methods"
  `#4798 <https://github.com/aio-libs/aiohttp/issues/4798>`_
- Fix HEAD requests for static content.
  `#4809 <https://github.com/aio-libs/aiohttp/issues/4809>`_
- Fix incorrect size calculation for memoryview
  `#4890 <https://github.com/aio-libs/aiohttp/issues/4890>`_
- Add HTTPMove to _all__.
  `#4897 <https://github.com/aio-libs/aiohttp/issues/4897>`_
- Fixed the type annotations in the ``tracing`` module.
  `#4912 <https://github.com/aio-libs/aiohttp/issues/4912>`_
- Fix typing for multipart ``__aiter__``.
  `#4931 <https://github.com/aio-libs/aiohttp/issues/4931>`_
- Fix for race condition on connections in BaseConnector that leads to exceeding the connection limit.
  `#4936 <https://github.com/aio-libs/aiohttp/issues/4936>`_
- Add forced UTF-8 encoding for ``application/rdap+json`` responses.
  `#4938 <https://github.com/aio-libs/aiohttp/issues/4938>`_
- Fix inconsistency between Python and C http request parsers in parsing pct-encoded URL.
  `#4972 <https://github.com/aio-libs/aiohttp/issues/4972>`_
- Fix connection closing issue in HEAD request.
  `#5012 <https://github.com/aio-libs/aiohttp/issues/5012>`_
- Fix type hint on BaseRunner.addresses (from ``List[str]`` to ``List[Any]``)
  `#5086 <https://github.com/aio-libs/aiohttp/issues/5086>`_
- Make `web.run_app()` more responsive to Ctrl+C on Windows for Python < 3.8. It slightly
  increases CPU load as a side effect.
  `#5098 <https://github.com/aio-libs/aiohttp/issues/5098>`_


Improved Documentation
----------------------

- Fix example code in client quick-start
  `#3376 <https://github.com/aio-libs/aiohttp/issues/3376>`_
- Updated the docs so there is no contradiction in ``ttl_dns_cache`` default value
  `#3512 <https://github.com/aio-libs/aiohttp/issues/3512>`_
- Add 'Deploy with SSL' to docs.
  `#4201 <https://github.com/aio-libs/aiohttp/issues/4201>`_
- Change typing of the secure argument on StreamResponse.set_cookie from ``Optional[str]`` to ``Optional[bool]``
  `#4204 <https://github.com/aio-libs/aiohttp/issues/4204>`_
- Changes ``ttl_dns_cache`` type from int to Optional[int].
  `#4270 <https://github.com/aio-libs/aiohttp/issues/4270>`_
- Simplify README hello word example and add a documentation page for people coming from requests.
  `#4272 <https://github.com/aio-libs/aiohttp/issues/4272>`_
- Improve some code examples in the documentation involving websockets and starting a simple HTTP site with an AppRunner.
  `#4285 <https://github.com/aio-libs/aiohttp/issues/4285>`_
- Fix typo in code example in Multipart docs
  `#4312 <https://github.com/aio-libs/aiohttp/issues/4312>`_
- Fix code example in Multipart section.
  `#4314 <https://github.com/aio-libs/aiohttp/issues/4314>`_
- Update contributing guide so new contributors read the most recent version of that guide. Update command used to create test coverage reporting.
  `#4810 <https://github.com/aio-libs/aiohttp/issues/4810>`_
- Spelling: Change "canonize" to "canonicalize".
  `#4986 <https://github.com/aio-libs/aiohttp/issues/4986>`_
- Add ``aiohttp-sse-client`` library to third party usage list.
  `#5084 <https://github.com/aio-libs/aiohttp/issues/5084>`_


Misc
----

- `#2856 <https://github.com/aio-libs/aiohttp/issues/2856>`_, `#4218 <https://github.com/aio-libs/aiohttp/issues/4218>`_, `#4250 <https://github.com/aio-libs/aiohttp/issues/4250>`_
@jmgurney
Copy link

jmgurney commented Nov 8, 2023

I'd like to see setup.py detect that the C files are missing, and automatically set AIOHTTP_NO_EXTENSIONS=1. W/o this, it means that it's not really possible to build from source as a dependency from another package.

something like:

diff --git a/setup.py b/setup.py
index 2622e5c5..33e8e6fe 100644
--- a/setup.py
+++ b/setup.py
@@ -44,6 +44,8 @@ extensions = [
     Extension("aiohttp._http_writer", ["aiohttp/_http_writer.c"]),
 ]
 
+if not all((pathlib.Path(fname).exists() for ext in extensions for fname in ext.sources)):
+    NO_EXTENSIONS = True
 
 build_type = "Pure" if NO_EXTENSIONS else "Accelerated"
 setup_kwargs = {} if NO_EXTENSIONS else {"ext_modules": extensions}

@webknjaz
Copy link
Member

webknjaz commented Nov 8, 2023

Absolutely not. This would mean that people unknowingly get a slow implementation by default, which is unaccessible. The proper solution is to use dists on PyPI that bundle everything necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug invalid This doesn't seem right
Projects
None yet
Development

No branches or pull requests

5 participants