Yarl should ship manulinux1 wheels for Python 3.7

[socketpair]

$ ls -l /usr/bin/python
lrwxrwxrwx 1 root root 9 апр 16  2018 /usr/bin/python -> python2.7

Collecting aiohttp==3.5.4
  Using cached https://files.pythonhosted.org/packages/33/60/c21acb7002110699761d3dec36fad3f5087c2752a4eb495444f877db6553/aiohttp-3.5.4-cp37-cp37m-manylinux1_x86_64.whl
Collecting attrs>=17.3.0 (from aiohttp==3.5.4)
  Using cached https://files.pythonhosted.org/packages/23/96/d828354fa2dbdf216eaa7b7de0db692f12c234f7ef888cc14980ef40d1d2/attrs-19.1.0-py2.py3-none-any.whl
Collecting yarl<2.0,>=1.0 (from aiohttp==3.5.4)
  Using cached https://files.pythonhosted.org/packages/fb/84/6d82f6be218c50b547aa29d0315e430cf8a23c52064c92d0a8377d7b7357/yarl-1.3.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-w0tfguog/yarl/setup.py", line 25, in <module>
        extensions = cythonize(extensions)
      File "/usr/lib/python3/dist-packages/Cython/Build/Dependencies.py", line 899, in cythonize
        ctx = c_options.create_context()
      File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 590, in create_context
        self.cplus, self.language_level, options=self)
      File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 75, in __init__
        from . import Builtin, CythonScope
      File "/usr/lib/python3/dist-packages/Cython/Compiler/CythonScope.py", line 5, in <module>
        from .UtilityCode import CythonUtilityCode
      File "/usr/lib/python3/dist-packages/Cython/Compiler/UtilityCode.py", line 3, in <module>
        from .TreeFragment import parse_from_strings, StringParseContext
      File "/usr/lib/python3/dist-packages/Cython/Compiler/TreeFragment.py", line 17, in <module>
        from .Visitor import VisitorTransform
      File "/usr/lib/python3/dist-packages/Cython/Compiler/Visitor.py", line 15, in <module>
        from . import ExprNodes
      File "/usr/lib/python3/dist-packages/Cython/Compiler/ExprNodes.py", line 2844
        await = None
              ^
    SyntaxError: invalid syntax
    Error in sys.excepthook:
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
        from apport.fileutils import likely_packaged, get_recent_crashes
      File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
        from apport.report import Report
      File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
        import apport.fileutils
      File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
        from apport.packaging_impl import impl as packaging
      File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 24, in <module>
        import apt
      File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
        import apt_pkg
    ModuleNotFoundError: No module named 'apt_pkg'
    
    Original exception was:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-w0tfguog/yarl/setup.py", line 25, in <module>
        extensions = cythonize(extensions)
      File "/usr/lib/python3/dist-packages/Cython/Build/Dependencies.py", line 899, in cythonize
        ctx = c_options.create_context()
      File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 590, in create_context
        self.cplus, self.language_level, options=self)
      File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 75, in __init__
        from . import Builtin, CythonScope
      File "/usr/lib/python3/dist-packages/Cython/Compiler/CythonScope.py", line 5, in <module>
        from .UtilityCode import CythonUtilityCode
      File "/usr/lib/python3/dist-packages/Cython/Compiler/UtilityCode.py", line 3, in <module>
        from .TreeFragment import parse_from_strings, StringParseContext
      File "/usr/lib/python3/dist-packages/Cython/Compiler/TreeFragment.py", line 17, in <module>
        from .Visitor import VisitorTransform
      File "/usr/lib/python3/dist-packages/Cython/Compiler/Visitor.py", line 15, in <module>
        from . import ExprNodes
      File "/usr/lib/python3/dist-packages/Cython/Compiler/ExprNodes.py", line 2844
        await = None
              ^
    SyntaxError: invalid syntax
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-w0tfguog/yarl/

[webknjaz]

And it doesn't.

[socketpair]

Well, what is wrong so ?

[webknjaz]

pip runs python setup.py install under the hood when it's forced to install from sdist. But then, python in such env refers to the same python executable anyway.
Even await = None suggests that the newest Python is used because in the recent versions await is a keyword which triggers an error if used for a var.

[webknjaz]

There's no matching manylinux1 wheel published to PyPI for Python 3.7 so your pip selects sdist which tries to compile speedups using Cython.
The problem is that Cython version used is outdated and doesn't support Python 3.7.

[webknjaz]

As a quick workaround, either remove Cython from that env or upgrade it to the sufficient version.

A long-term fix would be for us to upload matching wheels. But with the latest pip changes, I haven't yet figured out the right way of having an optional build dependency, it's still on my list.

[socketpair]

Why not to add Cython as a hard requirement ?

[socketpair]

well. python3.7 -m pip install --user cython helps, but....

Collecting idna>=2.0 (from yarl<2.0,>=1.0->aiohttp==3.5.4)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Building wheels for collected packages: yarl
  Running setup.py bdist_wheel for yarl ... error
  Complete output from command /usr/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-1rnep6e_/yarl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmptt_iv42hpip-wheel- --python-tag cp37:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Failed building wheel for yarl
  Running setup.py clean for yarl
Failed to build yarl
Installing collected packages: multidict, attrs, idna, yarl, chardet, async-timeout, aiohttp
  Running setup.py install for yarl ... done
Successfully installed aiohttp-3.5.4 async-timeout-3.0.1 attrs-19.1.0 chardet-3.0.4 idna-2.8 multidict-4.5.2 yarl-1.3.0

[socketpair]

python3.7 -m pip install --user wheel fix also that. So, dependency is missing somewhere.

[socketpair]

@webknjaz I have updated the title. Dependencies must include minimal version for both Cython and wheel.

[webknjaz]

Because Andrew wanted this to work on machines where people don't have GCC. Cython itself needs GCC. It'll screw a lot of users.

Wheel is not a dependency of yarl and never will be. It's a build dependency which is only needed sometimes if user needs a wheel. Probably your pip is screwed if it didn't bring it wheel as its own dependency but tries to use it.

[webknjaz]

The title now declares another wrong view. It's not how packaging works now. And also you cannot declare build tools as a part of build flow. Just like you cannot say that you depend on pip, because if you don't have pip you wouldn't trigger install anyway.

[socketpair]

OK, I'm not so good in Python buildsystem, so OK, I just mention problems that I have faced with.

[webknjaz]

Duplicate of #211.