Python 2.7 installation is broken

[mast]

Since you dropped python 2 support, you have to stop making universal wheels for this module, otherwise pypi is offering wrong version for python2 setups.
I see that this was addressed in commit 50fa96b

Please push new version to pypi, otherwise we are having issues with installation on python2.

[exarkun]

Tahoe-LAFS CI is now almost entirely failing because of this issue.

[kvitali]

datadog and pytest packages install with pip2 fails because of this issue

[sdatko]

Ech, a new release just before the Easter... I guess many people will get surprise after coming from Holidays/PTO ;-)

My tests for legacy builds are also failing due to this, though I am not even sure which package requires decorator as dependency.

[vphilippon]

So I saw that the 5.0.1 release was uploaded with the proper python_requires='>=3.5', which is good.
But it seems that the metadata didn't get sent to pypi.org though.

From my experience on other projects getting that issue, I would bet this is caused by performing the upload using an old setuptools or twine. I'd suggest uploading a 5.0.2 using the latest twine available. And then yanking 5.0.1.

Hoping you'll be able to get out of the Python 2 realm with peace of mind with that done :)

[MrMino]

@vphilippon damn, I dug through nearly half of pip codebase until I realized it's not taken from dist-info/METADATA...

Thanks for the insight, would've saved me a lot of time if I read it first 😄.

[altendky]

The uploaded wheel file name lists .py2.py3. This at least is decided on creation, not upload. The Python version is also listed as 3.8 in the table.

https://pypi.org/project/decorator/5.0.1/#files

I tried building several ways and didn't get .py2 in the filename.

venv/bin/python -m build --wheel .
venv/bin/python setup.py bdist_wheel
venv/bin/pip wheel --no-deps --out-dir dist/ .

Of course that was with everything updated, so perhaps it is outdated setuptools as suggested.

[micheles]

Uploaded release 5.0.2, let's hope this is the good one...

[altendky]

edit: updated to include updating

$ virtualenv -p python2 venv2    
created virtual environment CPython2.7.18.final.0-64 in 78ms
  creator CPython2Posix(dest=/home/altendky/venv2, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/altendky/.local/share/virtualenv)
    added seed packages: pip==20.3.1, setuptools==44.1.1, wheel==0.36.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator

$ venv2/bin/pip install --upgrade pip setuptools wheel
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting pip
  Using cached pip-20.3.4-py2.py3-none-any.whl (1.5 MB)
Requirement already up-to-date: setuptools in ./venv2/lib/python2.7/site-packages (44.1.1)
Collecting wheel
  Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Installing collected packages: pip, wheel
  Attempting uninstall: pip
    Found existing installation: pip 20.3.1
    Uninstalling pip-20.3.1:
      Successfully uninstalled pip-20.3.1
  Attempting uninstall: wheel
    Found existing installation: wheel 0.36.1
    Uninstalling wheel-0.36.1:
      Successfully uninstalled wheel-0.36.1
Successfully installed pip-20.3.4 wheel-0.36.2

$ venv2/bin/pip install decorator
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting decorator
  Using cached decorator-5.0.2.tar.gz (33 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/altendky/venv2/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-YVO1qP/decorator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-YVO1qP/decorator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-tf8mF2
         cwd: /tmp/pip-install-YVO1qP/decorator/
    Complete output (8 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-YVO1qP/decorator/setup.py", line 4, in <module>
        exec(open('src/decorator.py').read(), dic)  # extract the __version__
      File "<string>", line 162
        print('Error in generated code:', file=sys.stderr)
                                              ^
    SyntaxError: invalid syntax
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[sdatko]

I observe the same issue as @altendky on Python 2.7.5 with pip==20.3.4 and setuptools-44.1.1.

[mast]

I confirm, same issue as @altendky for my setup as well with 5.0.2

[micheles]

Sorry if I am being thick, but the point of version 5 of the decorator module was to drop support for Python 2.7, so it is right if it does not work with it. python_requires and the universal flag are fixed, what else needs to be done to finally stop automatic build systems from trying to use version 5.X?

[vphilippon]

So from the pypi.org page (https://pypi.org/project/decorator/), you can see that Requires: Python >=3.5 isn't shown in the metadata section.
An exemple that do: https://pypi.org/project/setuptools/

So that's a tell that for some reason, the metadata isn't sent to pypi.org. My best bet was the version of setuptools/twine used to upload the package, but if @micheles you can assure us you used the latest version of twine to do the upload, then I don't know right now why it doesn't work. This is going to be some diging for sure.

If you can share the versions that were used to perform the upload (like pip list in the right environment), I can try to investigate a spot something. Same if you have any logs related to the upload.

Edit: To be clear: even if the Requires-Python metadata is properly in the file (.whl, .tar.gz), that doesn't guarantee that pypi.org will be aware of that restriction and prevent the download of the package. This is why the tooling used (i.e. the version of twine) really matters to inform pypi.org of that metadata at the upload time.

[micheles]

I have installed the latest twine and then run python setup.py sdist bdist_wheel upload but perhaps it did not take twine, so now I tried again with

$ twine upload dist/*

Published 5.0.3

[altendky]

Sorry... Usually you run twine to run twine. And build is the way to build stuff these days. I rarely ever run setup.py.

Looks to be working now.

$ virtualenv -p python2 venv2
created virtual environment CPython2.7.18.final.0-64 in 77ms
  creator CPython2Posix(dest=/home/altendky/venv2, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/altendky/.local/share/virtualenv)
    added seed packages: pip==20.3.1, pip==20.3.4, setuptools==44.1.1, wheel==0.36.1, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator

$ venv2/bin/pip install --upgrade pip setuptools wheel
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Requirement already up-to-date: pip in ./venv2/lib/python2.7/site-packages (20.3.4)
Requirement already up-to-date: setuptools in ./venv2/lib/python2.7/site-packages (44.1.1)
Requirement already up-to-date: wheel in ./venv2/lib/python2.7/site-packages (0.36.2)

$ venv2/bin/pip install decorator
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting decorator
  Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Installing collected packages: decorator
Successfully installed decorator-4.4.2

[vphilippon]

According to my test (and the pypi.org page) this worked :) . It seems you're happily out of the Py2 realm!
Thanks for taking the time to fix this.

[micheles]

So much has changed in the Python packaging ecosystem and since I do a release per year I was still following the old ways.
Thanks for the help to everybody involved, I am closing the issue.