Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move test dependencies to package metadata. #1981

Merged
merged 8 commits into from Feb 4, 2020
Merged

Conversation

jaraco
Copy link
Member

@jaraco jaraco commented Jan 20, 2020

Summary of changes

By declaring the dependencies in metadata, they become inspectable by other tools at runtime and are declared in the canonical location (currently setup.cfg) rather than in a bespoke location.

Furthermore, by removing the ./tests directory, it avoids masking the issue reported in #1896.

Pull Request Checklist

  • Changes have tests
  • News fragment added in changelog.d. See documentation for details

@jaraco
Copy link
Member Author

@jaraco jaraco commented Jan 20, 2020

Unfortunately, this change causes the package to fail to install into tox on Python 2.7.

setuptools feature/deps-in-metadata $ tox -r -e py27                                                                                                                           
py27 recreate: /Users/jaraco/code/public/pypa/setuptools/.tox/py27
py27: pip_version is pip
py27 develop-inst: /Users/jaraco/code/public/pypa/setuptools
ERROR: invocation failed (exit code 1), logfile: /Users/jaraco/code/public/pypa/setuptools/.tox/py27/log/py27-2.log
================================================================================== log start ==================================================================================
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Uninstalling setuptools-41.4.0:
  Successfully uninstalled setuptools-41.4.0
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Obtaining file:///Users/jaraco/code/public/pypa/setuptools
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting pytest-flake8; extra == "tests"
  Using cached https://files.pythonhosted.org/packages/4b/99/a6e993c0927665522602058e1f2ea61ba1c8c51a60e3006f1eb1153b37e2/pytest_flake8-1.0.4-py2.py3-none-any.whl
Collecting virtualenv>=13.0.0; extra == "tests"
  Using cached https://files.pythonhosted.org/packages/05/f1/2e07e8ca50e047b9cc9ad56cf4291f4e041fa73207d000a095fe478abf84/virtualenv-16.7.9-py2.py3-none-any.whl
Collecting mock; extra == "tests"
  Using cached https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl
Collecting futures; python_version == "2.7" and extra == "tests"
  Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Requirement already satisfied: wheel; extra == "tests" in ./.tox/py27/lib/python2.7/site-packages (from setuptools===45.1.0.post20200120.post-20200120) (0.33.6)
Collecting coverage>=4.5.1; extra == "tests"
  Using cached https://files.pythonhosted.org/packages/b9/33/7f80c0835714386022ebd29a747e7d94008ddeeec4e2b937ff53b4c4352f/coverage-5.0.3-cp27-cp27m-macosx_10_13_intel.whl
Collecting pytest>=3.7; extra == "tests"
  Using cached https://files.pythonhosted.org/packages/eb/3d/19bac795b6e60f4d6127227d4e11c31f8d9555de5e1f1a5e01ebad1d2ccf/pytest-5.3.3.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting pytest-virtualenv>=1.2.7; extra == "tests"
  Using cached https://files.pythonhosted.org/packages/21/7f/0ec1ab9b8fcd4a3a063b97143a97d1c093d01227353b682c9027d14b14c2/pytest_virtualenv-1.7.0-py2.py3-none-any.whl
Requirement already satisfied: pip>=19.1; extra == "tests" in ./.tox/py27/lib/python2.7/site-packages (from setuptools===45.1.0.post20200120.post-20200120) (19.3.1)
Collecting pytest-cov>=2.5.1; extra == "tests"
  Using cached https://files.pythonhosted.org/packages/b9/54/3673ee8be482f81527678ac894276223b9814bb7262e4f730469bb7bf70e/pytest_cov-2.8.1-py2.py3-none-any.whl
Collecting flake8>=3.5
  Using cached https://files.pythonhosted.org/packages/f8/1f/7ea40d1e4146ea55dbab41cda1376db092a75794914169aabd7e8d7a7def/flake8-3.7.9-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3"
  Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting six
  Using cached https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Collecting packaging
  Using cached https://files.pythonhosted.org/packages/d8/5b/3098db49a61ccc8583ffead6aedc226f08ff56dc03106b6ec54451e27a30/packaging-20.0-py2.py3-none-any.whl
Collecting pathlib2>=2.2.0; python_version < "3.6"
  Using cached https://files.pythonhosted.org/packages/e9/45/9c82d3666af4ef9f221cbb954e1d77ddbb513faf552aea6df5f37f1a4859/pathlib2-2.3.5-py2.py3-none-any.whl
Collecting importlib-metadata>=0.12; python_version < "3.8"
  Using cached https://files.pythonhosted.org/packages/d7/31/74dcb59a601b95fce3b0334e8fc9db758f78e43075f22aeb3677dfb19f4c/importlib_metadata-1.4.0-py2.py3-none-any.whl
Collecting attrs>=17.4.0
  Using cached https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl
Collecting more-itertools>=4.0.0
  Using cached https://files.pythonhosted.org/packages/df/8c/c278395367a46c00d28036143fdc6583db8f98622b83875403f16473509b/more-itertools-8.1.0.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /Users/jaraco/code/public/pypa/setuptools/.tox/py27/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-75eq4e/more-itertools/setup.py'"'"'; __file__='"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-75eq4e/more-itertools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-75eq4e/more-itertools/pip-egg-info
         cwd: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-75eq4e/more-itertools/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ImportError: No module named setuptools
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Removing setuptools
Running bootstrap
Traceback (most recent call last):
  File "tools/tox_pip.py", line 55, in <module>
    pip(sys.argv[1:])
  File "tools/tox_pip.py", line 51, in pip
    subprocess.check_call(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/jaraco/code/public/pypa/setuptools/.tox/py27/bin/python', '-m', 'pip', 'install', '--exists-action', 'w', '-e', '/Users/jaraco/code/public/pypa/setuptools[tests]']' returned non-zero exit status 1

=================================================================================== log end ===================================================================================
___________________________________________________________________________________ summary ___________________________________________________________________________________
ERROR:   py27: InvocationError for command /Users/jaraco/code/public/pypa/setuptools/.tox/py27/bin/python tools/tox_pip.py install --exists-action w -e '/Users/jaraco/code/public/pypa/setuptools[tests]' (exited with code 1)

@jaraco
Copy link
Member Author

@jaraco jaraco commented Jan 20, 2020

It seems that although the tests are running on Python 2.7, pip is installing pytest 5.3 and more-itertools 8, both of which are Python 3-only. But why?

@jaraco
Copy link
Member Author

@jaraco jaraco commented Jan 20, 2020

Oh, I know why. Now that deps and . are installed during the same step, the workaround to selectively honor Requires-Python is no longer suitable.

…es in metadata to be installed without ignoring REQUIRES_PYTHON
@jaraco jaraco force-pushed the feature/deps-in-metadata branch from 7a31a9d to d92f778 Compare Jan 20, 2020
@jaraco
Copy link
Member Author

@jaraco jaraco commented Jan 20, 2020

Although this latest implementation now runs successfully on Python 2.7, it still has issues even on Python 3, when setuptools isn't present when installing some dependencies that are only available as source. I was able to replicate the issue locally by running:

env PIP_NO_CACHE_DIR=1 tox -r

Error is:

Collecting termcolor
  Downloading https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /Users/jaraco/code/public/pypa/setuptools/.tox/python/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-d_3p0c1o/termcolor/setup.py'"'"'; __file__='"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-d_3p0c1o/termcolor/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-d_3p0c1o/termcolor/pip-egg-info
         cwd: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-d_3p0c1o/termcolor/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'

@jaraco
Copy link
Member Author

@jaraco jaraco commented Jan 20, 2020

In 2e00ba3, I've taken a less aggressive approach and simply moved the dependencies into something other than ./tests to avoid masking the error reported in #1896. I'm going to defer this effort until it's less messy to do so.

@jaraco jaraco force-pushed the feature/deps-in-metadata branch from e3a7874 to 620bd62 Compare Jan 22, 2020
@jaraco jaraco force-pushed the feature/deps-in-metadata branch from 4f74375 to b2845ed Compare Jan 22, 2020
…s jaraco/skeleton to build docs in tox and rtd.
@jaraco jaraco force-pushed the feature/deps-in-metadata branch from 5871f6d to 641e088 Compare Jan 22, 2020
@webknjaz

This comment has been minimized.

@jaraco jaraco removed the deferred label Jan 22, 2020
@jaraco
Copy link
Member Author

@jaraco jaraco commented Jan 22, 2020

I believe this change is basically ready to go. I'm a little sad the implementation adds so many lines of code, but it does have some benefits:

  • requirements are consolidated into one location, reflected in metadata
  • tox logic is simpler
  • the special considerations for Python 2 and self install are captured and self-documenting
  • follows same pattern as other projects I maintain

Does this change merit a changelog entry? Probably.

@jaraco jaraco merged commit 54b76e6 into master Feb 4, 2020
6 of 10 checks passed
@jaraco jaraco mentioned this pull request Feb 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants