Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use python-version-specific requirements.txt instead of single generic requirements.txt #150

Merged
merged 10 commits into from
Dec 1, 2020

Commits on Nov 30, 2020

  1. Add requirements/ dir

    ... containing all top-level (*.in) and concrete requirements.txt, the
    latter parametrized by python version.
    
    We have a bunch of dependencies that are only needed on specific python
    versions (e.g. backports like dataclasses, importlib_resources, etc).
    We use conditional environment markers for those in the top-level
    requirements.in files.
    pip-compile does not yet support generating a combined requirements.txt
    file where differences across python versions/platforms/archs are
    reconciled using environmet markers:
    jazzband/pip-tools#826
    
    They currently recommend running pip-compile on each targeted python
    environment generating as many concrete requirements.txt files
    jazzband/pip-tools#651
    
    So this is what I have done here.
    anthrotype committed Nov 30, 2020
    Configuration menu
    Copy the full SHA
    958a2cb View commit details
    Browse the repository at this point in the history
  2. setup.py: read install_requires and extras_require from external requ…

    …irements.in files
    
    pip-compile setup.py doesn't support environment markers ('; python_version < 3.7'), whereas it does when compiling from requirements.in files.
    So we read top-level dependencies from external *.in files and stick
    them in the setup.py install_requires and extras_require keywords.
    
    The 'dev' extra is new. One can now do `pip install -e .[dev]` and have
    all the development requirements installed alongside the current module.
    This is handy for developers bootstrapping a new venv to work on.
    It's a common practice among python developers.
    anthrotype committed Nov 30, 2020
    Configuration menu
    Copy the full SHA
    a972b7d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    96f9329 View commit details
    Browse the repository at this point in the history
  4. add tox command to update all py*-requirements.txt files

    This is handy when any top-level deps change. The shell script is just a shorthand for the longish tox command
    anthrotype committed Nov 30, 2020
    Configuration menu
    Copy the full SHA
    dbc6dab View commit details
    Browse the repository at this point in the history
  5. tox.ini: usedevelop=true if tox env name contains '-dev' particle

    tox -e py39-dev will install in editable mode (for when one is in a rush).
    One can also use this as a way to quickly bootstrap a development virtual environment.
    E.g. after running the above command, a new venv is created in .tox/py39-dev with all
    the pinned requirements installed and the current module installed in editable mode,
    ready for editing source files and running directly the pytest command therein
    anthrotype committed Nov 30, 2020
    Configuration menu
    Copy the full SHA
    989c1ee View commit details
    Browse the repository at this point in the history
  6. ci: export TOXENV variable to select current GH python

    The generic invocation 'tox -e py' does not work any more, because we now install dependencies using python-version-specific requirements.txt files and we need to know the exact python major.minor version.
    Thus we set TOXENV environment variable such that invoking 'tox'
    without additional -e options will run only the specified python
    version.
    anthrotype committed Nov 30, 2020
    Configuration menu
    Copy the full SHA
    79ffc42 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2020

  1. read supported pythons from tox.ini envlist

    Using tox -l command to list them, so we have a single source of truth for what are the versions that we support. And we only need to change one file when, e.g., python3.10 is out.
    anthrotype committed Dec 1, 2020
    Configuration menu
    Copy the full SHA
    f0f97c3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7ddf048 View commit details
    Browse the repository at this point in the history
  3. expect script to be run from root dir and use relative paths throughout

    otherwise pip-compile will write full absolute paths in the generated requirements.txt..
    anthrotype committed Dec 1, 2020
    Configuration menu
    Copy the full SHA
    170a772 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    85e1a69 View commit details
    Browse the repository at this point in the history