Prequ

Tools for Python requirement handling. Helps in keeping your requirements files complete and up-to-date.

Note: Prequ is currently designed to work with a virtual env, so compatibility with non-virtual Python environments are not guaranteed to work at the moment.

Background

Every non-library Python project should have a requirements.txt file which lists required Python packages for the project, i.e. its dependencies. It would be easy to just list the dependencies with their minimum and maximum versions in there, but that's not a good practice. If versions of the dependencies are not pinned to exact versions, it's uncertain which version of the packages get installed. Even pinning the direct dependencies is not enough, since project dependencies might have their own dependencies (project's indirect dependencies) and those should be pinned too. That's where Prequ comes in: it makes it easy to generate the list of those pinned direct and indirect dependencies from the non-pinned requirements.

There is also a good article by Vincent Driessen which explains it more thoroughly why you should pin your packages.

Prequ is a fork of pip-tools by Vincent Driessen. Pip-tools was a fine project, but I wanted to add couple new features and make some changes to existing workflows. There were also couple bugs that I needed to be fixed sooner than later. Most of those bugs were already fixed in GitHub pull requests, but weren't merged to pip-tools. That's why I decided to create my own fork.

Installation

$ pip install prequ

Example usage for prequ update

Suppose you have a Flask project, and want to pin it for production. You need to specify a configuration file for Prequ. The configuration file minimally defines so-called source requirements, i.e. list of Python packages (with optional version specifiers). This can be done by writing following section to setup.cfg:

[prequ]
requirements = Flask

Now, run prequ update:

$ prequ update
*** Compiling requirements.txt

And it will produce your requirements.txt, with all the Flask dependencies and all underlying dependencies pinned. Put this file under version control as well. Generated file will look like this:

# This file is autogenerated by Prequ.  To update, run:
#
#   prequ update
#
flask==0.10.1
itsdangerous==0.24
jinja2==2.7.3
markupsafe==0.23
werkzeug==0.10.4

To add/remove packages, add/remove them to/from setup.cfg and re-run prequ update. To upgrade all packages, remove the generated requirements.txt and run prequ update again.

Example usage for prequ sync

Now that you have a requirements.txt, you can use prequ sync to update your virtual env to reflect exactly what's in there. Note: this will install/upgrade/uninstall everything necessary to match the requirements.txt contents.

$ prequ sync
Uninstalling flake8-2.4.1:
  Successfully uninstalled flake8-2.4.1
Collecting click==4.1
  Downloading click-4.1-py2.py3-none-any.whl (62kB)
  ...
  Found existing installation: click 4.0
    Uninstalling click-4.0:
      Successfully uninstalled click-4.0
Successfully installed click-4.1

To sync multiple *.txt dependency lists, just pass them in via command line arguments e.g.:

$ prequ sync requirements.txt requirements-dev.txt

Passing in empty arguments would cause it to default to requirements.txt.

More detailed example of Prequ configuration

Prequ supports defining couple options for the requirement compiling and automatically building wheels from pip URLs. Here is a more detailed example of a Prequ configuration to demonstrate those features:

[prequ]
annotate = yes
generate_hashes = no
header = yes
extra_index_urls =
    https://shuup.github.io/pypi/simple/
wheel_dir = wheels
wheel_sources =
    github_shuup = git+ssh://git@github.com/shuup/{pkg}@v{ver}

requirements =
    django~=1.9.5
    shuup~=0.5.0
    shuup-stripe==0.4.2 (wheel from github_shuup)

requirements-dev =
    flake8
    pep8-naming

Now running prequ update will first build a wheel package for shuup-stripe and then it will generate two files, requirements.txt and requirements-dev.txt:

$ prequ update
*** Building wheel for shuup-stripe 0.4.2 from
        git+ssh://git@github.com/shuup/shuup-stripe@v0.4.2
Collecting git+ssh://git@github.com/shuup/shuup-stripe@v0.4.2
...
Successfully built shuup-stripe
Cleaning up...
Removing source in /tmp/pip-b5rf3ioq-build
*** Built: wheels/shuup_stripe-0.4.2-py2.py3-none-any.whl
*** Compiling requirements.txt
*** Compiling requirements-dev.txt

The generated files will have extra-index-url option as specified and and find-links for the wheels directory:

$ cat requirements.txt
# This file is autogenerated by Prequ.  To update, run:
#
#   prequ update
#
--extra-index-url https://shuup.github.io/pypi/simple/
--find-links wheels

Babel==2.3.4              # via shuup
django-bootstrap3==6.2.2  # via shuup
...
$ cat requirements-dev.txt
# This file is autogenerated by Prequ.  To update, run:
#
#   prequ update
#
--extra-index-url https://shuup.github.io/pypi/simple/
--find-links wheels

flake8==3.3.0
mccabe==0.6.1             # via flake8
pep8-naming==0.4.1
pycodestyle==2.3.1        # via flake8
pyflakes==1.5.0           # via flake8