pipfile support ?

[dmerejkowsky]

So, I've stumbled upon the pipfile project and I'm thinking supporting pipfile inside tox would be a good idea.

Context

• My code uses ruamel.yaml and path.py.
• I'm using tox to run the tests with py.test and do static analysis with pylint.
• I already have a setup.py looking like:

setup(
   name="foo",
   install_requires=[
        "path.py",
        "ruamel.yaml",
   ]
)

• Calling python setup.py takes a long time, so I'm using skipsdist=True

Caveats

I'm pretty new to tox (been using invoke for some time before), so maybe there's something obvious I'm missing. In this case, feel free to correct me :)

The problem

Bot py.test and pylint needs all the dependencies installed to run properly (pylint will give an error if it can't import path.py for instance).

A naive tox.ini would look like:

[testenv:lint]
deps =
    path.py==0.42
    ruamel.yaml=1.2.3
    pylint
commands =
    pylint foo

[testenv:test]
deps =
    path.py==0.42
    ruamel.yaml=1.2.3
    pytest
    pytest-cov
commands =
    pytest foo --cov . --cov-report=term

You notice there is some code duplication between the lint and the test environments.

First solution

To fix this, we could use string interpolation in the config file:

[DEFAULT]
common_deps=
    path.py==0.42
    ruamel.yaml=1.2.3

[testenv:lint]
deps =
    %(common_deps)s
    pylint
# ...

[testenv:test]
deps =
    %(common_deps)s
    pytest
    pytest-cov
# ...

Unfortunately, tox uses py.iniconfig and does not support this feature.

Second solution

We could also use a requirements.txt with the frozen version inside, like so:

# requirements.txt
path.py==0.42
ruamel.yaml=1.2.3

[testenv:lint]
deps =
    -rrequirements.txt
    pylint
# ...

[testenv:test]
deps =
    -rrequirements.txt
    pytest
    pytest-cov
# ...

This works, but if we have a new dependency, we have to:

• patch setup.py
• Re-create a virtualenv and run pip freeze
• Update requirements.txt
• Re-create all the tox virtual environments.

The same problem occurs if we bump path.py to 0.43.
tox has no way to know the dependencies have changed, and we have to re-create all the virtualenvs again.

Third solution

The great thing about pipfile is that we get a pipfile.lock in json which is very easy to parse, and always contain frozen versions numbers.

So tox could simply read it:

# Pipfile
source('https://pypi.org/', verify_ssl=True)

package('path.py')
package('ruamel.yaml')
dev_package('pytest')
dev_package('pytest-cov')
dev_package('pylint')

// Pipfile.lock
    "default": [
        {"name": "path.py", "version": "0.42"},
        {"name": "ruamel.yaml", "version": "1.2.3"}
    ],
    "development": [
        {"name": "pytest", "version": "1.3.7"},
        {"name": "pytest-cov", "version": "3.4.5"},
        {"name": "pylint", "version": "1.6.4"}
    ]

# tox.ini

[testenv:lint]
deps=
  Pipfile:development
commands =
    pylint foo

[testenv:test]
deps=
  Pipfile:development
commands =
    pytest foo --cov . --cov-report=term

Proposed pull requests

So, there are many things we could do here:

• Allow string interpolation in tox.ini, either by:

  • Using configparser instead of py.iniconfig
  • Or patch py.iniconfig to add support for string interpolation

• Improve support of requirements.txt files in tox so that frozen versions are taken into account

• Implement pipfile support

In my humble opinion, working on pipfile is the best way to go, but it's not for me to decide.

What do you think?

[obestwalter]

Interesting. Never heard of pipfile. My first reflex without looking closely is the infamous xkcd about standards, but I will definitely have a look if this is something that one needs to keep an eye on :)

Diclaimer I am quite new to the project although I have a maintainer hat on, so don't expect too much in depth knowledge from me yet. At the moment I am wading through all unlabelled issues and try to get a feel for what is needed and where things could be moving.

So here are just some random thoughts to maybe help you assess a bit more in which direction this could move:

Backwards compatibility is very important and there are a lot tox.inis out there already, so whatever we do, we must be very careful not to break existing test setups. This makes changes to the way tox.ini is parsed the most dangerous route.

Tox is plugin driven to extend functionality for special use cases so maybe there is a way to provide what you want as a plugin.

[speedyleion]

For option 1, does https://tox.readthedocs.io/en/latest/config.html#substitution-for-values-from-other-sections not work?

[RonnyPfannschmidt]

pipfile is quite distinct from the use-case for tox - as far as i can tell if tox had better requirements.txt support, then pipfile could argument it with the tool to generate the pinned requirements.txt

[dmerejkowsky]

@speedyleion I somehow missed that, sorry.

I ended up using something like:

[base]
deps =
    colorama==0.3.7
    path.py==9.0
    ruamel.yaml==0.13.2

[lint]
deps =
    {[base]deps}
    pylint==1.6.4
    astroid==1.4.9

[testenv:lint]
    deps = {[lint]|deps}
    commands = pylint foo

Seems to work well so far ;)

[obestwalter]

Proposing to close this as not necessary/applicable if we solve the actual problems with dependency handling.

[bittner]

I believe it makes sense to refresh the discussion of Pipfile aka Pipenv.

@kennethreitz The Pipenv docs say that Pipenv is meant to be a deployment tool. Does that mean this contradicts with integrating Pipenv with Tox?

Ideally, you should only have one target Python version, as this is a deployment tool.

In my opinion it would make sense to integrate Pipfile/Pipenv with Tox. Pipenv's role related to deployment needs to be clarified for that, though. Especially since testing plays an important role in the concept of deployment pipelines.

See Also

• tox-pipenv - plugin for Tox
• Clarify core concepts for tox-pipenv usage tox-pipenv#39
• Use --dev flag in install tox-pipenv#2 (comment)

[obestwalter]

Hi @bittner, thanks for the overview - leaving this open for further discussion then.

[OrangeDog]

In version 2018.05.18 pipenv will detect if it's running in a virtual environment and not create a new one. You can therefore configure tox thus:

whitelist_externals = pipenv
install_command = pipenv update {opts} {packages}
deps = --dev

All the dependencies have to be defined in the Pipfile. Adding anything to deps will give this error:

Warning: --system is intended to be used for Pipfile installation, not installation of specific packages. Aborting.
See also: --deploy flag.

For older versions, I think the --system flag will have the same effect.

There's another suggestion of how to do it here: https://docs.pipenv.org/advanced/#tox-automation-project

[OrangeDog]

For more direct Pipfile support for tox (or a plugin), requirementslib may be useful.

[jleclanche]

Oh I just noticed there's an open issue here for this. I've been trying to discuss it in tox-dev/tox-pipenv#37.

The problem with the approach mentioned by @OrangeDog is that it's impossible to define multiple different versions of requirements. I think the final solution needs to:

• Support separate pipfiles for separate environments
• Read pipfiles just like it currently reads requirements.txt files (Meaning, other dependencies can still be specified)
• Support pipfile.lock as well

So as I was saying there, my approach looks something like this:

deps =
	-p{toxinidir}/Pipfile  # Or replace -p 
	pytest
	pytest-mock

To specify the pipfile environment, a few choices:

• pipfile_use_develop = True (a separate variable)
• Or maybe: pipfile_environment = develop
• Or maybe even -p{toxinidir}/Pipfile --env=develop or something similar, which would support multiple pipfiles per deps definition

I'd really love to get this natively in tox. Ideally, it'd even be native to pip, but I don't think there's much hope of seeing this any time soon (especially if Pipfiles aren't in their final format).

[OrangeDog]

Support separate pipfiles for separate environments

A single Pipfile already supports multiple environments. Any solution that involves multiple Pipfiles must be wrong.

[butla]

@OrangeDog Can you give a link to some docs or examples? The spec here shows dependency on only one version of Python.

[OrangeDog]

@butla You can have arbitrary [x-packages] sections (only packages and dev-packages are currently used by pipenv).
To support different Python versions, you use PEP 508.

[gaborbernat]

No support is planned at the moment. If anyone wants can pick it up as a plugin outside of the project.