[meta] Better interpreter discovery

[obestwalter]

Why?

One of the themes that emerges when going through the issues is a need for better interpreter discovery. There is an ever growing range of ways how and where Python interpreters can be installed which could be addressed in a way that "Testing out of the box" works for a wide range of these scenarios on all the major Operating Systems. IMO tox should be as clever as possible in finding interpreters and grab whatever it can (but be explicit about it, so the user knows what's going on).

The reason why this is high up on my list and why I hope that this will end up in tox core, is because it lowers the hurdle for new users when things just work in most of the cases.

How?

In the sprint we started thinking in that general direction. Please see draft and @nicoddemus notes. This can be a starting point.

As @RonnyPfannschmidt and @offbyone suggested this can be experimented with in a plugin. This should also build on the already existing plugins that accomplish better discovery for specific use cases.

This meta issue shall collect all issues dealing with suboptimal interpreter discovery and collect already existing plugins that enhance the ability of tox to discover interpreters.

Feedback and references to things I overlooked will be highly appreciated. Also if you think this is a bad idea let me know - I am used to having bad ideas :)

Things to consider

• Special distributions, tools, installation locations like conda, pyenv, MacPorts (please tell me more if there are others that need to be considered)
• Linux distributions with different default python (e.g. arch linux)

Related issues

• Support conda envs when using Miniconda #273
• PyPy3 5.2 #345
• Unable to find python on path (~/bin) #166
• InvocationError: could not find executable #278
• Consider supporting 'py2' and 'py3' default environments #348
• Provide a host-based method of finding interpreters. #217
• Support MacPorts-installed PyPy out of the box  #40

Related plugins

• https://pypi.org/pypi/tox-pyenv/1.0.3
• https://pypi.org/pypi/tox-globinterpreter/0.3

[The-Compiler]

FWIW this is definitely a pain point for me - I need a Python 3, and on Archlinux most linux distributions /usr/bin/python is python 2, so I need to do basepython = python3. However, that doesn't work on Windows where it's something like C:\Python34\python.exe (and something... more complicated for Python 3.5+)

[Ivoz]

@The-Compiler you have a non-standard Archlinux then! Expand the file list for python (python 3) and you'll see it's the default /usr/bin/python

[The-Compiler]

@Ivoz err, sorry, that was a brain fart - of course Archlinux is the only distribution where stuff works correctly for me with basepython = python, not the other way around.

[snoack]

Would you consider #435 related?

[obestwalter]

@snoack as this is about finding interpreters and your proposal ist about marking interpreters as optional I don't really see a relation. Do you?

[offbyone]

I'd like to ensure that the way tox gets the interpreter is decoupled from how those interpreters are installed; my particular use case has the interpreters created & deployed in a somewhat nonstandard way, and I'd like to ensure that I can support that.

Additionally, FWIW, the same might apply to how dependencies are installed; pip is not (it turns out) the only way to get dependencies. As part of this work if the requirements resolution can be made pluggable I'd be thrilled beyond words.

[obestwalter]

Hi @offbyone,

I'd like to ensure that the way tox gets the interpreter is decoupled from how those interpreters are installed; my particular use case has the interpreters created & deployed in a somewhat nonstandard way, and I'd like to ensure that I can support that.

Can you give some more details. Would be great to collect use cases here to keep in mind when working on it.

Additionally, FWIW, the same might apply to how dependencies are installed; pip is not (it turns out) the only way to get dependencies. As part of this work if the requirements resolution can be made pluggable I'd be thrilled beyond words.

Could you give a bit more detail? Maybe we have an issue for that already.

[RonnyPfannschmidt]

this may also play into deployments/ways where we might want to make tox ask homebrew/nix/nixos/pyenv/conda/anaconda for the interpreters and/or have the plugin for that suggest to the user how to install the interpreters

[obestwalter]

Do we have a naming concept of the distinct phases in tox runs already? I am starting to want to say that x happens in phase y but haven't seen anything in the docs ... something like:

1. initialization
2. interpreter discovery
3. environment creation
4. dependency installation
5. command execution

Phase 1 happens exactly once per tox invocation and all others happen as often as the number of environments being run.

Phases 3 and 4 run the biggest danger to be muddled together and are definitely interdependent, but should still be seen and handled as different phases.

Does that fit? Did I miss something?

Sorry if this is off topic but your comments lead me to start thinking about this, and as this is a meta issue I just throw it in here.

[offbyone]

That's a pretty good breakdown of the phases that are interesting to me. Doing it in those phases will give opportunities to customize tox the way I need to.

[obestwalter]

Just realized though that I somehow missed the central point - the installation of the package under test 😊 - which is another chance to get things muddled and TBH I am not quite sure about how, when and why that works, because this is different e.g. in usedevelop and sdist scenarios.

So these would be the updated phases:

initialization
interpreter discovery
environment creation
dependency installation
package under test installation
command execution

[huntcsg]

I think it might be good to specify that "dependency installation" is installation of the necessary software to support the commands being run. The package itself may then also do it's own dependency installation.

But, for instance, I have seen projects that do command dependency installation via python extras_require sections

commands:
    pip install mypackage[testing]  # <--- Installs pytest, mock, etc.
    py.test
    ...

Or, for instance, if something like cython is needed as an installation dependency, that would go in the dependency installation phase (since it is too late if you are in the package under test installation phase)

I just wanted to clarify that the dependencies that are being installed during dependency installation are a slightly different class of dependencies than are installed during the package under test installation and possibly the command execution phase.

[obestwalter]

Thanks @huntcsg - I think this whole phases thing is worthy of some further discussion and might end up at least in the docs, so I will open a new Issue for this topic.

[RonnyPfannschmidt]

@huntcsg note that these days you can use extras = testing instead of the extras install of the package

[jvanasco]

I'm not sure if this should be a different ticket or not, but it's in the same realm of this ticket and the linked ones...

I have a few projects that run on Python2.7. I had run into Unicode issues because of the how Python2.x-3.3 stored unicode character data -- it is stored in UCS-2 and UCS-4 formats. Many linux distributions and osx version shipped with a 'narrow' build of unicode support (UCS-2), and will break on trying to decode a lot of modern characters (for example, emojis). The UCS is a compile-time option, so I need to build separate Python binaries for a few Python versions and run those tests twice -- once on a wide binary and once on a narrow one. The easiest way I've been able to do this so far, is to run tox twice on two different machines and then aggregate the tests. I'd love to be able to do this within a single tox run.

[obestwalter]

@jvanasco I think you can leave that here ... thanks. I really hope to be able to work on this over the summer.

[AvdN]

@jvanasco I ran into the narrow / wide problem after accepting a PR for ruamel.yaml. In the process of fixing that, I realized I have always only been testing ucs-4 and never ucs-2 when running tox. My answer on stack-overflow gives a description on how I solved testing wide py27 (i.e. py27mu) and narrow py27m on the same box with a single tox run.

[gaborbernat]

In theory you could have python2.7 and python2.7-wide available. At which point to use this can be configured via the config syntax. That being said now tox interpreter discovery is customizable via a plugin so for more exotic use cases I recommend creating a plugin. once the logic there is proven we can merge it into core. In the meantime this will be a won't fix at core level.