Add a tox.ini

[JelleZijlstra]

Can be improved, but wanted to get this in to make it easier to test against the mypy test suite.

We should add the "mypytests" target to Travis.

[gvanrossum]

I've never used tox -- isn't there a way to do this without it? It seems weirdly overkill somehow, and yet another place where we'd be maintaining the same dependency info.

[ambv]

@JelleZijlstra, regardless of how we proceed, move the contents of the mypytests testenv to a separate Bash script (say, tests/mypy_selftest.sh), so it's easier to include it in both runtests.sh and .travis.yml. Also, change requirements-tests-py3.txt to include master instead of a release, for consistency with what we're doing in .travis.yml. That will be uncontroversial and can be merged right away.

@gvanrossum, Tox is a pretty popular and mature tool. In its essence it's just a replacement for our primitive runtests.sh script. The important improvement over our shell script is that all testenvs listed in tox.ini create their own isolated virtualenvs. This increases reproducibility (no more "it worked on my laptop!").

It's also rather trivial to use:

$ pip install tox
$ tox

If you're okay with us moving to it, we should remove the runtests.sh script and update the README.md. Note how the entire section on "Running the tests" will now just become the commands I listed above. Later in the future we can even consolidate .travis.yml to depend on our Tox configuration instead.

[gvanrossum]

No, tox is not "trivial to use". It's a black box that encourages cargo-cult coding. I know exactly what that shell script does. I have no idea how the different clauses of the tox.ini file relate to each other, and I don't have a desire to learn about it.

[ambv]

I don't have a desire to learn about it.

Fair enough. @JelleZijlstra, let's do that tests/mypy_selftest.sh part instead.

[JelleZijlstra]

Sounds good, I'll close this PR and submit a separate one adding mypy_selftest.sh.