New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No longer hard-code pythons #914

Merged
merged 1 commit into from Jul 19, 2018

Conversation

Projects
None yet
4 participants
@asottile
Member

asottile commented Jul 17, 2018

WIP -- I'll fill out changelog and such if it works and if it seems like a good idea. It always struck me as odd that tox hardcoded this list of interpreters and that I'd be forced to upgrade my tox installation to use new things. This patch adds support for python2.0 - python9.9, hopefully leaving us some headroom.

Here's hoping python3.10 isn't coming out soon, eh?

If this is good, the lists of interpreters can be deprecated and eventually removed I would imagine

@asottile

This comment has been minimized.

Member

asottile commented Jul 17, 2018

for instance:

$ tox -e py28 --notest
GLOB sdist-make: /tmp/tox/setup.py
py28 create: /tmp/tox/.tox/py28
SKIPPED: InterpreterNotFound: python2.8
___________________________________ summary ____________________________________
SKIPPED:  py28: InterpreterNotFound: python2.8
  congratulations :)
$ tox -e py99 --notest
GLOB sdist-make: /tmp/tox/setup.py
py99 create: /tmp/tox/.tox/py99
SKIPPED: InterpreterNotFound: python9.9
___________________________________ summary ____________________________________
SKIPPED:  py99: InterpreterNotFound: python9.9
  congratulations :)
$ tox -e py6 --notest
GLOB sdist-make: /tmp/tox/setup.py
py6 create: /tmp/tox/.tox/py6
SKIPPED: InterpreterNotFound: python6
___________________________________ summary ____________________________________
SKIPPED:  py6: InterpreterNotFound: python6
  congratulations :)
@obestwalter

This comment has been minimized.

Member

obestwalter commented Jul 17, 2018

Good idea :)

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Jul 17, 2018

Why not add support for (\d).(\d){2,}?

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Jul 17, 2018

Note the VSTS CI failed with valid failures.

@gaborbernat gaborbernat force-pushed the tox-dev:master branch from e80f69e to bb4a94c Jul 17, 2018

@asottile

This comment has been minimized.

Member

asottile commented Jul 17, 2018

Why not add support for (\d).(\d){2,}?

because it doesn't already exist -- wanted to replicate existing behaviour as much as possible

The failure as far as I can tell isn't actually an issue since python falls to sys.executable anyway (just not at that portion in the code) but I'll fix before shipping

@asottile

This comment has been minimized.

Member

asottile commented Jul 17, 2018

@gaborbernat did you force push master? my history no longer aligns

@asottile asottile force-pushed the asottile:unhardcode_pythons branch from a19d603 to c6a22f4 Jul 17, 2018

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Jul 17, 2018

Yes sadly by mistake, and had to rebuild master 👎 if you want to force push back and cherry pick the one new merge you're free 👍 made the master protected.

@asottile

This comment has been minimized.

Member

asottile commented Jul 17, 2018

ok, I'm going to reset to e80f69e and then re-apply cc3d3e6

@asottile

This comment has been minimized.

Member

asottile commented Jul 17, 2018

oh right, need to unprotect to force push myself. I've put the old master I'm about to force push over at _old-master (we can delete this branch once I'm done and we're happy)

@asottile asottile force-pushed the tox-dev:master branch from cc3d3e6 to 95f5476 Jul 17, 2018

@asottile

This comment has been minimized.

Member

asottile commented Jul 17, 2018

@gaborbernat can you re setup the branch protection (if master looks good) -- I've forgotten which checks were required 🤦‍♂️

@asottile asottile force-pushed the asottile:unhardcode_pythons branch from c6a22f4 to 3167435 Jul 17, 2018

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Jul 17, 2018

sure 🆗

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Jul 17, 2018

Done, your tests still fail at:

2018-07-17T14:35:36.9813333Z     def test_py_venv(self, newconfig):
2018-07-17T14:35:36.9829608Z         config = newconfig(["-epy"], "")
2018-07-17T14:35:36.9846066Z         env = config.envconfigs["py"]
2018-07-17T14:35:36.9861785Z >       assert str(env.basepython) == sys.executable
2018-07-17T14:35:36.9877452Z E       AssertionError: assert 'python' == '/opt/vsts/work/1/s/.tox/py/bin/python'
2018-07-17T14:35:36.9894093Z E         - python
2018-07-17T14:35:36.9909134Z E         + /opt/vsts/work/1/s/.tox/py/bin/python
2018-07-17T14:35:36.9917827Z 
2018-07-17T14:35:36.9939780Z config     = <tox.config.Config object at 0x7f84845f56d0>
2018-07-17T14:35:36.9955010Z env        = <tox.config.TestenvConfig instance at 0x7f84842a3d88>
2018-07-17T14:35:36.9970433Z newconfig  = <function create_new_config_file_ at 0x7f84841be578>
@asottile

This comment has been minimized.

Member

asottile commented Jul 17, 2018

yeah haven't "fixed" that yet -- just rebased :)

@asottile asottile force-pushed the asottile:unhardcode_pythons branch from 3167435 to 68fae99 Jul 17, 2018

@codecov

This comment has been minimized.

codecov bot commented Jul 17, 2018

Codecov Report

Merging #914 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #914      +/-   ##
==========================================
+ Coverage   92.61%   92.63%   +0.01%     
==========================================
  Files          12       12              
  Lines        2330     2335       +5     
  Branches      409      409              
==========================================
+ Hits         2158     2163       +5     
  Misses        109      109              
  Partials       63       63
Impacted Files Coverage Δ
src/tox/config.py 95.91% <100%> (+0.01%) ⬆️
src/tox/constants.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66bbfef...f837b1a. Read the comment docs.

@asottile asottile force-pushed the asottile:unhardcode_pythons branch from 68fae99 to 319e97b Jul 17, 2018

@gaborbernat

Is this ready then @asottile? In general, I agree with the approach although I think we might as well make the minor take two version numbers, not just one. I know 3.10 is not out yet, but neither is 3.9 and we support that via this construction. Will need a news fragment, otherwise looks great.

default = tox.PYTHON.DEFAULT_FACTORS[factor]
match = tox.PYTHON.PY_FACTORS_RE.match(factor)
if match:
default = tox.PYTHON.PY_FACTORS_MAP[match.group(1)] + ".".join(

This comment has been minimized.

@gaborbernat

gaborbernat Jul 18, 2018

Member

would prefer a format here 🤔

@asottile

This comment has been minimized.

Member

asottile commented Jul 18, 2018

According to http://www.curiousefficiency.org/posts/2014/08/python-4000.html 3.10 will never come -- so I'm going to leave this at two digits for now

Curious Efficiency
Newcomers to python-ideas occasionally make reference to the idea of "Python 4000" when proposing backwards incompatible changes that don't offer a clear migration path from currently legal Python 3 c
@obestwalter

This comment has been minimized.

Member

obestwalter commented Jul 18, 2018

My current expectation is that Python 4.0 will merely be "the release that comes after Python 3.9".

Sounds simple enough :)

@asottile asottile force-pushed the asottile:unhardcode_pythons branch from 319e97b to f837b1a Jul 18, 2018

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Jul 19, 2018

Taken from there what you linked:

Update: After this post was originally written back in 2014, subsequent discussions on the core python-dev mailing list led to the conclusion that the release after 3.9 will probably just be 3.10. However, a 4.0 will presumably still happen some day, and the premise of this article is expected to hold for that release: it will be held to the same backwards compatibility obligations as a Python 3.X to 3.X+1 update.

So there highly likely will be a 3.10.

@obestwalter

This comment has been minimized.

Member

obestwalter commented Jul 19, 2018

oh, ok then maybe support it right away after all, then we can't forget it in 2023 ...

@gaborbernat

I'm fine to merge this at it is. However just pointing out that py40 probably will come after py310.

@asottile

This comment has been minimized.

Member

asottile commented Jul 19, 2018

@gaborbernat is the "this branch is out of date" thing new? Really don't like it, it is very tedious.

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Jul 19, 2018

Fixed.

@gaborbernat gaborbernat merged commit 95b9e1c into tox-dev:master Jul 19, 2018

5 checks passed

VSTS: tox ci tox ci_20180718.05 succeeded
Details
WIP ready for review
Details
codecov/patch 100% of diff hit (target 92.61%)
Details
codecov/project 92.63% (+0.01%) compared to 66bbfef
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@asottile asottile deleted the asottile:unhardcode_pythons branch Jul 19, 2018

@asottile asottile referenced this pull request Jul 27, 2018

Closed

bug bad python version #927

@anlutro

This comment has been minimized.

anlutro commented Aug 3, 2018

Could we get a pypi release with this fix included? 3.1.2 is affected by the bug. I was super confused when -e py33 was working despite python3.3 not being installed.

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Aug 3, 2018

@anlutro are you sure you're referring to this? This PR allows specifying python 3.9 and so on. Has not much to do with -e py33.

@anlutro

This comment has been minimized.

anlutro commented Aug 3, 2018

Yes, I'm sure. If I revert the commit of this PR, tox -e py33 creates a virtualenv using python3.6.

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented Aug 3, 2018

Can you provide more details in an issue, please? Thanks!

@anlutro

This comment has been minimized.

anlutro commented Aug 3, 2018

Sure, if you insist, but the problem is already solved in master by this PR.

@anlutro

This comment has been minimized.

anlutro commented Aug 3, 2018

Opened issue here: #932

guykisel added a commit to guykisel/inline-plz that referenced this pull request Sep 24, 2018

Update tox to 3.4.0 (#380)
This PR updates [tox](https://pypi.org/project/tox) from **3.1.2** to **3.4.0**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.4.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this &lt;https://pip.pypa.io/en/latest/reference/pip/exists-action-option&gt;`_) - by :user:`gaborbernat` (`503 &lt;https://github.com/tox-dev/tox/issues/503&gt;`_)
- instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 &lt;https://github.com/tox-dev/tox/issues/908&gt;`_)
- PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 &lt;https://github.com/pypa/setuptools/pull/1481&gt;`_ - by :user:`gaborbernat` (`1003 &lt;https://github.com/tox-dev/tox/issues/1003&gt;`_)


Features
^^^^^^^^

- add ``commands_pre`` and ``commands_post`` that run before and after running
  the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all
  run until the first failing command)  - by :user:`gaborbernat` (`167 &lt;https://github.com/tox-dev/tox/issues/167&gt;`_)
- ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 &lt;https://github.com/tox-dev/tox/issues/814&gt;`_)
- use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``)  - by :user:`gaborbernat` (`824 &lt;https://github.com/tox-dev/tox/issues/824&gt;`_)
- always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 &lt;https://github.com/tox-dev/tox/issues/838&gt;`_)
- tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory,
  ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 &lt;https://github.com/tox-dev/tox/issues/847&gt;`_)
- While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 &lt;https://github.com/tox-dev/tox/issues/851&gt;`_)
- skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 &lt;https://github.com/tox-dev/tox/issues/903&gt;`_)
- keep additional environments config order when listing them - by :user:`gaborbernat` (`921 &lt;https://github.com/tox-dev/tox/issues/921&gt;`_)
- allow injecting config value inside the ini file dependent of the fact that we&#39;re connected to an interactive shell or not  - by :user:`gaborbernat` (`947 &lt;https://github.com/tox-dev/tox/issues/947&gt;`_)
- do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 &lt;https://github.com/tox-dev/tox/issues/974&gt;`_)
- when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 &lt;https://github.com/tox-dev/tox/issues/982&gt;`_)
- when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 &lt;https://github.com/tox-dev/tox/issues/994&gt;`_)
- ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 &lt;https://github.com/tox-dev/tox/issues/1004&gt;`_)


Documentation
^^^^^^^^^^^^^

- clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 &lt;https://github.com/tox-dev/tox/issues/305&gt;`_)
- add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 &lt;https://github.com/tox-dev/tox/issues/374&gt;`_)
- specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 &lt;https://github.com/tox-dev/tox/issues/545&gt;`_)
   ```
   
  
  
   ### 3.3.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 &lt;https://github.com/tox-dev/tox/issues/931&gt;`_)


Features
^^^^^^^^

- `PEP-517 &lt;https://www.python.org/dev/peps/pep-0517/&gt;`_ source distribution support (create a
  ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 &lt;https://github.com/tox-dev/tox/issues/573&gt;`_)
- `flit &lt;https://flit.readthedocs.io&gt;`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 &lt;https://github.com/tox-dev/tox/issues/820&gt;`_)
- packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 &lt;https://github.com/tox-dev/tox/issues/951&gt;`_)


Miscellaneous
^^^^^^^^^^^^^

- Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 &lt;https://github.com/tox-dev/tox/issues/955&gt;`_)
   ```
   
  
  
   ### 3.2.1
   ```
   ------------------

Bugfixes
^^^^^^^^

- ``--parallel--safe-build`` no longer cleans up its folders (``distdir``, ``distshare``, ``log``). - by :user:`gaborbernat` (`849 &lt;https://github.com/tox-dev/tox/issues/849&gt;`_)
   ```
   
  
  
   ### 3.2.0
   ```
   ------------------

Features
^^^^^^^^

- Switch pip invocations to use the module ``-m pip`` instead of direct invocation. This could help
  avoid some of the shebang limitations.  - by :user:`gaborbernat` (`935 &lt;https://github.com/tox-dev/tox/issues/935&gt;`_)
- Ability to specify package requirements for the tox run via the ``tox.ini`` (``tox`` section under key ``requires`` - PEP-508 style): can be used to specify both plugin requirements or build dependencies. - by :user:`gaborbernat` (`783 &lt;https://github.com/tox-dev/tox/issues/783&gt;`_)
- Allow to run multiple tox instances in parallel by providing the
  ``--parallel--safe-build`` flag. - by :user:`gaborbernat` (`849 &lt;https://github.com/tox-dev/tox/issues/849&gt;`_)
   ```
   
  
  
   ### 3.1.3
   ```
   ------------------

Bugfixes
^^^^^^^^

- A caching issue that caused the ``develop-inst-nodeps`` action, which
  reinstalls the package under test, to always run has been resolved. The
  ``develop-inst-noop`` action, which, as the name suggests, is a no-op, will now
  run unless there are changes to ``setup.py`` or ``setup.cfg`` files that have
  not been reflected - by stephenfin (`909 &lt;https://github.com/tox-dev/tox/issues/909&gt;`_)


Features
^^^^^^^^

- Python version testenvs are now automatically detected instead of comparing
  against a hard-coded list of supported versions.  This enables ``py38`` and
  eventually ``py39`` / ``py40`` / etc. to work without requiring an upgrade to
  ``tox``.  As such, the following public constants are now deprecated
  (and scheduled for removal in ``tox`` 4.0: ``CPYTHON_VERSION_TUPLES``,
  ``PYPY_VERSION_TUPLES``, ``OTHER_PYTHON_INTERPRETERS``, and ``DEFAULT_FACTORS`` -
  by :user:`asottile` (`914 &lt;https://github.com/tox-dev/tox/issues/914&gt;`_)


Documentation
^^^^^^^^^^^^^

- Add a system overview section on the index page that explains briefly how tox works -
  by :user:`gaborbernat`. (`867 &lt;https://github.com/tox-dev/tox/issues/867&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/tox
  - Changelog: https://pyup.io/changelogs/tox/
  - Docs: https://tox.readthedocs.org/
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment