Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No longer hard-code pythons #914

Merged
merged 1 commit into from Jul 19, 2018
Merged

No longer hard-code pythons #914

merged 1 commit into from Jul 19, 2018

Conversation

@asottile
Copy link
Member

@asottile asottile commented Jul 17, 2018

WIP -- I'll fill out changelog and such if it works and if it seems like a good idea. It always struck me as odd that tox hardcoded this list of interpreters and that I'd be forced to upgrade my tox installation to use new things. This patch adds support for python2.0 - python9.9, hopefully leaving us some headroom.

Here's hoping python3.10 isn't coming out soon, eh?

If this is good, the lists of interpreters can be deprecated and eventually removed I would imagine

@asottile
Copy link
Member Author

@asottile asottile commented Jul 17, 2018

for instance:

$ tox -e py28 --notest
GLOB sdist-make: /tmp/tox/setup.py
py28 create: /tmp/tox/.tox/py28
SKIPPED: InterpreterNotFound: python2.8
___________________________________ summary ____________________________________
SKIPPED:  py28: InterpreterNotFound: python2.8
  congratulations :)
$ tox -e py99 --notest
GLOB sdist-make: /tmp/tox/setup.py
py99 create: /tmp/tox/.tox/py99
SKIPPED: InterpreterNotFound: python9.9
___________________________________ summary ____________________________________
SKIPPED:  py99: InterpreterNotFound: python9.9
  congratulations :)
$ tox -e py6 --notest
GLOB sdist-make: /tmp/tox/setup.py
py6 create: /tmp/tox/.tox/py6
SKIPPED: InterpreterNotFound: python6
___________________________________ summary ____________________________________
SKIPPED:  py6: InterpreterNotFound: python6
  congratulations :)

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jul 17, 2018

Good idea :)

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 17, 2018

Why not add support for (\d).(\d){2,}?

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 17, 2018

Note the VSTS CI failed with valid failures.

@asottile
Copy link
Member Author

@asottile asottile commented Jul 17, 2018

Why not add support for (\d).(\d){2,}?

because it doesn't already exist -- wanted to replicate existing behaviour as much as possible

The failure as far as I can tell isn't actually an issue since python falls to sys.executable anyway (just not at that portion in the code) but I'll fix before shipping

@asottile
Copy link
Member Author

@asottile asottile commented Jul 17, 2018

@gaborbernat did you force push master? my history no longer aligns

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 17, 2018

Yes sadly by mistake, and had to rebuild master 👎 if you want to force push back and cherry pick the one new merge you're free 👍 made the master protected.

@asottile
Copy link
Member Author

@asottile asottile commented Jul 17, 2018

ok, I'm going to reset to e80f69e and then re-apply cc3d3e6

@asottile
Copy link
Member Author

@asottile asottile commented Jul 17, 2018

oh right, need to unprotect to force push myself. I've put the old master I'm about to force push over at _old-master (we can delete this branch once I'm done and we're happy)

@asottile
Copy link
Member Author

@asottile asottile commented Jul 17, 2018

@gaborbernat can you re setup the branch protection (if master looks good) -- I've forgotten which checks were required 🤦‍♂️

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 17, 2018

sure 🆗

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 17, 2018

Done, your tests still fail at:

2018-07-17T14:35:36.9813333Z     def test_py_venv(self, newconfig):
2018-07-17T14:35:36.9829608Z         config = newconfig(["-epy"], "")
2018-07-17T14:35:36.9846066Z         env = config.envconfigs["py"]
2018-07-17T14:35:36.9861785Z >       assert str(env.basepython) == sys.executable
2018-07-17T14:35:36.9877452Z E       AssertionError: assert 'python' == '/opt/vsts/work/1/s/.tox/py/bin/python'
2018-07-17T14:35:36.9894093Z E         - python
2018-07-17T14:35:36.9909134Z E         + /opt/vsts/work/1/s/.tox/py/bin/python
2018-07-17T14:35:36.9917827Z 
2018-07-17T14:35:36.9939780Z config     = <tox.config.Config object at 0x7f84845f56d0>
2018-07-17T14:35:36.9955010Z env        = <tox.config.TestenvConfig instance at 0x7f84842a3d88>
2018-07-17T14:35:36.9970433Z newconfig  = <function create_new_config_file_ at 0x7f84841be578>

@asottile
Copy link
Member Author

@asottile asottile commented Jul 17, 2018

yeah haven't "fixed" that yet -- just rebased :)

@codecov
Copy link

@codecov codecov bot commented Jul 17, 2018

Codecov Report

Merging #914 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #914      +/-   ##
==========================================
+ Coverage   92.61%   92.63%   +0.01%     
==========================================
  Files          12       12              
  Lines        2330     2335       +5     
  Branches      409      409              
==========================================
+ Hits         2158     2163       +5     
  Misses        109      109              
  Partials       63       63
Impacted Files Coverage Δ
src/tox/config.py 95.91% <100%> (+0.01%) ⬆️
src/tox/constants.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66bbfef...f837b1a. Read the comment docs.

Copy link
Member

@gaborbernat gaborbernat left a comment

Is this ready then @asottile? In general, I agree with the approach although I think we might as well make the minor take two version numbers, not just one. I know 3.10 is not out yet, but neither is 3.9 and we support that via this construction. Will need a news fragment, otherwise looks great.

default = tox.PYTHON.DEFAULT_FACTORS[factor]
match = tox.PYTHON.PY_FACTORS_RE.match(factor)
if match:
default = tox.PYTHON.PY_FACTORS_MAP[match.group(1)] + ".".join(
Copy link
Member

@gaborbernat gaborbernat Jul 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would prefer a format here 🤔

@asottile
Copy link
Member Author

@asottile asottile commented Jul 18, 2018

According to http://www.curiousefficiency.org/posts/2014/08/python-4000.html 3.10 will never come -- so I'm going to leave this at two digits for now

Curious Efficiency
Newcomers to python-ideas occasionally make reference to the idea of "Python 4000" when proposing backwards incompatible changes that don't offer a clear migration path from currently legal Python 3 c

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jul 18, 2018

My current expectation is that Python 4.0 will merely be "the release that comes after Python 3.9".

Sounds simple enough :)

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 19, 2018

Taken from there what you linked:

Update: After this post was originally written back in 2014, subsequent discussions on the core python-dev mailing list led to the conclusion that the release after 3.9 will probably just be 3.10. However, a 4.0 will presumably still happen some day, and the premise of this article is expected to hold for that release: it will be held to the same backwards compatibility obligations as a Python 3.X to 3.X+1 update.

So there highly likely will be a 3.10.

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jul 19, 2018

oh, ok then maybe support it right away after all, then we can't forget it in 2023 ...

Copy link
Member

@gaborbernat gaborbernat left a comment

I'm fine to merge this at it is. However just pointing out that py40 probably will come after py310.

@asottile
Copy link
Member Author

@asottile asottile commented Jul 19, 2018

@gaborbernat is the "this branch is out of date" thing new? Really don't like it, it is very tedious.

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 19, 2018

Fixed.

@gaborbernat gaborbernat merged commit 95b9e1c into tox-dev:master Jul 19, 2018
5 checks passed
@asottile asottile deleted the unhardcode_pythons branch Jul 19, 2018
@asottile asottile mentioned this pull request Jul 27, 2018
@anlutro
Copy link

@anlutro anlutro commented Aug 3, 2018

Could we get a pypi release with this fix included? 3.1.2 is affected by the bug. I was super confused when -e py33 was working despite python3.3 not being installed.

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Aug 3, 2018

@anlutro are you sure you're referring to this? This PR allows specifying python 3.9 and so on. Has not much to do with -e py33.

@anlutro
Copy link

@anlutro anlutro commented Aug 3, 2018

Yes, I'm sure. If I revert the commit of this PR, tox -e py33 creates a virtualenv using python3.6.

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Aug 3, 2018

Can you provide more details in an issue, please? Thanks!

@anlutro
Copy link

@anlutro anlutro commented Aug 3, 2018

Sure, if you insist, but the problem is already solved in master by this PR.

@anlutro
Copy link

@anlutro anlutro commented Aug 3, 2018

Opened issue here: #932

guykisel added a commit to guykisel/inline-plz that referenced this issue Sep 24, 2018
This PR updates [tox](https://pypi.org/project/tox) from **3.1.2** to **3.4.0**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.4.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this &lt;https://pip.pypa.io/en/latest/reference/pip/exists-action-option&gt;`_) - by :user:`gaborbernat` (`503 &lt;https://github.com/tox-dev/tox/issues/503&gt;`_)
- instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 &lt;https://github.com/tox-dev/tox/issues/908&gt;`_)
- PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 &lt;https://github.com/pypa/setuptools/pull/1481&gt;`_ - by :user:`gaborbernat` (`1003 &lt;https://github.com/tox-dev/tox/issues/1003&gt;`_)


Features
^^^^^^^^

- add ``commands_pre`` and ``commands_post`` that run before and after running
  the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all
  run until the first failing command)  - by :user:`gaborbernat` (`167 &lt;https://github.com/tox-dev/tox/issues/167&gt;`_)
- ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 &lt;https://github.com/tox-dev/tox/issues/814&gt;`_)
- use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``)  - by :user:`gaborbernat` (`824 &lt;https://github.com/tox-dev/tox/issues/824&gt;`_)
- always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 &lt;https://github.com/tox-dev/tox/issues/838&gt;`_)
- tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory,
  ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 &lt;https://github.com/tox-dev/tox/issues/847&gt;`_)
- While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 &lt;https://github.com/tox-dev/tox/issues/851&gt;`_)
- skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 &lt;https://github.com/tox-dev/tox/issues/903&gt;`_)
- keep additional environments config order when listing them - by :user:`gaborbernat` (`921 &lt;https://github.com/tox-dev/tox/issues/921&gt;`_)
- allow injecting config value inside the ini file dependent of the fact that we&#39;re connected to an interactive shell or not  - by :user:`gaborbernat` (`947 &lt;https://github.com/tox-dev/tox/issues/947&gt;`_)
- do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 &lt;https://github.com/tox-dev/tox/issues/974&gt;`_)
- when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 &lt;https://github.com/tox-dev/tox/issues/982&gt;`_)
- when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 &lt;https://github.com/tox-dev/tox/issues/994&gt;`_)
- ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 &lt;https://github.com/tox-dev/tox/issues/1004&gt;`_)


Documentation
^^^^^^^^^^^^^

- clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 &lt;https://github.com/tox-dev/tox/issues/305&gt;`_)
- add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 &lt;https://github.com/tox-dev/tox/issues/374&gt;`_)
- specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 &lt;https://github.com/tox-dev/tox/issues/545&gt;`_)
   ```
   
  
  
   ### 3.3.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 &lt;https://github.com/tox-dev/tox/issues/931&gt;`_)


Features
^^^^^^^^

- `PEP-517 &lt;https://www.python.org/dev/peps/pep-0517/&gt;`_ source distribution support (create a
  ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 &lt;https://github.com/tox-dev/tox/issues/573&gt;`_)
- `flit &lt;https://flit.readthedocs.io&gt;`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 &lt;https://github.com/tox-dev/tox/issues/820&gt;`_)
- packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 &lt;https://github.com/tox-dev/tox/issues/951&gt;`_)


Miscellaneous
^^^^^^^^^^^^^

- Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 &lt;https://github.com/tox-dev/tox/issues/955&gt;`_)
   ```
   
  
  
   ### 3.2.1
   ```
   ------------------

Bugfixes
^^^^^^^^

- ``--parallel--safe-build`` no longer cleans up its folders (``distdir``, ``distshare``, ``log``). - by :user:`gaborbernat` (`849 &lt;https://github.com/tox-dev/tox/issues/849&gt;`_)
   ```
   
  
  
   ### 3.2.0
   ```
   ------------------

Features
^^^^^^^^

- Switch pip invocations to use the module ``-m pip`` instead of direct invocation. This could help
  avoid some of the shebang limitations.  - by :user:`gaborbernat` (`935 &lt;https://github.com/tox-dev/tox/issues/935&gt;`_)
- Ability to specify package requirements for the tox run via the ``tox.ini`` (``tox`` section under key ``requires`` - PEP-508 style): can be used to specify both plugin requirements or build dependencies. - by :user:`gaborbernat` (`783 &lt;https://github.com/tox-dev/tox/issues/783&gt;`_)
- Allow to run multiple tox instances in parallel by providing the
  ``--parallel--safe-build`` flag. - by :user:`gaborbernat` (`849 &lt;https://github.com/tox-dev/tox/issues/849&gt;`_)
   ```
   
  
  
   ### 3.1.3
   ```
   ------------------

Bugfixes
^^^^^^^^

- A caching issue that caused the ``develop-inst-nodeps`` action, which
  reinstalls the package under test, to always run has been resolved. The
  ``develop-inst-noop`` action, which, as the name suggests, is a no-op, will now
  run unless there are changes to ``setup.py`` or ``setup.cfg`` files that have
  not been reflected - by stephenfin (`909 &lt;https://github.com/tox-dev/tox/issues/909&gt;`_)


Features
^^^^^^^^

- Python version testenvs are now automatically detected instead of comparing
  against a hard-coded list of supported versions.  This enables ``py38`` and
  eventually ``py39`` / ``py40`` / etc. to work without requiring an upgrade to
  ``tox``.  As such, the following public constants are now deprecated
  (and scheduled for removal in ``tox`` 4.0: ``CPYTHON_VERSION_TUPLES``,
  ``PYPY_VERSION_TUPLES``, ``OTHER_PYTHON_INTERPRETERS``, and ``DEFAULT_FACTORS`` -
  by :user:`asottile` (`914 &lt;https://github.com/tox-dev/tox/issues/914&gt;`_)


Documentation
^^^^^^^^^^^^^

- Add a system overview section on the index page that explains briefly how tox works -
  by :user:`gaborbernat`. (`867 &lt;https://github.com/tox-dev/tox/issues/867&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/tox
  - Changelog: https://pyup.io/changelogs/tox/
  - Docs: https://tox.readthedocs.org/
</details>
tomasaschan added a commit to tomasaschan/chartify that referenced this issue Apr 22, 2020
Mainly to get a version that includes tox-dev/tox#914
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants