option to skip a factor level #824

gaborbernat · 2018-05-14T20:57:27Z

For example if {py27,py36}-{nocov,cov,diffcov}. @warsaw

In this case the user might want to not run all non diffcov envs.

warsaw · 2018-05-15T01:53:47Z

A little more detail. In an example such as above, we generally run all the factors, however if we create an sdist, then unpack it and run tox in the unpacked directory, all -diffcov environments will fail. This is because diffcov only runs in a git repo (because it does diff comparisons against master). So when running in a non-git repository (e.g. an unpacked sdist), we want to skip all the diffcov environments.

gaborbernat · 2018-05-15T02:54:38Z

@obestwalter @asottile @warsaw

Here's what I propose, what if the users could do fnmatch/regex filtering in env specification?

e.g. -e '.*-(nocov|diffcov)'

rpkilby · 2018-05-15T03:32:29Z

btw, this would conflict with passing a list of test env names (eg, tox -e isort,lint)

gaborbernat · 2018-05-15T12:19:23Z

maybe have a separate env var for it then?

rpkilby · 2018-05-15T14:17:58Z

Maybe -re?

gaborbernat · 2018-05-15T14:51:16Z

I would be even fine with having just an environment variable (TOX_ENV_FILTER). This is an advanced feature for advanced users I would believe.

warsaw · 2018-05-15T14:54:23Z

@gaborbernat I think that would be good enough for me.

### Update [tox](https://pypi.org/project/tox) from **3.2.1** to **3.4.0**. <details> <summary>Changelog</summary> ### 3.4.0 ``` ------------------ Bugfixes ^^^^^^^^ - add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this <https://pip.pypa.io/en/latest/reference/pip/exists-action-option>`_) - by :user:`gaborbernat` (`503 <https://github.com/tox-dev/tox/issues/503>`_) - instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 <https://github.com/tox-dev/tox/issues/908>`_) - PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 <https://github.com/pypa/setuptools/pull/1481>`_ - by :user:`gaborbernat` (`1003 <https://github.com/tox-dev/tox/issues/1003>`_) Features ^^^^^^^^ - add ``commands_pre`` and ``commands_post`` that run before and after running the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all run until the first failing command) - by :user:`gaborbernat` (`167 <https://github.com/tox-dev/tox/issues/167>`_) - ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 <https://github.com/tox-dev/tox/issues/814>`_) - use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``) - by :user:`gaborbernat` (`824 <https://github.com/tox-dev/tox/issues/824>`_) - always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 <https://github.com/tox-dev/tox/issues/838>`_) - tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory, ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 <https://github.com/tox-dev/tox/issues/847>`_) - While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 <https://github.com/tox-dev/tox/issues/851>`_) - skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 <https://github.com/tox-dev/tox/issues/903>`_) - keep additional environments config order when listing them - by :user:`gaborbernat` (`921 <https://github.com/tox-dev/tox/issues/921>`_) - allow injecting config value inside the ini file dependent of the fact that we're connected to an interactive shell or not - by :user:`gaborbernat` (`947 <https://github.com/tox-dev/tox/issues/947>`_) - do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 <https://github.com/tox-dev/tox/issues/974>`_) - when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 <https://github.com/tox-dev/tox/issues/982>`_) - when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 <https://github.com/tox-dev/tox/issues/994>`_) - ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 <https://github.com/tox-dev/tox/issues/1004>`_) Documentation ^^^^^^^^^^^^^ - clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 <https://github.com/tox-dev/tox/issues/305>`_) - add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 <https://github.com/tox-dev/tox/issues/374>`_) - specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 <https://github.com/tox-dev/tox/issues/545>`_) ``` ### 3.3.0 ``` ------------------ Bugfixes ^^^^^^^^ - fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 <https://github.com/tox-dev/tox/issues/931>`_) Features ^^^^^^^^ - `PEP-517 <https://www.python.org/dev/peps/pep-0517/>`_ source distribution support (create a ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 <https://github.com/tox-dev/tox/issues/573>`_) - `flit <https://flit.readthedocs.io>`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 <https://github.com/tox-dev/tox/issues/820>`_) - packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 <https://github.com/tox-dev/tox/issues/951>`_) Miscellaneous ^^^^^^^^^^^^^ - Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 <https://github.com/tox-dev/tox/issues/955>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/tox - Changelog: https://pyup.io/changelogs/tox/ - Docs: https://tox.readthedocs.org/ </details> ### Update [Sphinx](https://pypi.org/project/Sphinx) from **1.8.0** to **1.8.1**. <details> <summary>Changelog</summary> ### 1.8.1 ``` ===================================== Incompatible changes -------------------- * LaTeX ``\pagestyle`` commands have been moved to the LaTeX template. No changes in PDF, except possibly if ``\sphinxtableofcontents``, which contained them, had been customized in :file:`conf.py`. (refs: 5455) Bugs fixed ---------- * 5418: Incorrect default path for sphinx-build -d/doctrees files * 5421: autodoc emits deprecation warning for :confval:`autodoc_default_flags` * 5422: lambda object causes PicklingError on storing environment * 5417: Sphinx fails to build with syntax error in Python 2.7.5 * 4911: add latexpdf to make.bat for non make-mode * 5436: Autodoc does not work with enum subclasses with properties/methods * 5437: autodoc: crashed on modules importing eggs * 5433: latex: ImportError: cannot import name 'DEFAULT_SETTINGS' * 5431: autodoc: ``autofunction`` emits a warning for callable objects * 5457: Fix TypeError in error message when override is prohibited * 5453: PDF builds of 'howto' documents have no page numbers * 5463: mathbase: math_role and MathDirective was disappeared in 1.8.0 * 5454: latex: Index has disappeared from PDF for Japanese documents * 5432: py domain: ``:type:`` field can't process ``:term:`` references * 5426: py domain: TypeError has been raised for class attribute ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/sphinx - Changelog: https://pyup.io/changelogs/sphinx/ - Homepage: http://sphinx-doc.org/ </details> ### Update [twine](https://pypi.org/project/twine) from **1.11.0** to **1.12.0**. <details> <summary>Changelog</summary> ### 1.12.0 ``` * :feature:`395 major` Add ``twine check`` command to check long description * :feature:`392 major` Drop support for Python 3.3 * :feature:`363` Empower ``--skip-existing`` for Artifactory repositories * 🐛`367` Avoid MD5 when Python is compiled in FIPS mode ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/twine - Changelog: https://pyup.io/changelogs/twine/ - Docs: https://twine.readthedocs.io/ </details>

This PR updates [tox](https://pypi.org/project/tox) from **3.1.2** to **3.4.0**. <details> <summary>Changelog</summary> ### 3.4.0 ``` ------------------ Bugfixes ^^^^^^^^ - add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this <https://pip.pypa.io/en/latest/reference/pip/exists-action-option>`_) - by :user:`gaborbernat` (`503 <https://github.com/tox-dev/tox/issues/503>`_) - instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 <https://github.com/tox-dev/tox/issues/908>`_) - PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 <https://github.com/pypa/setuptools/pull/1481>`_ - by :user:`gaborbernat` (`1003 <https://github.com/tox-dev/tox/issues/1003>`_) Features ^^^^^^^^ - add ``commands_pre`` and ``commands_post`` that run before and after running the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all run until the first failing command) - by :user:`gaborbernat` (`167 <https://github.com/tox-dev/tox/issues/167>`_) - ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 <https://github.com/tox-dev/tox/issues/814>`_) - use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``) - by :user:`gaborbernat` (`824 <https://github.com/tox-dev/tox/issues/824>`_) - always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 <https://github.com/tox-dev/tox/issues/838>`_) - tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory, ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 <https://github.com/tox-dev/tox/issues/847>`_) - While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 <https://github.com/tox-dev/tox/issues/851>`_) - skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 <https://github.com/tox-dev/tox/issues/903>`_) - keep additional environments config order when listing them - by :user:`gaborbernat` (`921 <https://github.com/tox-dev/tox/issues/921>`_) - allow injecting config value inside the ini file dependent of the fact that we're connected to an interactive shell or not - by :user:`gaborbernat` (`947 <https://github.com/tox-dev/tox/issues/947>`_) - do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 <https://github.com/tox-dev/tox/issues/974>`_) - when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 <https://github.com/tox-dev/tox/issues/982>`_) - when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 <https://github.com/tox-dev/tox/issues/994>`_) - ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 <https://github.com/tox-dev/tox/issues/1004>`_) Documentation ^^^^^^^^^^^^^ - clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 <https://github.com/tox-dev/tox/issues/305>`_) - add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 <https://github.com/tox-dev/tox/issues/374>`_) - specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 <https://github.com/tox-dev/tox/issues/545>`_) ``` ### 3.3.0 ``` ------------------ Bugfixes ^^^^^^^^ - fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 <https://github.com/tox-dev/tox/issues/931>`_) Features ^^^^^^^^ - `PEP-517 <https://www.python.org/dev/peps/pep-0517/>`_ source distribution support (create a ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 <https://github.com/tox-dev/tox/issues/573>`_) - `flit <https://flit.readthedocs.io>`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 <https://github.com/tox-dev/tox/issues/820>`_) - packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 <https://github.com/tox-dev/tox/issues/951>`_) Miscellaneous ^^^^^^^^^^^^^ - Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 <https://github.com/tox-dev/tox/issues/955>`_) ``` ### 3.2.1 ``` ------------------ Bugfixes ^^^^^^^^ - ``--parallel--safe-build`` no longer cleans up its folders (``distdir``, ``distshare``, ``log``). - by :user:`gaborbernat` (`849 <https://github.com/tox-dev/tox/issues/849>`_) ``` ### 3.2.0 ``` ------------------ Features ^^^^^^^^ - Switch pip invocations to use the module ``-m pip`` instead of direct invocation. This could help avoid some of the shebang limitations. - by :user:`gaborbernat` (`935 <https://github.com/tox-dev/tox/issues/935>`_) - Ability to specify package requirements for the tox run via the ``tox.ini`` (``tox`` section under key ``requires`` - PEP-508 style): can be used to specify both plugin requirements or build dependencies. - by :user:`gaborbernat` (`783 <https://github.com/tox-dev/tox/issues/783>`_) - Allow to run multiple tox instances in parallel by providing the ``--parallel--safe-build`` flag. - by :user:`gaborbernat` (`849 <https://github.com/tox-dev/tox/issues/849>`_) ``` ### 3.1.3 ``` ------------------ Bugfixes ^^^^^^^^ - A caching issue that caused the ``develop-inst-nodeps`` action, which reinstalls the package under test, to always run has been resolved. The ``develop-inst-noop`` action, which, as the name suggests, is a no-op, will now run unless there are changes to ``setup.py`` or ``setup.cfg`` files that have not been reflected - by stephenfin (`909 <https://github.com/tox-dev/tox/issues/909>`_) Features ^^^^^^^^ - Python version testenvs are now automatically detected instead of comparing against a hard-coded list of supported versions. This enables ``py38`` and eventually ``py39`` / ``py40`` / etc. to work without requiring an upgrade to ``tox``. As such, the following public constants are now deprecated (and scheduled for removal in ``tox`` 4.0: ``CPYTHON_VERSION_TUPLES``, ``PYPY_VERSION_TUPLES``, ``OTHER_PYTHON_INTERPRETERS``, and ``DEFAULT_FACTORS`` - by :user:`asottile` (`914 <https://github.com/tox-dev/tox/issues/914>`_) Documentation ^^^^^^^^^^^^^ - Add a system overview section on the index page that explains briefly how tox works - by :user:`gaborbernat`. (`867 <https://github.com/tox-dev/tox/issues/867>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/tox - Changelog: https://pyup.io/changelogs/tox/ - Docs: https://tox.readthedocs.org/ </details>

gaborbernat added feature:new level:easy labels May 15, 2018

gaborbernat added this to the 3.2 milestone Jul 18, 2018

gaborbernat modified the milestones: 3.2, 3.4 Sep 14, 2018

gaborbernat mentioned this issue Sep 16, 2018

option to skip some environments from the default envlist #990

Merged

helpr bot added the pr-available label Sep 16, 2018

gaborbernat closed this in #990 Sep 16, 2018

gaborbernat added a commit that referenced this issue Sep 16, 2018

option to skip some envlist from the default envlist #824

Loading status checks…

e8f877e

helpr bot added pr-merged and removed pr-available labels Sep 16, 2018

tox-dev / tox

option to skip a factor level #824

option to skip a factor level #824

gaborbernat commented May 14, 2018 •

edited

warsaw commented May 15, 2018

gaborbernat commented May 15, 2018 •

edited

rpkilby commented May 15, 2018 •

edited

gaborbernat commented May 15, 2018

rpkilby commented May 15, 2018

gaborbernat commented May 15, 2018

warsaw commented May 15, 2018

tox-dev / tox

Join GitHub today

option to skip a factor level #824

option to skip a factor level #824

Comments

gaborbernat commented May 14, 2018 • edited

warsaw commented May 15, 2018

gaborbernat commented May 15, 2018 • edited

rpkilby commented May 15, 2018 • edited

gaborbernat commented May 15, 2018

rpkilby commented May 15, 2018

gaborbernat commented May 15, 2018

warsaw commented May 15, 2018

Essential cookies

Always active

Analytics cookies

gaborbernat commented May 14, 2018 •

edited

gaborbernat commented May 15, 2018 •

edited

rpkilby commented May 15, 2018 •

edited