Skip to content

Commit

Permalink
feat: support for reuse_venv option (#730)
Browse files Browse the repository at this point in the history
* feat: support for reuse_venv option

* typing: adding Literal for reuse_venv choices

* chore: ruff-format

* chore: codespell

* chore: move `generate_noxfile_options` fixture to `conftest.py`

* chore: ruff-format

* chore: fix lints

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* docs: clarify reuse_existing_venv function decision matrix

* chore: remove ``# pragma: no cover` due to recent main branch changes

* docs: call out that --reuse-existing-virtualenvs/--no-reuse-existing-virtualenvs is alias to --reuse-venv=yes|no in usage.rst

* docs: small update to config.rst

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
  • Loading branch information
samypr100 and henryiii committed Feb 25, 2024
1 parent 55e09cd commit 5a097cd
Show file tree
Hide file tree
Showing 9 changed files with 311 additions and 33 deletions.
5 changes: 3 additions & 2 deletions docs/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ Use of :func:`session.install()` is deprecated without a virtualenv since it mod
def tests(session):
session.run("pip", "install", "nox")
You can also specify that the virtualenv should *always* be reused instead of recreated every time:
You can also specify that the virtualenv should *always* be reused instead of recreated every time unless ``--reuse-venv=never``:

.. code-block:: python
Expand Down Expand Up @@ -432,7 +432,8 @@ The following options can be specified in the Noxfile:
* ``nox.options.tags`` is equivalent to specifying :ref:`-t or --tags <opt-sessions-pythons-and-keywords>`.
* ``nox.options.default_venv_backend`` is equivalent to specifying :ref:`-db or --default-venv-backend <opt-default-venv-backend>`.
* ``nox.options.force_venv_backend`` is equivalent to specifying :ref:`-fb or --force-venv-backend <opt-force-venv-backend>`.
* ``nox.options.reuse_existing_virtualenvs`` is equivalent to specifying :ref:`--reuse-existing-virtualenvs <opt-reuse-existing-virtualenvs>`. You can force this off by specifying ``--no-reuse-existing-virtualenvs`` during invocation.
* ``nox.options.reuse_venv`` is equivalent to specifying :ref:`--reuse-venv <opt-reuse-venv>`. Preferred over using ``nox.options.reuse_existing_virtualenvs``.
* ``nox.options.reuse_existing_virtualenvs`` is equivalent to specifying :ref:`--reuse-existing-virtualenvs <opt-reuse-existing-virtualenvs>`. You can force this off by specifying ``--no-reuse-existing-virtualenvs`` during invocation. Alias of ``nox.options.reuse_venv=yes|no``.
* ``nox.options.stop_on_first_error`` is equivalent to specifying :ref:`--stop-on-first-error <opt-stop-on-first-error>`. You can force this off by specifying ``--no-stop-on-first-error`` during invocation.
* ``nox.options.error_on_missing_interpreters`` is equivalent to specifying :ref:`--error-on-missing-interpreters <opt-error-on-missing-interpreters>`. You can force this off by specifying ``--no-error-on-missing-interpreters`` during invocation.
* ``nox.options.error_on_external_run`` is equivalent to specifying :ref:`--error-on-external-run <opt-error-on-external-run>`. You can force this off by specifying ``--no-error-on-external-run`` during invocation.
Expand Down
25 changes: 21 additions & 4 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -179,34 +179,51 @@ Finally note that the ``--no-venv`` flag is a shortcut for ``--force-venv-backen
nox --no-venv
.. _opt-reuse-existing-virtualenvs:
.. _opt-reuse-venv:

Re-using virtualenvs
--------------------

By default, Nox deletes and recreates virtualenvs every time it is run. This is usually fine for most projects and continuous integration environments as `pip's caching <https://pip.pypa.io/en/stable/cli/pip_install/#caching>`_ makes re-install rather quick. However, there are some situations where it is advantageous to reuse the virtualenvs between runs. Use ``-r`` or ``--reuse-existing-virtualenvs``:
By default, Nox deletes and recreates virtualenvs every time it is run. This is
usually fine for most projects and continuous integration environments as
`pip's caching <https://pip.pypa.io/en/stable/cli/pip_install/#caching>`_ makes
re-install rather quick. However, there are some situations where it is
advantageous to reuse the virtualenvs between runs. Use ``-r`` or
``--reuse-existing-virtualenvs`` or for fine-grained control use
``--reuse-venv=yes|no|always|never``:

.. code-block:: console
nox -r
nox --reuse-existing-virtualenvs
nox --reuse-venv=yes # preferred
If the Noxfile sets ``nox.options.reuse_existing_virtualenvs``, you can override the Noxfile setting from the command line by using ``--no-reuse-existing-virtualenvs``.
Similarly you can override ``nox.options.reuse_venvs`` from the Noxfile via the command line by using ``--reuse-venv=yes|no|always|never``.

.. note::

Additionally, you can skip the re-installation of packages when a virtualenv is reused. Use ``-R`` or ``--reuse-existing-virtualenvs --no-install``:
``--reuse-existing-virtualenvs`` is a alias for ``--reuse-venv=yes`` and ``--no-reuse-existing-virtualenvs`` is an alias for ``--reuse-venv=no``.

Additionally, you can skip the re-installation of packages when a virtualenv is reused.
Use ``-R`` or ``--reuse-existing-virtualenvs --no-install`` or ``--reuse-venv=yes --no-install``:

.. code-block:: console
nox -R
nox --reuse-existing-virtualenvs --no-install
nox --reuse-venv=yes --no-install
The ``--no-install`` option causes the following session methods to return early:

- :func:`session.install <nox.sessions.Session.install>`
- :func:`session.conda_install <nox.sessions.Session.conda_install>`
- :func:`session.run_install <nox.sessions.Session.run_install>`

This option has no effect if the virtualenv is not being reused.
The ``never`` and ``always`` options in ``--reuse-venv`` gives you more fine-grained control
as it ignores when a ``@nox.session`` has ``reuse_venv=True|False`` defined.

These options have no effect if the virtualenv is not being reused.

.. _opt-running-extra-pythons:

Expand Down
58 changes: 56 additions & 2 deletions nox/_options.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,13 @@
from nox import _option_set
from nox.tasks import discover_manifest, filter_manifest, load_nox_module

if sys.version_info < (3, 8):
from typing_extensions import Literal
else:
from typing import Literal

ReuseVenvType = Literal["no", "yes", "never", "always"]

"""All of Nox's configuration options."""

options = _option_set.OptionSet(
Expand Down Expand Up @@ -146,6 +153,30 @@ def _envdir_merge_func(
return command_args.envdir or noxfile_args.envdir or ".nox"


def _reuse_venv_merge_func(
command_args: argparse.Namespace, noxfile_args: argparse.Namespace
) -> ReuseVenvType:
"""Merge reuse_venv from command args and Noxfile while maintaining
backwards compatibility with reuse_existing_virtualenvs. Default is "no".
Args:
command_args (_option_set.Namespace): The options specified on the
command-line.
noxfile_Args (_option_set.Namespace): The options specified in the
Noxfile.
"""
# back-compat scenario with no_reuse_existing_virtualenvs/reuse_existing_virtualenvs
if command_args.no_reuse_existing_virtualenvs:
return "no"
if (
command_args.reuse_existing_virtualenvs
or noxfile_args.reuse_existing_virtualenvs
):
return "yes"
# regular option behavior
return command_args.reuse_venv or noxfile_args.reuse_venv or "no"


def default_env_var_list_factory(env_var: str) -> Callable[[], list[str] | None]:
"""Looks at the env var to set the default value for a list of env vars.
Expand Down Expand Up @@ -197,12 +228,22 @@ def _force_pythons_finalizer(


def _R_finalizer(value: bool, args: argparse.Namespace) -> bool:
"""Propagate -R to --reuse-existing-virtualenvs and --no-install."""
"""Propagate -R to --reuse-existing-virtualenvs and --no-install and --reuse-venv=yes."""
if value:
args.reuse_venv = "yes"
args.reuse_existing_virtualenvs = args.no_install = value
return value


def _reuse_existing_virtualenvs_finalizer(
value: bool, args: argparse.Namespace
) -> bool:
"""Propagate --reuse-existing-virtualenvs to --reuse-venv=yes."""
if value:
args.reuse_venv = "yes"
return value


def _posargs_finalizer(
value: Sequence[Any], args: argparse.Namespace
) -> Sequence[Any] | list[Any]:
Expand Down Expand Up @@ -412,6 +453,18 @@ def _tag_completer(
" creating a venv. This is an alias for '--force-venv-backend none'."
),
),
_option_set.Option(
"reuse_venv",
"--reuse-venv",
group=options.groups["environment"],
noxfile=True,
merge_func=_reuse_venv_merge_func,
help=(
"Controls existing virtualenvs recreation. This is ``'no'`` by"
" default, but any of ``('yes', 'no', 'always', 'never')`` are accepted."
),
choices=["yes", "no", "always", "never"],
),
*_option_set.make_flag_pair(
"reuse_existing_virtualenvs",
("-r", "--reuse-existing-virtualenvs"),
Expand All @@ -420,7 +473,8 @@ def _tag_completer(
"--no-reuse-existing-virtualenvs",
),
group=options.groups["environment"],
help="Reuse existing virtualenvs instead of recreating them.",
help="This is an alias for '--reuse-venv=yes|no'.",
finalizer_func=_reuse_existing_virtualenvs_finalizer,
),
_option_set.Option(
"R",
Expand Down
49 changes: 46 additions & 3 deletions nox/sessions.py
Original file line number Diff line number Diff line change
Expand Up @@ -767,9 +767,7 @@ def _create_venv(self) -> None:
self.venv = PassthroughEnv()
return

reuse_existing = (
self.func.reuse_venv or self.global_config.reuse_existing_virtualenvs
)
reuse_existing = self.reuse_existing_venv()

if backend is None or backend in {"virtualenv", "venv", "uv"}:
self.venv = VirtualEnv(
Expand All @@ -795,6 +793,51 @@ def _create_venv(self) -> None:

self.venv.create()

def reuse_existing_venv(self) -> bool:
"""
Determines whether to reuse an existing virtual environment.
The decision matrix is as follows:
+--------------------------+-----------------+-------------+
| global_config.reuse_venv | func.reuse_venv | Reuse venv? |
+==========================+=================+=============+
| "always" | N/A | Yes |
+--------------------------+-----------------+-------------+
| "never" | N/A | No |
+--------------------------+-----------------+-------------+
| "yes" | True|None | Yes |
+--------------------------+-----------------+-------------+
| "yes" | False | No |
+--------------------------+-----------------+-------------+
| "no" | True | Yes |
+--------------------------+-----------------+-------------+
| "no" | False|None | No |
+--------------------------+-----------------+-------------+
Summary
~~~~~~~
- "always" forces reuse regardless of `func.reuse_venv`.
- "never" forces recreation regardless of `func.reuse_venv`.
- "yes" and "no" respect `func.reuse_venv` being ``False`` or ``True`` respectively.
Returns:
bool: True if the existing virtual environment should be reused, False otherwise.
"""

return any(
(
# "always" forces reuse regardless of func.reuse_venv
self.global_config.reuse_venv == "always",
# Respect func.reuse_venv when it's explicitly True, unless global_config is "never"
self.func.reuse_venv is True
and self.global_config.reuse_venv != "never",
# Delegate to reuse ("yes") when func.reuse_venv is not explicitly False
self.func.reuse_venv is not False
and self.global_config.reuse_venv == "yes",
)
)

def execute(self) -> Result:
logger.warning(f"Running session {self.friendly_name}")

Expand Down
46 changes: 46 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,23 @@
# Copyright 2023 Alethea Katherine Flowers
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations

import re
from pathlib import Path
from string import Template
from typing import Callable

import pytest


Expand All @@ -6,3 +26,29 @@ def reset_color_envvars(monkeypatch):
"""Remove color-related envvars to fix test output"""
monkeypatch.delenv("FORCE_COLOR", raising=False)
monkeypatch.delenv("NO_COLOR", raising=False)


RESOURCES = Path(__file__).parent.joinpath("resources")


@pytest.fixture
def generate_noxfile_options(tmp_path: Path) -> Callable[..., str]:
"""Generate noxfile.py with test and templated options.
The options are enabled (if disabled) and the values are applied
if a matching format string is encountered with the option name.
"""

def generate_noxfile(**option_mapping: str | bool) -> str:
path = Path(RESOURCES) / "noxfile_options.py"
text = path.read_text(encoding="utf8")
if option_mapping:
for opt, _val in option_mapping.items():
# "uncomment" options with values provided
text = re.sub(rf"(# )?nox.options.{opt}", f"nox.options.{opt}", text)
text = Template(text).safe_substitute(**option_mapping)
path = tmp_path / "noxfile.py"
path.write_text(text)
return str(path)

return generate_noxfile
5 changes: 3 additions & 2 deletions tests/resources/noxfile_options.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@

import nox

nox.options.reuse_existing_virtualenvs = True
# nox.options.error_on_missing_interpreters = {error_on_missing_interpreters} # used by tests
# nox.options.reuse_existing_virtualenvs = ${reuse_existing_virtualenvs}
# nox.options.reuse_venv = "${reuse_venv}"
# nox.options.error_on_missing_interpreters = ${error_on_missing_interpreters}
nox.options.sessions = ["test"]


Expand Down

0 comments on commit 5a097cd

Please sign in to comment.