Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option --no-install to skip install commands in reused environments #432

Merged
merged 15 commits into from
Jun 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 16 additions & 1 deletion docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ Finally note that the ``--no-venv`` flag is a shortcut for ``--force-venv-backen
Re-using virtualenvs
--------------------

By default, Nox deletes and recreates virtualenvs every time it is run. This is usually fine for most projects and continuous integration environments as `pip's caching <https://pip.pypa.io/en/stable/reference/pip_install/#caching>`_ makes re-install rather quick. However, there are some situations where it is advantageous to re-use the virtualenvs between runs. Use ``-r`` or ``--reuse-existing-virtualenvs``:
By default, Nox deletes and recreates virtualenvs every time it is run. This is usually fine for most projects and continuous integration environments as `pip's caching <https://pip.pypa.io/en/stable/cli/pip_install/#caching>`_ makes re-install rather quick. However, there are some situations where it is advantageous to re-use the virtualenvs between runs. Use ``-r`` or ``--reuse-existing-virtualenvs``:

.. code-block:: console

Expand All @@ -159,6 +159,21 @@ By default, Nox deletes and recreates virtualenvs every time it is run. This is

If the Noxfile sets ``nox.options.reuse_existing_virtualenvs``, you can override the Noxfile setting from the command line by using ``--no-reuse-existing-virtualenvs``.

Additionally, you can skip the re-installation of packages when a virtualenv is reused. Use ``-R`` or ``--reuse-existing-virtualenvs --no-install``:

.. code-block:: console

nox -R
nox --reuse-existing-virtualenvs --no-install

The ``--no-install`` option causes the following session methods to return early:

- :func:`session.install <nox.sessions.Session.install>`
- :func:`session.conda_install <nox.sessions.Session.conda_install>`
- :func:`session.run_always <nox.sessions.Session.run_always>`

This option has no effect if the virtualenv is not being reused.

.. _opt-running-extra-pythons:

Running additional Python versions
Expand Down
31 changes: 31 additions & 0 deletions nox/_options.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,13 @@ def _force_pythons_finalizer(
return value


def _R_finalizer(value: bool, args: argparse.Namespace) -> bool:
"""Propagate -R to --reuse-existing-virtualenvs and --no-install."""
if value:
args.reuse_existing_virtualenvs = args.no_install = value
return value


def _posargs_finalizer(
value: Sequence[Any], args: argparse.Namespace
) -> Union[Sequence[Any], List[Any]]:
Expand Down Expand Up @@ -320,6 +327,18 @@ def _session_completer(
group=options.groups["secondary"],
help="Re-use existing virtualenvs instead of recreating them.",
),
_option_set.Option(
"R",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be lowercase, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The option is uppercase (-R), so using r as the option name would lead to an error:

Traceback (most recent call last):
  File ".../nox/venv/bin/nox", line 33, in <module>
    sys.exit(load_entry_point('nox', 'console_scripts', 'nox')())
  File ".../nox/nox/__main__.py", line 30, in main
    args = _options.options.parse_args()
  File ".../nox/nox/_option_set.py", line 267, in parse_args
    self._finalize_args(args)
  File ".../nox/nox/_option_set.py", line 255, in _finalize_args
    value = getattr(args, option.name)
AttributeError: 'Namespace' object has no attribute 'r'

Our option.name must match the name that argparse derives for the option. In this case, the option is uppercase -R with no equivalent long option, so argparse derives R as the option name.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha

"-R",
default=False,
group=options.groups["secondary"],
action="store_true",
help=(
"Re-use existing virtualenvs and skip package re-installation."
" This is an alias for '--reuse-existing-virtualenvs --no-install'."
),
finalizer_func=_R_finalizer,
),
_option_set.Option(
"noxfile",
"-f",
Expand Down Expand Up @@ -384,6 +403,18 @@ def _session_completer(
action="store_true",
help="Skip session.run invocations in the Noxfile.",
),
_option_set.Option(
"no_install",
"--no-install",
default=False,
group=options.groups["secondary"],
action="store_true",
help=(
"Skip invocations of session methods for installing packages"
" (session.install, session.conda_install, session.run_always)"
" when a virtualenv is being reused."
),
),
_option_set.Option(
"report",
"--report",
Expand Down
29 changes: 26 additions & 3 deletions nox/sessions.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,8 +272,16 @@ def run_always(
) -> Optional[Any]:
"""Run a command **always**.

This is a variant of :meth:`run` that runs in all cases, including in
the presence of ``--install-only``.
This is a variant of :meth:`run` that runs even in the presence of
``--install-only``. This method returns early if ``--no-install`` is
specified and the virtualenv is being reused.

Here are some cases where this method is useful:

- You need to install packages using a command other than ``pip
install`` or ``conda install``.
- You need to run a command as a prerequisite of package installation,
such as building a package or compiling a binary extension.

:param env: A dictionary of environment variables to expose to the
command. By default, all environment variables are passed.
Expand All @@ -290,6 +298,13 @@ def run_always(
do not have a virtualenv.
:type external: bool
"""
if (
self._runner.global_config.no_install
and self._runner.venv is not None
and self._runner.venv._reused
):
return None

if not args:
raise ValueError("At least one argument required to run_always().")

Expand Down Expand Up @@ -368,6 +383,9 @@ def conda_install(
if not args:
raise ValueError("At least one argument required to install().")

if self._runner.global_config.no_install and venv._reused:
return None

# Escape args that should be (conda-specific; pip install does not need this)
args = _dblquote_pkg_install_args(args)

Expand Down Expand Up @@ -417,15 +435,20 @@ def install(self, *args: str, **kwargs: Any) -> None:

.. _pip: https://pip.readthedocs.org
"""
venv = self._runner.venv

if not isinstance(
self._runner.venv, (CondaEnv, VirtualEnv, PassthroughEnv)
venv, (CondaEnv, VirtualEnv, PassthroughEnv)
): # pragma: no cover
raise ValueError(
"A session without a virtualenv can not install dependencies."
)
if not args:
raise ValueError("At least one argument required to install().")

if self._runner.global_config.no_install and venv._reused:
return None

if "silent" not in kwargs:
kwargs["silent"] = True

Expand Down
7 changes: 7 additions & 0 deletions nox/virtualenv.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ class ProcessEnv:
def __init__(self, bin_paths: None = None, env: Mapping[str, str] = None) -> None:
self._bin_paths = bin_paths
self.env = os.environ.copy()
self._reused = False

if env is not None:
self.env.update(env)
Expand Down Expand Up @@ -224,6 +225,9 @@ def create(self) -> bool:
logger.debug(
"Re-using existing conda env at {}.".format(self.location_name)
)

self._reused = True

return False

cmd = ["conda", "create", "--yes", "--prefix", self.location]
Expand Down Expand Up @@ -423,6 +427,9 @@ def create(self) -> bool:
self.location_name
)
)

self._reused = True

return False

if self.venv_or_virtualenv == "virtualenv":
Expand Down
9 changes: 9 additions & 0 deletions tests/test_main.py
Original file line number Diff line number Diff line change
Expand Up @@ -587,3 +587,12 @@ def test_main_force_python(monkeypatch):
nox.__main__.main()
config = execute.call_args[1]["global_config"]
assert config.pythons == config.extra_pythons == ["3.10"]


def test_main_reuse_existing_virtualenvs_no_install(monkeypatch):
monkeypatch.setattr(sys, "argv", ["nox", "-R"])
with mock.patch("nox.workflow.execute", return_value=0) as execute:
with mock.patch.object(sys, "exit"):
nox.__main__.main()
config = execute.call_args[1]["global_config"]
assert config.reuse_existing_virtualenvs and config.no_install
63 changes: 63 additions & 0 deletions tests/test_sessions.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,25 @@ def test_run_always_install_only(self, caplog):

assert session.run_always(operator.add, 23, 19) == 42

@pytest.mark.parametrize(
("no_install", "reused", "run_called"),
[
(True, True, False),
(True, False, True),
(False, True, True),
(False, False, True),
],
)
def test_run_always_no_install(self, no_install, reused, run_called):
session, runner = self.make_session_and_runner()
runner.global_config.no_install = no_install
runner.venv._reused = reused

with mock.patch.object(nox.command, "run") as run:
session.run_always("python", "-m", "pip", "install", "requests")

assert run.called is run_called

def test_conda_install_bad_args(self):
session, runner = self.make_session_and_runner()
runner.venv = mock.create_autospec(nox.virtualenv.CondaEnv)
Expand Down Expand Up @@ -380,6 +399,31 @@ class SessionNoSlots(nox.sessions.Session):
external="error",
)

@pytest.mark.parametrize(
("no_install", "reused", "run_called"),
[
(True, True, False),
(True, False, True),
(False, True, True),
(False, False, True),
],
)
def test_conda_venv_reused_with_no_install(self, no_install, reused, run_called):
session, runner = self.make_session_and_runner()

runner.venv = mock.create_autospec(nox.virtualenv.CondaEnv)
runner.venv.location = "/path/to/conda/env"
runner.venv.env = {}
runner.venv.is_offline = lambda: True

runner.global_config.no_install = no_install
runner.venv._reused = reused

with mock.patch.object(nox.command, "run") as run:
session.conda_install("baked beans", "eggs", "spam")

assert run.called is run_called

@pytest.mark.parametrize(
"version_constraint",
["no", "yes", "already_dbl_quoted"],
Expand Down Expand Up @@ -538,6 +582,25 @@ def test_skip_no_log(self):
with pytest.raises(nox.sessions._SessionSkip):
session.skip()

@pytest.mark.parametrize(
("no_install", "reused", "run_called"),
[
(True, True, False),
(True, False, True),
(False, True, True),
(False, False, True),
],
)
def test_session_venv_reused_with_no_install(self, no_install, reused, run_called):
session, runner = self.make_session_and_runner()
runner.global_config.no_install = no_install
runner.venv._reused = reused

with mock.patch.object(nox.command, "run") as run:
session.install("eggs", "spam")

assert run.called is run_called

def test___slots__(self):
session, _ = self.make_session_and_runner()
with pytest.raises(AttributeError):
Expand Down
4 changes: 4 additions & 0 deletions tests/test_virtualenv.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,7 @@ def test_condaenv_create(make_conda):
venv.reuse_existing = True
venv.create()
assert dir_.join("test.txt").check()
assert venv._reused


@pytest.mark.skipif(not HAS_CONDA, reason="Missing conda command.")
Expand Down Expand Up @@ -334,7 +335,10 @@ def test_create(monkeypatch, make_one):
assert dir_.join("test.txt").check()
venv.reuse_existing = True
monkeypatch.setattr(nox.virtualenv.nox.command, "run", mock.MagicMock())

venv.create()

assert venv._reused
assert dir_.join("test.txt").check()


Expand Down