From a31ba72735fb2150d387c8e01cba37a06b9b1a49 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Thu, 15 Oct 2020 19:52:57 +0100 Subject: [PATCH 1/4] Clarify how Windows path search affects virtual environments --- Doc/library/venv.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst index 5d4a36481f1dcc..be9e2ea6cf08aa 100644 --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -84,6 +84,18 @@ Creating virtual environments without there needing to be any reference to its virtual environment in ``PATH``. + On Windows, the actual Python interpreter running your script is the *base* + interpreter, rather than the executable in the virtual environment. This is + normally transparent to your script, as the value of ``sys.executable`` is + set to the virtual environment's executable. But if you run a subprocess, + Windows searches the directory containing the *actual* executable before + checking ``PATH``. This means that if you run the command ``python`` via + the ``subprocess`` module, Windows will find the base executable rather than + the virtual environment's executable, *even if the virtual environment is + activated*. The fix for this is to use ``sys.executable`` when calling + Python, rather than relying on the system resolving the ``python`` command + to the executable you expect. + .. _venv-api: From d8ef1619cb35bb618f2d4a570ba89351f64c9c31 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Sun, 18 Oct 2020 17:04:28 +0100 Subject: [PATCH 2/4] Update based on feedback - put the information in the subprocess docs instead --- Doc/library/subprocess.rst | 9 +++++++++ Doc/library/venv.rst | 12 ------------ 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index e37cc980e97575..a6233f4bb1b202 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -356,6 +356,15 @@ functions. arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass *args* as a sequence. + .. warning:: + + Using the string ``"python"`` as the program to execute is not a good + idea, as there are a number of circumstances where the Python interpreter + may not be visible under that name. Instead, use :data:`sys.executable` + if you want to start a copy of the current interpreter, or use + :meth:`shutil.which` if you want to find the Python interpreter by name + using the shell's current search path. + An example of passing some arguments to an external program as a sequence is:: diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst index be9e2ea6cf08aa..5d4a36481f1dcc 100644 --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -84,18 +84,6 @@ Creating virtual environments without there needing to be any reference to its virtual environment in ``PATH``. - On Windows, the actual Python interpreter running your script is the *base* - interpreter, rather than the executable in the virtual environment. This is - normally transparent to your script, as the value of ``sys.executable`` is - set to the virtual environment's executable. But if you run a subprocess, - Windows searches the directory containing the *actual* executable before - checking ``PATH``. This means that if you run the command ``python`` via - the ``subprocess`` module, Windows will find the base executable rather than - the virtual environment's executable, *even if the virtual environment is - activated*. The fix for this is to use ``sys.executable`` when calling - Python, rather than relying on the system resolving the ``python`` command - to the executable you expect. - .. _venv-api: From 5577cd62c9b28679702fe6d8d925301e359d8a5d Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Mon, 19 Oct 2020 20:39:18 +0100 Subject: [PATCH 3/4] Third attempt to clarify the recommendations --- Doc/library/subprocess.rst | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index a6233f4bb1b202..3a726f4fc26ca0 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -358,12 +358,22 @@ functions. .. warning:: - Using the string ``"python"`` as the program to execute is not a good - idea, as there are a number of circumstances where the Python interpreter - may not be visible under that name. Instead, use :data:`sys.executable` - if you want to start a copy of the current interpreter, or use - :meth:`shutil.which` if you want to find the Python interpreter by name - using the shell's current search path. + For maximum reliability, use a fully-qualified path for the executable. + To search for an unqualified name on :envvar:`PATH`, use + :meth:`shutil.which`. On all platforms, passing :data:`sys.executable` + is the recommended way to launch the current Python interpreter again, + and use the ``-m`` command-line format to launch an installed module. + + Resolving the path of *executable* (or the first item of *args*) is + platform dependent. For POSIX, see :meth:`os.execvpe`, and note that + when resolving or searching for the executable path, *cwd* overrides the + current working directory and *env* can override the ``PATH`` + environment variable. For Windows, see the documentation of the + ``lpApplicationName`` and ``lpCommandLine`` parameters of WinAPI + ``CreateProcess``, and note that when resolving or searching for the + executable path with ``shell=False``, *cwd* does not override the + current working directory and *env* cannot override the ``PATH`` + environment variable. Using a full path avoids all of these variations. An example of passing some arguments to an external program as a sequence is:: From 43c92d404fac631a0765ff09843b3322adff39a1 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Tue, 20 Oct 2020 20:41:18 +0100 Subject: [PATCH 4/4] Minor corrections to other parts of the document --- Doc/library/subprocess.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 3a726f4fc26ca0..54ccb388f7ac25 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -344,7 +344,7 @@ functions. encoding=None, errors=None, text=None) Execute a child program in a new process. On POSIX, the class uses - :meth:`os.execvp`-like behavior to execute the child program. On Windows, + :meth:`os.execvpe`-like behavior to execute the child program. On Windows, the class uses the Windows ``CreateProcess()`` function. The arguments to :class:`Popen` are as follows. @@ -543,7 +543,7 @@ functions. If *cwd* is not ``None``, the function changes the working directory to *cwd* before executing the child. *cwd* can be a string, bytes or - :term:`path-like ` object. In particular, the function + :term:`path-like ` object. In POSIX, the function looks for *executable* (or for the first item in *args*) relative to *cwd* if the executable path is a relative path.