Try alternate filenames for system_executable

[vfazio]

The value of sys._base_executable may not be a real file due to changes made in CPython 3.11. The value is derived from the current executable name and the "home" key from pyvenv.cfg.

On POSIX systems, virtual environments deploy python for use within the venv however CPython's make install and a number of distributions do not provide a system "python" in part because of PEP 394.

Virtualenv exposes this via PythonInfo.system_executable and can encounter issues when attempting to execute a non-existent file.

Attempt to fallback to python<MAJOR> and python<MAJOR>.<MINOR> if python does not exist.

Thanks for contributing, make sure you address all the checklists (for details on how see development documentation)!

• ran the linter to address style issues (tox -e fix_lint)
• wrote descriptive pull request text
• ensured there are test(s) validating the fix
• added news fragment in docs/changelog folder
• updated/extended the documentation

[vfazio]

I'll add a news entry later but wanted to get this up to get feedback

[gaborbernat]

test too please

[vfazio]

test too please

I haven't quite figured out how i'm going to be able to test this.

[gaborbernat]

exe is already a string we don't need this format, no?

[vfazio]

@gaborbernat you're absolutely correct and this is a good example of why i shouldn't be making commits late at night.

Do you prefer subsequent commits to address comments or can i amend and force push this branch?

[vfazio]

fixed in the latest push

diff --git a/src/virtualenv/discovery/py_info.py b/src/virtualenv/discovery/py_info.py
index a2eb633..2a39e50 100644
--- a/src/virtualenv/discovery/py_info.py
+++ b/src/virtualenv/discovery/py_info.py
@@ -151,12 +151,11 @@ class PythonInfo(object):
                             # search relative to the directory of sys._base_executable
                             base_dir = os.path.dirname(base_executable)
                             for base_executable in [
-                                os.path.join(base_dir, "{}".format(exe))
+                                os.path.join(base_dir, exe)
                                 for exe in ("python{}".format(major), "python{}.{}".format(major, minor))
                             ]:
                                 if os.path.exists(base_executable):
                                     return base_executable
-                        return None  # File doesn't exist and no alternate was available
             return None  # in this case we just can't tell easily without poking around FS and calling them, bail
         # if we're not in a virtual environment, this is already a system python, so return the original executable
         # note we must choose the original and not the pure executable as shim scripts might throw us off

[gaborbernat]

I don't think we need this line as would then go to next line that already does this, no?

[vfazio]

yes, you're correct, i just wasn't sure if that was a convention you were ok with. i'll fix shortly

[vfazio]

fixed in the latest push

diff --git a/src/virtualenv/discovery/py_info.py b/src/virtualenv/discovery/py_info.py
index a2eb633..2a39e50 100644
--- a/src/virtualenv/discovery/py_info.py
+++ b/src/virtualenv/discovery/py_info.py
@@ -151,12 +151,11 @@ class PythonInfo(object):
                             # search relative to the directory of sys._base_executable
                             base_dir = os.path.dirname(base_executable)
                             for base_executable in [
-                                os.path.join(base_dir, "{}".format(exe))
+                                os.path.join(base_dir, exe)
                                 for exe in ("python{}".format(major), "python{}.{}".format(major, minor))
                             ]:
                                 if os.path.exists(base_executable):
                                     return base_executable
-                        return None  # File doesn't exist and no alternate was available
             return None  # in this case we just can't tell easily without poking around FS and calling them, bail
         # if we're not in a virtual environment, this is already a system python, so return the original executable
         # note we must choose the original and not the pure executable as shim scripts might throw us off

[vfazio]

The ubuntu pipelines are also failing here due to nushell

[gaborbernat]

The ubuntu pipelines are also failing here due to nushell

Reached out to them separately, hopefully they fix it soon.

[vfazio]

The ubuntu pipelines are also failing here due to nushell

Reached out to them separately, hopefully they fix it soon.

@gaborbernat
It looks like as part of 0.71 they're now including a subdirectory in their releases archives which is actually pretty typical for most releases i've seen from other projects. 0.70 did not have a subdirectory

vfazio@vfazio2 ~/Downloads :( $ tar -tvf nu-0.70.0-x86_64-unknown-linux-gnu.tar.gz
-rw-r--r-- runner/docker  1094 2022-10-18 14:11 LICENSE
-rw-r--r-- runner/docker   124 2022-10-18 14:11 README.txt
-rwxr-xr-x runner/docker 70621816 2022-10-18 14:11 nu
-rwxr-xr-x runner/docker  2752208 2022-10-18 14:11 nu_plugin_custom_values
-rwxr-xr-x runner/docker  2699672 2022-10-18 14:11 nu_plugin_example
-rwxr-xr-x runner/docker  6006152 2022-10-18 14:11 nu_plugin_gstat
-rwxr-xr-x runner/docker  2130760 2022-10-18 14:11 nu_plugin_inc
-rwxr-xr-x runner/docker  4950368 2022-10-18 14:11 nu_plugin_query

vfazio@vfazio2 ~/Downloads $ tar -tvf nu-0.71.0-x86_64-unknown-linux-gnu.tar.gz 
drwxr-xr-x runner/docker     0 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/
-rw-r--r-- runner/docker  1094 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/LICENSE
-rwxr-xr-x runner/docker 2050928 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/nu_plugin_inc
-rw-r--r-- runner/docker     124 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/README.txt
-rwxr-xr-x runner/docker 65975456 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/nu
-rwxr-xr-x runner/docker  2655216 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/nu_plugin_example
-rwxr-xr-x runner/docker  4827000 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/nu_plugin_query
-rwxr-xr-x runner/docker  2708240 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/nu_plugin_custom_values
-rwxr-xr-x runner/docker  5961760 2022-11-08 12:11 nu-0.71.0-x86_64-unknown-linux-gnu/nu_plugin_gstat

So i think the options are:

• update the pipeline to pull 0.70 to abide by the old release archive structure
• modify the current pipeline script to use --strip-components=1 here https://github.com/pypa/virtualenv/blob/main/.github/workflows/check.yml#L48

[gaborbernat]

• modify the current pipeline script to use --strip-components=1 here https://github.com/pypa/virtualenv/blob/main/.github/workflows/check.yml#L48

Trying this out here https://github.com/pypa/virtualenv/actions/runs/3438815774; seems it does not work 🤔

[vfazio]

• modify the current pipeline script to use --strip-components=1 here https://github.com/pypa/virtualenv/blob/main/.github/workflows/check.yml#L48

Trying this out here https://github.com/pypa/virtualenv/actions/runs/3438815774; seems it does not work thinking

@gaborbernat You have to place the commandline option somewhere else. -f expects a filename immediately after so it's treating --strip-compoments=1 as a filename instead of an argument

https://www.gnu.org/software/tar/manual/html_section/transform.html

[vfazio]

Surprisingly, my cpython fix got merged and was backported into 3.11 so this likely won't be as big of an issue in starting in 3.11.1 and 3.12.

It's still worthwhile to perform this check because cpython will still return a base executable that may not exist. The primary fix for that would be #2441 to ensure it searches in the right path. The code introduced here still only applies to 3.11 going forward and based on our discussion should still return None if the file does not exist.

[vfazio]

Surprisingly, my cpython fix got merged and was backported into 3.11 so this likely won't be as big of an issue in starting in 3.11.1 and 3.12.

It's still worthwhile to perform this check because cpython will still return a base executable that may not exist. The primary fix for that would be #2441 to ensure it searches in the right path. The code introduced here still only applies to 3.11 going forward and based on our discussion should still return None if the file does not exist.