VirtualEnv fails with Homebrewed Python 3.9 on macOS Big Sur

[arcivanov]

Issue

PyBuilder bundles VirtualEnv verbatim (with relativized imports). One of our users is reporting that trying to create a VirtualEnv from Homebrew'ed version of Python 3.9 fails within VirtualEnv when trying to create a venv based on that Python install:

pybuilder/pybuilder#771 (comment)

While I fully understand the impulse to close this as a "bundled dependency issue", PyBuilder tests on Linux, Windows and macOS using both system and virtualenv as source Python environments but not on Homebrew. And in all our tests the bundled VirtualEnv works like a charm. Except, as it appears, with Homebrew'ed one. The failure occurs in the VirtualEnv's code.

The currently bundled VirtualEnv is 20.7.2

Environment

Provide at least:

• OS: macOS Big Sur

$ pyb -vX
[DEBUG] Registered Python environment 'system': CPython version 3.9.6.final.0 on darwin in ['/usr/local/opt/python@3.9/bin/python3.9']
PyBuilder version 0.13.0.dev20210905004524
Build started at 2021-09-06 08:54:14
------------------------------------------------------------
[DEBUG] Loading project module from ./build.py
[DEBUG] Setting up plugins VEnv at '/Users/username/XXX/.pybuilder/plugins/cpython-3.9.6.final.0'
Traceback (most recent call last):
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/cli.py", line 461, in main
    reactor.prepare_build(property_overrides=options.property_overrides,
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/reactor.py", line 203, in prepare_build
    self._setup_plugin_directory(reset_plugins, no_venvs)
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/reactor.py", line 517, in _setup_plugin_directory
    plugin_env = per["pybuilder"] = PythonEnv(plugin_dir, self).create_venv(with_pip=True,
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/python_env.py", line 214, in create_venv
    create_venv(self._env_dir,
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/python_env.py", line 453, in create_venv
    virtualenv.cli_run(args, setup_logging=False)
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/_vendor/virtualenv/run/__init__.py", line 30, in cli_run
    of_session = session_via_cli(args, options, setup_logging, env)
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/_vendor/virtualenv/run/__init__.py", line 48, in session_via_cli
    parser, elements = build_parser(args, options, setup_logging, env)
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/_vendor/virtualenv/run/__init__.py", line 69, in build_parser
    discover = get_discover(parser, args)
  File "/Users/username/Library/Python/3.9/lib/python/site-packages/pybuilder/_vendor/virtualenv/run/plugin/discovery.py", line 22, in get_discover
    default=next(iter(choices)),
StopIteration
------------------------------------------------------------
BUILD FAILED - StopIteration:  (virtualenv/run/plugin/discovery.py:22)
------------------------------------------------------------
Build finished at 2021-09-06 08:54:14
Build took 0 seconds (129 ms)

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

I will ask the user to provide the output and will update.

[arcivanov]

I stand corrected, it's likely a vendoring issue due to Homebrew pathing. I'll reopen if it's not the case.

[gaborbernat]

Yeah, was just about to say that the stack points no entrypoints being discovered for virtualenv.discovery - https://github.com/pypa/virtualenv/blob/main/src/virtualenv/run/plugin/discovery.py#L11 So someone messes up that part.

[arcivanov]

It looks like there may be an issue with determining the proper prefix when there is a chain of symlinks, but that's within the Python itself, cause __file__-based derivatives produce weird paths on Homebrewed OSX.