Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better error message with failing discovery #2493

Open
jthunes opened this issue Feb 1, 2023 · 10 comments
Open

Better error message with failing discovery #2493

jthunes opened this issue Feb 1, 2023 · 10 comments
Labels
enhancement help-wanted

Comments

@jthunes
Copy link

jthunes commented Feb 1, 2023

What's the problem this feature will solve?

pre-commit failed due to StopIteration. Investigation shows that this is caused by an error thrown by virtualenv. See below for full trace when attempting to construct a virtual environment. Through discussion on the virtualenv discord, this closed issue was suggested as related: #2176

Investigation strongly suggested that the issue was indeed related. Pip uninstalling all packages and reinstalling only virtualenv allowed for creation of the virtual environment. pip list before uninstalling shows that there were some horribly out of date packages for the python version used. Removing or updating the packages fixed the issue.

Describe the solution you'd like

While I was able to identify and fix the issue with a bit of help from the virtualenv community, the error was cryptic and did not provide information about what problem was or what steps could be taken to solve it.

I am not familiar with the discovery process, but a message stating which package or path was causing the issue would be helpful. Perhaps:

> python -m virtualenv my_venv
Error: package discovery failed for <packageName>. Please check for package updates.

Alternative Solutions

pip list -o does show the installed and latest version of all packages which was able to help me find which were significantly out of date, but this is a partial solution unless a user knows that other packages are the issue. Adding a note to the Users Guide may be helpful.

This is the current output (with full traceback):

> python -m virtualenv --with-traceback my_venv
Traceback (most recent call last):
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\site-packages\virtualenv\__main__.py", line 69, in <module>
    run_with_catch()  # pragma: no cov
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\site-packages\virtualenv\__main__.py", line 54, in run_with_catch
    run(args, options, env)
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\site-packages\virtualenv\__main__.py", line 16, in run
    session = cli_run(args, options, env)
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\site-packages\virtualenv\run\__init__.py", line 28, in cli_run
    of_session = session_via_cli(args, options, setup_logging, env)
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\site-packages\virtualenv\run\__init__.py", line 46, in session_via_cli
    parser, elements = build_parser(args, options, setup_logging, env)
  File "C:\ANSYSDev\develop\Core_Dependencies\CPython\3_10\winx64\Release\python\lib\site-packages\virtualenv\run\plugin\discovery.py", line 20, in get_discover
    default=next(iter(choices)),
StopIteration
@gaborbernat
Copy link
Contributor

PR welcome.

@jthunes
Copy link
Author

jthunes commented Feb 2, 2023

I'm not familiar with the codebase, but if you've any suggestions I can take a look. I'm honestly not terribly sure what discovery is doing. It appears from a quick glance at run/plugin/discovery.py that it is attempting to add arguments to a specialized argparse parser.

The code is failing on ln20 which sets the default argument for add_argument(). I'm guessing that the only way this would happen would be if choices is an empty list? That is the only reason that I see that next() would throw a StopIteration. That should happen only if the list iterator is exhausted I believe.

I'm not sure what Discovery.entry_points_for("virtualenv.discovery") should be though.

@frenzymadness
Copy link
Contributor

I'm having the same problem when packaging the latest version for Fedora Linux. tests/unit/config/test___main__.py::test_main failed so I've tried to run the same command manually with this result:

# PYTHONPATH=/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/ python3 -m virtualenv --with-traceback venv
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/virtualenv/__main__.py", line 69, in <module>
    run_with_catch()  # pragma: no cov
    ^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/virtualenv/__main__.py", line 54, in run_with_catch
    run(args, options, env)
  File "/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/virtualenv/__main__.py", line 16, in run
    session = cli_run(args, options, env)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/virtualenv/run/__init__.py", line 28, in cli_run
    of_session = session_via_cli(args, options, setup_logging, env)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/virtualenv/run/__init__.py", line 46, in session_via_cli
    parser, elements = build_parser(args, options, setup_logging, env)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/virtualenv/run/__init__.py", line 67, in build_parser
    discover = get_discover(parser, args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/virtualenv/run/plugin/discovery.py", line 20, in get_discover
    default=next(iter(choices)),
            ^^^^^^^^^^^^^^^^^^^
StopIteration

I've tried to load entry points manually and this is the result:

# PYTHONPATH=/builddir/build/BUILDROOT/python-virtualenv-20.19.0-1.fc39.x86_64/usr/lib/python3.11/site-packages/ python3
Python 3.11.1 (main, Jan 20 2023, 00:00:00) [GCC 13.0.1 20230117 (Red Hat 13.0.1-0)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib_metadata as im
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'importlib_metadata'
>>> import importlib.metadata as im
>>> im.distribution("virtualenv")
<importlib.metadata.PathDistribution object at 0x7f5bcbf7e0d0>
>>> dist = im.distribution("virtualenv")
>>> dist.entry_points
[]

Could you please help me understand where the problem might be?

@gaborbernat
Copy link
Contributor

That should never happen because we register at least once discovery entry here https://github.com/pypa/virtualenv/blob/main/pyproject.toml#L95, so the only way for that to happen is a broken packaging. We moved over the hatchling a while back from setuptools - https://github.com/pypa/virtualenv/blob/main/pyproject.toml#L3, so perhaps you did not change the build dependencies?

@gaborbernat
Copy link
Contributor

That change happened a month ago here #2474

@frenzymadness
Copy link
Contributor

So, the problem in my case was that the build backend creates a .dist-info directory when you use the hook to get build dependencies, and then importlib.metadata loaded the directory in CWD instead of the one in site-packages and therefore the list of entry points was empty.

@gaborbernat
Copy link
Contributor

build backend creates a .dist-info directory

I don't think hatchling does such a thing; this is some setuptools legacy thing.

@frenzymadness
Copy link
Contributor

I don't think hatchling does such a thing; this is some setuptools legacy thing.

No, it's not, see https://github.com/pypa/hatch/blob/a122f0cc022c428418b9afe7acd10c5cbcb3157d/backend/src/hatchling/build.py#L97-L112

@hroncok hroncok mentioned this issue Mar 3, 2023
Open
@gaborbernat
Copy link
Contributor

@frenzymadness any ideas how to fix it? 😆

@frenzymadness
Copy link
Contributor

What about just having a better error message (something about broken installation maybe) when the list of loaded entry points is empty?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement help-wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@gaborbernat @frenzymadness @jthunes and others