Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix editable install finder handling of nested packages #4020

Merged
merged 5 commits into from Aug 18, 2023

Conversation

abravalheri
Copy link
Contributor

Summary of changes

  • Only handle 1 level of nesting (necessary for the namespace packages to work properly)
  • Rely on the importlib.machinery to find other levels of nested packages to find the correct files based on the parent path.

Closes #4019

Pull Request Checklist

@abravalheri
Copy link
Contributor Author

abravalheri commented Aug 17, 2023

Docs likely to be breaking due to readthedocs/sphinx-notfound-page#219.
Temporary solution => disable extension.

@abravalheri
Copy link
Contributor Author

There are still some errors with Sphinx. E.g. now it raises exceptions due to cyclic imports in setuptools code - which probably need further investigation:

CI logs
WARNING: autodoc: failed to import class 'Extension' from module 'setuptools'; the following exception was raised:
Traceback (most recent call last):
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 64, in import_module
    return importlib.import_module(modname)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 6[88](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:89), in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/setuptools/setuptools/setuptools/__init__.py", line 17, in <module>
    from setuptools.dist import Distribution
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/_virtualenv.py", line [90](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:91), in exec_module
    old(module)
  File "/home/runner/work/setuptools/setuptools/setuptools/dist.py", line 37, in <module>
    from setuptools.config import setupcfg, pyprojecttoml
  File "/home/runner/work/setuptools/setuptools/setuptools/config/__init__.py", line 8, in <module>
    from . import setupcfg
  File "/home/runner/work/setuptools/setuptools/setuptools/config/setupcfg.py", line 38, in <module>
    from . import expand
  File "/home/runner/work/setuptools/setuptools/setuptools/config/expand.py", line 53, in <module>
    from setuptools.dist import Distribution  # noqa
ImportError: cannot import name 'Distribution' from partially initialized module 'setuptools.dist' (most likely due to a circular import) (/home/runner/work/setuptools/setuptools/setuptools/dist.py)

The above exception was the direct cause of the following exception:

reading sources... [ [92](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:93)%] userguide/index
reading sources... [ [94](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:95)%] userguide/miscellaneous
reading sources... [ [96](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:97)%] userguide/package_discovery
reading sources... [ [98](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:99)%] userguide/pyproject_config
reading sources... [[100](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:101)%] userguide/quickstart
Traceback (most recent call last):
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 88, in import_object
    module = import_module(modname, warningiserror=warningiserror)
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 68, in import_module
    raise ImportError(exc, traceback.format_exc()) from exc
looking for now-outdated files... none found
ImportError: (ImportError("cannot import name 'Distribution' from partially initialized module 'setuptools.dist' (most likely due to a circular import) (/home/runner/work/setuptools/setuptools/setuptools/dist.py)"), 'Traceback (most recent call last):\n  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 64, in import_module\n    return importlib.import_module(modname)\n  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import\n  File "<frozen importlib._bootstrap>", line [102](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:103)7, in _find_and_load\n  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked\n  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked\n  File "<frozen importlib._bootstrap_external>", line 883, in exec_module\n  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed\n  File "/home/runner/work/setuptools/setuptools/setuptools/__init__.py", line 17, in <module>\n    from setuptools.dist import Distribution\n  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/_virtualenv.py", line 90, in exec_module\n    old(module)\n  File "/home/runner/work/setuptools/setuptools/setuptools/dist.py", line 37, in <module>\n    from setuptools.config import setupcfg, pyprojecttoml\n  File "/home/runner/work/setuptools/setuptools/setuptools/config/__init__.py", line 8, in <module>\n    from . import setupcfg\n  File "/home/runner/work/setuptools/setuptools/setuptools/config/setupcfg.py", line 38, in <module>\n    from . import expand\n  File "/home/runner/work/setuptools/setuptools/setuptools/config/expand.py", line 53, in <module>\n    from setuptools.dist import Distribution  # noqa\nImportError: cannot import name \'Distribution\' from partially initialized module \'setuptools.dist\' (most likely due to a circular import) (/home/runner/work/setuptools/setuptools/setuptools/dist.py)\n')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 64, in import_module
    return importlib.import_module(modname)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line [105](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:106)0, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/setuptools/setuptools/setuptools/__init__.py", line 35, in <module>
    __version__ = setuptools.version.__version__
AttributeError: partially initialized module 'setuptools' has no attribute 'version' (most likely due to a circular import)

WARNING: autodoc: failed to import class 'Command' from module 'setuptools'; the following exception was raised:
Traceback (most recent call last):
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 64, in import_module
    return importlib.import_module(modname)
  File "/usr/lib/python3.10/importlib/__init__.py", line [126](https://github.com/pypa/setuptools/actions/runs/5900146250/job/16003850907?pr=4020#step:5:127), in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/setuptools/setuptools/setuptools/__init__.py", line 35, in <module>
    __version__ = setuptools.version.__version__
AttributeError: partially initialized module 'setuptools' has no attribute 'version' (most likely due to a circular import)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 88, in import_object
    module = import_module(modname, warningiserror=warningiserror)
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 68, in import_module
    raise ImportError(exc, traceback.format_exc()) from exc
ImportError: (AttributeError("partially initialized module 'setuptools' has no attribute 'version' (most likely due to a circular import)"), 'Traceback (most recent call last):\n  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 64, in import_module\n    return importlib.import_module(modname)\n  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import\n  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load\n  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked\n  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked\n  File "<frozen importlib._bootstrap_external>", line 883, in exec_module\n  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed\n  File "/home/runner/work/setuptools/setuptools/setuptools/__init__.py", line 35, in <module>\n    __version__ = setuptools.version.__version__\nAttributeError: partially initialized module \'setuptools\' has no attribute \'version\' (most likely due to a circular import)\n')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/setuptools/setuptools/.tox/docs/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 64, in import_module
    return importlib.import_module(modname)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/setuptools/setuptools/setuptools/__init__.py", line 35, in <module>
    __version__ = setuptools.version.__version__
AttributeError: partially initialized module 'setuptools' has no attribute 'version' (most likely due to a circular import)

/home/runner/work/setuptools/setuptools/docs/userguide/ext_modules.rst:5: WARNING: py:class reference target not found: setuptools.Extension

This is likely to be caused by sphinx-doc/sphinx#11608.

I think I will go ahead and pin sphinx<=7.1.2, because this PR is supposed to be a bugfix for a specific issue and there might be people depending on it. Investigation about cyclic dependencies will take more time and hold the process.

@abravalheri abravalheri marked this pull request as ready for review August 18, 2023 08:54
@abravalheri abravalheri merged commit 4d6f44b into pypa:main Aug 18, 2023
22 checks passed
@abravalheri abravalheri deleted the issue-4019 branch August 18, 2023 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

editable_wheel is broken in 68.1.0
1 participant