Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace pkg_resources #7943

Merged
merged 18 commits into from Sep 21, 2023
Merged

Replace pkg_resources #7943

merged 18 commits into from Sep 21, 2023

Commits on Sep 19, 2023

  1. compat: resolve importlib.metadata vs importlib_metadata

    Import `importlib.metadata` as `importlib_metadata` if stdlib
    version is recent enough (python >= 3.10). Otherwise import
    the `importlib_metadata`, but explicitly verify the version
    of `importlib-metadata` dist, which must be >= 4.6. This
    version is needed for extended functionality, such as `group`
    and `name` keyword arguments having been added to the
    `entry_points()` function.
    
    Add `importlib-metadata >= 4.6` to requirements if python < 3.10.
    
    Add `packaging` >= 20.0 to requirements; used for parsing of
    version and requirements strings.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    98955cf View commit details
    Browse the repository at this point in the history
  2. compat: generalize setup.py hack for missing run-time dependencies

    Replace the `PYINSTALLER_NO_PYWIN32_FAILURE` environment variable
    set by `setup.py` into more general `_PYINSTALLER_SETUP_PY`, which
    now covers the following run-time dependencies in the `compat`
    module:
    * `importlib_metadata` on python < 3.10
    * `pywin32-ctypes` on Windows
    
    This "setup.py mode" prevents `compat` from trying to import
    afore-mentioned run-time dependencies, which may not be present
    when PyInstaller wheels are being built (nor are required for
    the `compat` functionality used by the `setup.py`).
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    4e1b33e View commit details
    Browse the repository at this point in the history
  3. building: use importlib.metadata for discovery of hook directories

    Switch to `importlib.metadata` (or `importlib_metadata`) for entry
    point enumeration during the discovery of hook directories.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    bc4d9c5 View commit details
    Browse the repository at this point in the history
  4. tests: add tests for run-time use of metadata collected from .egg

    Add two test that try to query metadata that was collected from
    (non-zipped) .egg. In such cases, we collect the EGG-INFO directory
    containing the metadata under its original parent directory
    (the package_name-version.egg) to prevent clashes between
    metadata from multiple .egg sources. The first test queries the
    metadata using `pkg_resources`, while the second uses
    `importlib.metadata` / `importlib_metadata`.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    cae4fc3 View commit details
    Browse the repository at this point in the history
  5. bootstrap: add .egg directories in sys._MEIPASS to sys.path

    It seems that `pkg_resources` is able to discover metadata
    collected from eggs as long as the parent directory of the
    .egg directory is in `sys.path`. This is automatically true in
    our case, because we collect the  `package_name-version.egg`
    directories (that contain `EGG-INFO` metadata directories) into
    `sys._MEIPASS`.
    
    In contrast, `importlib.metadata` does not seem to pick up
    such metadata unless the `.egg` directories themselves are
    in `sys.path`. So have the bootstrap script scan for directories
    with `.egg` suffix and append them so `sys.path`.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    19d8829 View commit details
    Browse the repository at this point in the history
  6. hookutils: port metadata collection to importlib.metadata

    Port metadata collection (`copy_metadata` hook utility function)
    from `pkg_resources` to `importlib.metadata` and `packaging`.
    The latter is required for parsing package requirements and
    evaluating markers to exclude the requirements that are not
    applicable to the running python version and/or platform.
    
    Simplify the metadata destination handling code; the only
    special case we need to care about is EGG-INFO directory in
    .egg  directories, where we need to preserve the parent
    directory instead of collecting into top-level application
    directory.
    
    `importlib.metadata` seems to properly resolve the path to
    egg-info files used by Debian/Ubuntu packages, so we can also
    remove the legacy name/path handling.
    
    Have the remaining hook utility functions that make use of
    `pkg_resources` import it locally, so we can port them one-by-one.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    de74438 View commit details
    Browse the repository at this point in the history
  7. hookutils: port get_installer to importlib.metadata

    Port the `get_installer` helper from `pkg_resources` to
    `importlib.metadata`. Requires python >= 3.10 or equivalent
    `importlib-metadata` (>= 4.6) due to use of the
    `packages_distributions()` convenience function.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    c423706 View commit details
    Browse the repository at this point in the history
  8. analysis: use importlib.metadata in error handling in _metadata_from

    Since the `copy_metadata` calls in `_metadata_from` now use
    `importlib.metadata` instead of `pkg_resources`, we now need
    to handle the `importlib.metadata.PackageNotFoundError` instead
    of `pkg_resources.PackageNotFoundError`.
    
    Use `compat.importlib_metadata` to properly resolve
    `importlib.metadata` vs `importlib_metadata` regardless if the
    latter is available on python > 3.10, where our `compat` prefers
    the stdlib version.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    6ff661a View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    7889dbe View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    7325e7a View commit details
    Browse the repository at this point in the history
  11. modulegraph: remove the modulegraph.replacePackage function

    The only use for `modulegraph.replacePackage` is for the `_xmplus`
    module from python2-era PyXML package. Remove the function and
    associated code.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    fcb521d View commit details
    Browse the repository at this point in the history
  12. modulegraph: remove parsing of setuptools' *-nspkg.pth files

    In contemporary python, namespace packages should be handled without
    having to resort to parsing the *-nspkg.pth files.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    43e9639 View commit details
    Browse the repository at this point in the history
  13. modulegraph: rework support for legacy setuptools namespace packages

    Rework the support for legacy setuptools-based namespace packages
    and port it to `importlib.metadata`.
    
    The legacy namespace packages are regular packages with `__init__.py`
    that calls deprecated  `pkg_resources.declare_namespace(__name__)`.
    
    We could get rid of this altogether once we are fine with not
    supporting legacy namespace packages that are split across
    multiple locations. Currently, our test suite contains such an
    example (e.g., `test_import.py::test_nspkg1`). The code has been
    refactored so that dropping support will also be easier.
    
    To properly support legacy namespace sub-packages that are split
    across different locations, we need to scan the metadata of all
    dists, similarly to how it was done by the original `pkg_resources`
    based implementation. However, within the life-time of a module
    graph instance, we can perform the scan only once, and cache the
    results.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    88b9769 View commit details
    Browse the repository at this point in the history
  14. utils: run_tests: use importlib.metadata to discover tests

    Port the entry-point scan for test discovery from `pkg_resources`
    to `importlib.metadata`.
    rokm committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    44fb411 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. hookutils: implement replacement for is_module_satisfies

    Replace `is_module_satisfies` with `check_requirement`, which
    uses `packaging` to parse the requirement and check the version
    against the optional specifier(s), and `importlib.metadata` to
    query the dist version.
    
    Keep `is_module_satisfies` as an alias for `check_requirements`
    to maintain compatibility with existing hooks (from the contrib
    hooks repository) that call the function with only the
    `requirement` string. The calls with additional `version` and
    `version_attr` arguments will not work anymore, nor will the
    fall back to checking module's version when the specified
    requirement is not satisfied.
    
    Have the @requires test decorator use `check_requirement`.
    
    Update hooks in this repository to use `check_requirement`
    instead of `is_module_satisfies`.
    
    Update Qt hook utilities, where we need to account for the
    fact that `check_requirement` does not attempt to fall back
    to checking module's version (which in this case was
    undesirable and had to be worked around).
    rokm committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    35d303a View commit details
    Browse the repository at this point in the history
  2. hookutils: have collect_all look up the dist from package name

    Have `collect_all` look up the dist name for the given package
    name before attempting to call `copy_metadata` and
    `requirements_for_package`. This fixes the collection (and
    spurious warnings) for cases when dist name does not match the
    importable/package name.
    rokm committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    c7fe105 View commit details
    Browse the repository at this point in the history
  3. hookutils: adjust behavior of collect_data_files with include_py_files

    Adjust the behavior of `collect_data_files` when the `include_py_files`
    flag is enabled: collect only `.py` and `.pyc` files, and never
    collect `.pyc` files from the `__pycache__` directory.
    
    Update the description to note that module collection mode is
    preferable way of ensuring that source .py files are collected
    (in addition or in lieu to byte-compiled modules in PYZ).
    
    Adjust the `test_collect_data_all_included` accordingly.
    rokm committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    701fdb3 View commit details
    Browse the repository at this point in the history
  4. hookutils: remove requirements_for_package

    Remove `requirements_for_package` hook utility, which was
    used by `collect_all`. Consequently, the latter does not include
    the top-level modules of metadata-declared requirements among the
    returned hidden imports anymore.
    rokm committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    86ae9b2 View commit details
    Browse the repository at this point in the history