Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

packages_distributions doesn’t work for some dist-info packages built outside Setuptools #330

Closed
flying-sheep opened this issue Jul 12, 2021 · 5 comments

Comments

@flying-sheep
Copy link

@flying-sheep flying-sheep commented Jul 12, 2021

I assume that for those packages, something similar to the following will work to retrieve the top level packages/modules contained in a distribution

{
    f.parts[0] if len(f.parts) > 1 else f.with_suffix('').name
    for f in pkg.files if f.suffix == ".py"
}

Of course there’s a lot of edgecases

@FFY00
Copy link
Member

@FFY00 FFY00 commented Jul 12, 2021

Yeah, because we are looking for top_level.txt, which AFAIK is a setuptools-specific thing. As you point out, the logic you suggest is fragile, so we need a better mechanism. top_level.txt is not defined in any PEP I could find, it's only defined in eggs. I think we may want to propose this to be added as an optional part of the project metadata.

@FFY00
Copy link
Member

@FFY00 FFY00 commented Jul 12, 2021

FYI, the original discussion is in #131.

@jaraco
Copy link
Member

@jaraco jaraco commented Aug 14, 2021

It works for me:

~ $ pip-run importlib_metadata requests -- -c "import importlib_metadata as md; print(md.packages_distributions()['requests'])"
Collecting importlib_metadata
  Using cached importlib_metadata-4.6.4-py3-none-any.whl (17 kB)
Collecting requests
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting zipp>=0.5
  Using cached zipp-3.5.0-py3-none-any.whl (5.7 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Installing collected packages: zipp, urllib3, idna, charset-normalizer, certifi, requests, importlib-metadata
Successfully installed certifi-2021.5.30 charset-normalizer-2.0.4 idna-3.2 importlib-metadata-4.6.4 requests-2.26.0 urllib3-1.26.6 zipp-3.5.0
['requests']

The title says "some dist-info packages". Can you provide maybe one example that doesn't work?

@FFY00
Copy link
Member

@FFY00 FFY00 commented Aug 14, 2021

Non-setuptools ones, see my comment above.

$ pip-run importlib_metadata hidabc -- -c "import importlib_metadata as md; print(md.packages_distributions()['hidabc'])"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
KeyError: 'hidabc'

@jaraco jaraco changed the title packages_distributions doesn’t work for some dist-info packages installed from wheels packages_distributions doesn’t work for some dist-info packages built outside Setuptools Aug 14, 2021
@jaraco
Copy link
Member

@jaraco jaraco commented Aug 14, 2021

In #340, I've drafted a change that implements the proposed approach above as a fallback when top-level isn't available.

@jaraco jaraco closed this in 1a8e808 Aug 26, 2021
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Oct 13, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
splitice pushed a commit to HalleyAssist/poky that referenced this issue Oct 13, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 01eb9d4384ae78b02780cea3b8690d99484b2602)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
splitice pushed a commit to HalleyAssist/poky that referenced this issue Oct 13, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 01eb9d4384ae78b02780cea3b8690d99484b2602)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
kraj pushed a commit to YoeDistro/poky that referenced this issue Oct 13, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 01eb9d4384ae78b02780cea3b8690d99484b2602)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
armcc pushed a commit to lgirdk/poky that referenced this issue Oct 13, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 01eb9d4384ae78b02780cea3b8690d99484b2602)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
splitice pushed a commit to HalleyAssist/poky that referenced this issue Oct 14, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 01eb9d4384ae78b02780cea3b8690d99484b2602)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
splitice pushed a commit to HalleyAssist/poky that referenced this issue Oct 14, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 01eb9d4384ae78b02780cea3b8690d99484b2602)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
splitice pushed a commit to HalleyAssist/poky that referenced this issue Oct 14, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 4272ca45d137b91ec368c94b3e0dbd7d56c616dd)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Oct 14, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Oct 14, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
seambot pushed a commit to hello-seam/poky that referenced this issue Oct 14, 2021
v4.8.1
  #348: Restored support for EntryPoint access by item, deprecating
        support in the process. Users are advised to use direct member
        access instead of item-based access:

    - ep[0] -> ep.name
    - ep[1] -> ep.value
    - ep[2] -> ep.group
    - ep[:] -> ep.name, ep.value, ep.group

v4.8.0
  #337: Rewrote EntryPoint as a simple class, still immutable and
        still with the attributes, but without any expectation for
        namedtuple functionality such as _asdict.

v4.7.1
  #344: Fixed regression in packages_distributions when neither
        top-level.txt nor a files manifest is present.

v4.7.0
  #330: In packages_distributions, now infer top-level names from
        .files() when a top-level.txt (Setuptools-specific metadata)
        is not present.

References:
  python/importlib_metadata#348
  python/importlib_metadata#337
  python/importlib_metadata#344
  python/importlib_metadata#330

(From OE-Core rev: 21d72ace8f9486bd1b478e28d53da64087d790fa)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants