Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

duplicate extensions in asdf 2.14 #1254

Closed
braingram opened this issue Dec 1, 2022 · 2 comments
Closed

duplicate extensions in asdf 2.14 #1254

braingram opened this issue Dec 1, 2022 · 2 comments

Comments

@braingram
Copy link
Contributor

In certain circumstances, weldx.asdf.utils.get_weldx_extension fails due to duplicate extensions: BAMWelDX/weldx#825

Installing a local clone of the weldx repository and running the following while in the repository parent directory produces the error:

import asdf
import weldx

af = asdf.AsdfFile()
ctx = af._create_serialization_context()
weldx.asdf.util.get_weldx_extension(ctx)
  File "/Users/bgraham/projects/221201_weldx_downstream_test/weldx/get_extension.py", line 6, in <module>
    weldx.asdf.util.get_weldx_extension(ctx)
  File "/Users/bgraham/projects/221201_weldx_downstream_test/weldx/weldx/asdf/util.py", line 459, in get_weldx_extension
    raise ValueError("Could not determine correct weldx extension.")
ValueError: Could not determine correct weldx extension.

Running the same code outside the weldx repository does not result in an error. There appears to be an issue with importlib:
pypa/setuptools#3649

Forcing the use of importlib-metadata==4.11.4 (installed via pip, tested with python 3.10.6) by modifying these lines fixes the issue:

if sys.version_info < (3, 10):
from importlib_metadata import entry_points
else:
from importlib.metadata import entry_points

It is unclear to me if this is intended behavior of importlib or a bug that per this comment appears to be in the python stdlib pypa/setuptools#3649 (comment)

I'm not sure if other extensions are being duplicated or if this duplication is causing any issues (aside from the one in weldx).

One possible fix would be to put a lower pin on importlib-metadata and to not use the stdlib importlib for python 3.10/3.11.

@marscher
Copy link

marscher commented Dec 6, 2022

Thanks! This precisely describes the problem and the workaround of only relying on one dependency (importlib_metadata) seems like a good approach.

I think the duplication of the extensions is not a problem per se. It would be hard to tell which extension is used then (there is some sorting going on somewhere, but you cannot be sure, which version you'll get). In weldx there is an explicit check, that we only receive one (matching) extension, so the underlying "problem" actually appears.

braingram added a commit to braingram/asdf that referenced this issue Dec 6, 2022
importlib in the standard library and importlib_metadata prior to
4.11.4 library return duplicate entry points in some cases. This
leads to duplication of extensions in entry_points.get_extensions.

remove use of importlib for python >= 3.10 and require
use of importlib_metadata >= 4.11.4

fixes issue asdf-format#1254
braingram added a commit to braingram/asdf that referenced this issue Dec 6, 2022
importlib in the standard library (for python < 3.11) and
importlib_metadata (<4.11.4) return duplicate entry points in some cases.
This leads to duplication of extensions in entry_points.get_extensions.

remove use of importlib for python 3.10 and require
use of importlib_metadata >= 4.11.4 for python < 3.11

fixes issue asdf-format#1254
braingram added a commit to braingram/asdf that referenced this issue Dec 9, 2022
The stdlib importlib.metadata returns duplicate distributions
(and entry points) in some circumstances for all python versions
up to and including 3.11.

python/importlib_metadata#410 (comment)

The above issue can create duplicate asdf extensions.

This fixes issue asdf-format#1254
braingram added a commit to braingram/asdf that referenced this issue Dec 9, 2022
The stdlib importlib.metadata returns duplicate distributions
(and entry points) in some circumstances for all python versions
up to and including 3.11.

python/importlib_metadata#410 (comment)

The above issue can create duplicate asdf extensions.

This fixes issue asdf-format#1254

fix changelog typo
braingram added a commit to braingram/asdf that referenced this issue Dec 9, 2022
The stdlib importlib.metadata returns duplicate distributions
(and entry points) in some circumstances for all python versions
up to and including 3.11.

python/importlib_metadata#410 (comment)

The above issue can create duplicate asdf extensions.

This fixes issue asdf-format#1254

fix changelog typo
braingram added a commit to braingram/asdf that referenced this issue Dec 12, 2022
The stdlib importlib.metadata returns duplicate distributions
(and entry points) in some circumstances for all python versions
up to and including 3.11.

python/importlib_metadata#410 (comment)

The above issue can create duplicate asdf extensions.

This fixes issue asdf-format#1254

fix changelog typo
@braingram
Copy link
Contributor Author

Fixed in #1260

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants