Mismatching importlib-resources and importlib_resources

[bollwyvl]

Over on conda-forge/jupyter-book-feedstock#10 , seeing a fail when an env resolves with two versions of the twins:
logs

    importlib-resources:           3.3.1-pyhd8ed1ab_0        conda-forge
    importlib_resources:           5.4.0-pyhd8ed1ab_0        conda-forge

whcih then yields a bunch of clobber warnings.

I guess I'll pin both for now, but seems potentially problematic.

[bollwyvl]

In that specific case it looks like this upstream will fix it:

conda-forge/sphinx-book-theme-feedstock#24

[ocefpaf]

We probably need to move to broken any importlib(-/_)resources that predates the multiple output in the feedstock.

[moorepants]

Reported here: conda/conda#11270 also.

[moorepants]

We probably need to move to broken any importlib(-/_)resources that predates the multiple output in the feedstock.

What is the recommended way to purge the - or _ version from your environment?

[moorepants]

I did mamba repoquery whoneeds importlib-resources and get:

Executing the query importlib-resources



 Name              Version Build        Depends                        Channel
───────────────────────────────────────────────────────────────────────────────
 sphinx-book-theme 0.1.10  pyhd8ed1ab_0 importlib-resources >=3.0,<3.5  

So it seems the dud version with the - is tied to sphinx-book-theme.

Conda couldn't remove sphinx-book-theme or importlib-resources from my env (timed out), but mamba was able to.

I'm still getting the error with conda env list and this is still present:

moorepants@nandi:~$ conda list | grep importlib
importlib-metadata        4.11.2           py37h89c1867_0    conda-forge
importlib_metadata        4.11.2               hd8ed1ab_0    conda-forge
importlib_resources       5.4.0              pyhd8ed1ab_0    conda-forge
moorepants@nandi:~$ mamba repoquery whoneeds importlib-metadata

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.15.3) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Executing the query importlib-metadata



 Name               Version Build          Depends                                 Channel
───────────────────────────────────────────────────────────────────────────────────────────
 alembic            1.7.6   pyhd8ed1ab_0   importlib-metadata                             
 pytest             7.0.1   py37h89c1867_0 importlib-metadata >=0.12                      
 importlib_metadata 4.11.2  hd8ed1ab_0     importlib-metadata >=4.11.2,<4.11.3.0a0        
 pyppeteer          1.0.2   pyhd8ed1ab_0   importlib-metadata >=1.4                       
 flake8             4.0.1   pyhd8ed1ab_0   importlib-metadata                             
 sphinx             4.4.0   pyh6c4a22f_1   importlib-metadata >=4.4                       
 jsonschema         4.4.0   pyhd8ed1ab_0   importlib-metadata                             

moorepants@nandi:~$ mamba repoquery whoneeds importlib_metadata

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.15.3) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Executing the query importlib_metadata



 Name    Version Build          Depends                   Channel
──────────────────────────────────────────────────────────────────
 pluggy  1.0.0   py37h89c1867_2 importlib_metadata >=0.12        
 pyface  7.3.0   pyh44b312d_1   importlib_metadata               
 twine   3.8.0   pyhd8ed1ab_0   importlib_metadata >=3.6         
 keyring 23.4.0  py37h89c1867_2 importlib_metadata >=3.6         

[moorepants]

Notice that the feedstock for alembic depends on importlib-metadata instead of importlib_metadata:
https://github.com/conda-forge/alembic-feedstock/blob/master/recipe/meta.yaml

Should we open prs to change all the feedstocks from importlib-metadata to importlib_metadata?

[bollwyvl]

importlib(-|_)metadata isn't likely to be causing any issues (as pertains to this ticket), as those all have effective co-dependency between them: one is just a metapackage.

[moorepants]

I managed to remove importlib-resources but I still get the error with jsonschema==4.4.0 (works fine with jsonschema==3.2.0):

moorepants@nandi:~$ conda list | grep "importlib\|jsonschema\|nbformat\|^conda"
conda                     4.11.0           py37h89c1867_0    conda-forge
conda-build               3.21.8           py37h89c1867_0    conda-forge
conda-package-handling    1.7.3            py37h5e8e339_1    conda-forge
importlib-metadata        4.11.2           py37h89c1867_0    conda-forge
importlib_metadata        4.11.2               hd8ed1ab_0    conda-forge
importlib_resources       5.4.0              pyhd8ed1ab_0    conda-forge
jsonschema                4.4.0              pyhd8ed1ab_0    conda-forge
nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
moorepants@nandi:~$ which conda
/home/moorepants/miniconda/bin/conda
moorepants@nandi:~$ conda env list
Traceback (most recent call last):
  File "/home/moorepants/miniconda/bin/conda-env", line 6, in <module>
    from conda_env.cli.main import main
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/conda_env/cli/main.py", line 44, in <module>
    from . import main_create
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/conda_env/cli/main_create.py", line 19, in <module>
    from .. import exceptions, specs
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/conda_env/specs/__init__.py", line 9, in <module>
    from .notebook import NotebookSpec
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/conda_env/specs/notebook.py", line 5, in <module>
    import nbformat
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/nbformat/__init__.py", line 32, in <module>
    from .validator import validate, ValidationError
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/nbformat/validator.py", line 12, in <module>
    from .json_compat import get_current_validator, ValidationError
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/nbformat/json_compat.py", line 10, in <module>
    import jsonschema
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/jsonschema/__init__.py", line 29, in <module>
    from jsonschema.protocols import Validator
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/jsonschema/protocols.py", line 33, in <module>
    from jsonschema.validators import RefResolver
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/jsonschema/validators.py", line 356, in <module>
    meta_schema=_utils.load_schema("draft3"),
  File "/home/moorepants/miniconda/lib/python3.7/site-packages/jsonschema/_utils.py", line 60, in load_schema
    path = resources.files(__package__).joinpath(f"schemas/{name}.json")
AttributeError: module 'importlib_resources' has no attribute 'files'

[moorepants]

This is odd:

moorepants@nandi:~$ ipython
Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.32.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import importlib_resources

In [2]: importlib_resources.__version__
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-c9a5358378bb> in <module>
----> 1 importlib_resources.__version__

AttributeError: module 'importlib_resources' has no attribute '__version__'

In [3]: dir(importlib_resources)
Out[3]: 
['__doc__',
 '__file__',
 '__loader__',
 '__name__',
 '__package__',
 '__path__',
 '__spec__']
In [4]: importlib_resources.__file__

In [5]: 

[bollwyvl]

yeah, the two packages exist in the same place (usually this would throw a ClobberWarning). When one is removed, it removes all the files it installed (even if it didn't do it last), leaving the difference of the two versions which... isn't very useful, of course. If the correct version was then install -fed, it would likely start working.

[moorepants]

Ok, finally solved.

conda update --force-reinstall importlib_resources leaves me with a working environment. I guess this was all the result of me installing jupyter-book in the base environment. I won't do that again!

[moorepants]

Thanks for the help!