'uri_template' distribution was not found and is required by jsonschema with newer versions of jsonschema

[TheChymera]

Should we add a dependency constraint for jsonschema?
I get the following error with =jsonschema-4.1.0 but not =jsonschema-3.2.0.

chymera@decohost ~/lulu $ dandi download https://dandiarchive.org/dandiset/000005/draft
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.9/dandi", line 33, in <module>
    sys.exit(load_entry_point('dandi==0+unknown', 'console_scripts', 'dandi')())
  File "/usr/lib/python-exec/python3.9/dandi", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/lib/python3.9/site-packages/dandi/cli/command.py", line 18, in <module>
    from ..utils import get_module_version
  File "/usr/lib/python3.9/site-packages/dandi/utils.py", line 26, in <module>
    import ruamel.yaml
  File "/usr/lib/python3.9/site-packages/ruamel/__init__.py", line 1, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3243, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3226, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3255, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 568, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 886, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 772, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'uri_template' distribution was not found and is required by jsonschema

[yarikoptic]

Just to make sure: how dandi-cli was installed?

[yarikoptic]

Afaik https://github.com/dandi/dandischema/blob/master/setup.cfg#L30 should have taken care about installing those dependencies but may be something changed in dandi schema extra depends recently - didn't check yet

[TheChymera]

@yarikoptic it's installed via the system package manager, which has been distributing up-to-date versions of dandi since October. With newer versions of jsonschema the dandi CLI no longer fails with:

pkg_resources.DistributionNotFound: The 'uri_template' distribution was not found and is required by jsonschema

but with

pkg_resources.DistributionNotFound: The 'uri-template; extra == "format"' distribution was not found and is required by jsonschema

The current solution we rely on is to use <jsonschema-4 but that's hardly future-proof. I followed up with this and asked upstream, but they seem to think uri-template; extra == "format" is not something their package is supposed to be providing :-/

Any ideas?

[jwodder]

@TheChymera Are you saying that your system package manager is installing jsonschema 4.x when installing dandi? That seems wrong, since hdmf (a dependency of pynwb) requires jsonschema >=2.6.0, <4.

[TheChymera]

@jwodder well, yes, it does, but that's because I told it to. The DANDI stack on Gentoo is maintained by me, so if anything is wrong I can fix it quite rapidly.

I actually checked in with HDMF upstream a while ago (since >=jsonschema-4 has been available for quite a while now), and they agreed that the jsonschema limitation was not needed. Is it that DANDI (or pynwb) expects HDMF to be using <jsonschema-4 even though HDMF works fine with newer versions?

[jwodder]

@TheChymera I don't know about pynwb, but dandi doesn't care. However, dandischema requires jsonschema[format] (no version restrictions specified), so if you're installing jsonschema 4.x alongside dandi & dandischema, you'll need to install all of the dependencies in its format extra. As of jsonschema 4.4.0, that includes uri-template — note the hyphen, so you want https://github.com/plinss/uri_template/ instead of https://github.com/python-hyper/uritemplate.

[TheChymera]

@jwodder thanks again for clarifying this. It was indeed a name mix-up after the jsonschema update. Solved as of: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e7963135d008f2a75f05b51d2ed4dcde078f5fcc