pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType' #626

Torxed · 2023-06-25T17:10:57Z

This might be an issue in my environment, and it might belong in packaging but throwing out a question/bug report here just in case as I drew a judgement call that it's build supplying packaging with information in a potentially wrong way. And just to note if it matters, I'm managing packages via my operating system.

When I manually build pymilter I get the following error:

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
    built = build_call(
            ^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
    out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
    return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
    missing = builder.check_dependencies(distribution)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 370, in check_dependencies
    return {u for d in dependencies for u in check_dependency(d)}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 370, in <setcomp>
    return {u for d in dependencies for u in check_dependency(d)}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 186, in check_dependency
    yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 186, in check_dependency
    yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 186, in check_dependency
    yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 180, in check_dependency
    if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
    item = Version(item)
           ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
    match = self._regex.search(version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'

ERROR expected string or bytes-like object, got 'NoneType'

(The source of build is modified by adding one print just to see what package and/or version was causing the hiccup)
Here, dist.version becomes None for pydantic specifically in this case:

build/src/build/__init__.py

Lines 173 to 179 in cd06da2

    
           try: 
        
               dist = importlib_metadata.distribution(req.name)  # type: ignore[no-untyped-call] 
        
           except importlib_metadata.PackageNotFoundError: 
        
               # dependency is not installed in the environment. 
        
               yield ancestral_req_strings + (normalised_req_string,) 
        
           else: 
        
               if req.specifier and not req.specifier.contains(dist.version, prereleases=True):

It feels either like a bug in pydantic not presenting the version in the right way, or build for passing in unexpected data to packaging or packaging not handling the case. But what if anything is wrong with my environment? And how to debug/correct this, as I most likely have more packages down the line that will produce the same issue.

Steps to potentially reproduce:

anton@bigrigv2 ~/github $ git clone git@github.com:sdgathman/pymilter.git
anton@bigrigv2 ~/github $ cd pymilter/
anton@bigrigv2 ~/github/pymilter (master)$ python -m build --wheel --no-isolation
* Getting build dependencies for wheel...
running egg_info
creating pymilter.egg-info
writing pymilter.egg-info/PKG-INFO
writing dependency_links to pymilter.egg-info/dependency_links.txt
writing top-level names to pymilter.egg-info/top_level.txt
writing manifest file 'pymilter.egg-info/SOURCES.txt'
reading manifest file 'pymilter.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'milter-template.py'
warning: no files found matching 'start.sh'
adding license file 'COPYING'
writing manifest file 'pymilter.egg-info/SOURCES.txt'
wheel version: 0.40.0
setuptools version: 68.0.0
platformdirs version: 3.5.1
jaraco.text version: 3.11.1
jaraco.functools version: 3.7.0
more-itertools version: 9.1.0
jaraco.context version: 4.3.0
autocommand version: 2.2.2
inflect version: 6.0.4
pydantic version: None

My environment:

Python --version = 3.11.3
pydantic.__version__ = '1.10.9'
python-build = 0.10.0-4
python-packaging = 23.1-1
python-installer = 0.7.0-3
Arch Linux = 2023-06-25

The text was updated successfully, but these errors were encountered:

layday · 2023-06-25T18:26:19Z

How did you install Pydantic and where is it located? I'm suspecting you have an old Pydantic dist-info folder lying around that's been emptied but not deleted. Unfortunately importlib.metadata won't raise for IO errors.

Torxed · 2023-06-25T19:16:13Z

How did you install Pydantic

Both with pip and the OS package manager pacman.
I cleaned the environment between attempts just to see if one worked better than the other.

where is it located

>>> import pydantic
>>> pydantic
<module 'pydantic' from '/usr/lib/python3.11/site-packages/pydantic/__init__.cpython-311-x86_64-linux-gnu.so'>

I'm suspecting you have an old Pydantic dist-info folder lying around

I attempted a sudo find / -iname '*pydantic*' but couldn't find anything that stood out.
Only thing that I could find was /usr/lib/python3.11/site-packages/pydantic-1.10.7.dist-info as well as remanence of an old Python installation that got left in the great upgrade from 3.10 to 3.11 a while back: /home/anton/.local/lib/python3.10/site-packages/pydantic as well as /usr/lib/python3.10/site-packages/pydantic-1.9.0.dist-info.

Removed all those but still the same issue.

layday · 2023-06-25T19:20:21Z

You should look at locations on your sys.path. Ensure that there's only one pydantic-*.dist-info folder and that it contains a METADATA file with a valid Version.

Torxed · 2023-06-25T19:34:25Z

You should look at locations on your sys.path. Ensure that there's only one pydantic-*.dist-info folder and that it contains a METADATA file with a valid Version.

Only thing containing the name pydantic would be:

find: ‘/usr/lib/python311.zip’: No such file or directory
/usr/lib/python3.11/site-packages/archinstall/lib/models/pydantic.py
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.pyc
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.opt-1.pyc
/usr/lib/python3.11/site-packages/archinstall/lib/models/pydantic.py
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.pyc
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.opt-1.pyc

I didn't think those findings were worth mentioning, but could they be relevant?

In case my bash sucks (list taken from sys.path):

#!/bin/bash

find /usr/lib/python311.zip -iname '*pydantic*'
find /usr/lib/python3.11 -iname '*pydantic*'
find /usr/lib/python3.11/lib-dynload -iname '*pydantic*'
find /home/anton/.local/lib/python3.11/site-packages -iname '*pydantic*'
find /usr/lib/python3.11/site-packages -iname '*pydantic*'

layday · 2023-06-25T20:02:18Z

Are these your sys.path entries? And are you still able to load the Pydantic distribution? Try running python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic')._path)".

Torxed · 2023-06-25T20:16:58Z

Those were my sys.path yes.
The result from your command when pydantic is not installed is:

anton@bigrigv2 ~ $ python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic')._path)"
Traceback (most recent call last):
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
    return next(cls.discover(name=name))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 981, in distribution
    return Distribution.from_name(distribution_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 565, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for pydantic

While installed it's:

anton@bigrigv2 ~ $ python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic')._path)"
/usr/lib/python3.11/site-packages/pydantic-1.10.9.dist-info

layday · 2023-06-25T20:24:01Z

And python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic').version)" returns the correct version number and build still errors?

Torxed · 2023-06-25T21:43:17Z

and build still errors?

Yepp sadly :/

layday · 2023-06-25T21:53:16Z

Are you certain this is still Pydantic having a None version?

Torxed · 2023-06-25T22:16:32Z

Glad you asked, now it's typing_extensions that's breaking.
Cleared it out as previously instructed and now builds work.

Thanks for the assistance!

Is there anything that could be done to improve packaging or build here? Or to avoid this in the future?
For instance check for empty paths as you mentioned in importlib.metadata or even printing the package that's breaking things to give the user a hint?

Not sure how I even ended up in this state, probably mixing pip and pacman at some point. But it would feel more stable if the user did not have to alter installed libs source to figure out which package is causing the issue. Maybe it's a real one-off, but on the off chance that it something to do with PEP 668 or user error, it would be nice to enhance this area.

layday · 2023-06-25T22:23:17Z

I think it'd be best to address this in importlib.metadata. I opened python/importlib_metadata#457 some time ago, but I'm not sure it'd help in your case. I don't know why importlib.metadata suppresses IO errors; ideally it would error when encountering a dist-info folder wihout METADATA.

In the meantime, build should handle None versions and raise something informative instead of letting the value bubble through to packaging.

Torxed · 2023-06-25T22:25:56Z

Both routes sounds ideal to me.
I'll close this one as it will at best serve a reference for the issue at hand, not really helpful for future development tasks.

I'll also subscribe to your ticket, it seems quite related.

pypa#626

Torxed closed this as completed Jun 25, 2023

Torxed changed the title ~~pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType'~~ pydantic dist.version is None becauses TypeError: expected string or bytes-like object, got 'NoneType' Jun 25, 2023

Torxed changed the title ~~pydantic dist.version is None becauses TypeError: expected string or bytes-like object, got 'NoneType'~~ pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType' Jun 25, 2023

layday mentioned this issue Jul 8, 2023

Raise if multiple dist-info folders for same dist exist python/importlib_metadata#457

Closed

timkpaine added a commit to timkpaine/build that referenced this issue Oct 25, 2023

Add more helpful error message when metadata is malformed, xref pypa#558

0fd1d70

pypa#626

timkpaine mentioned this issue Oct 25, 2023

Add more helpful error message when metadata is malformed #695

Draft

timkpaine added a commit to timkpaine/build that referenced this issue Oct 25, 2023

Add more helpful error message when metadata is malformed, xref pypa#558

c7d4233

pypa#626

timkpaine added a commit to timkpaine/build that referenced this issue Feb 1, 2024

Add more helpful error message when metadata is malformed, xref pypa#558

02f3a81

pypa#626

timkpaine added a commit to timkpaine/build that referenced this issue Feb 1, 2024

Add more helpful error message when metadata is malformed, xref pypa#558

74c60a4

pypa#626

timkpaine added a commit to timkpaine/build that referenced this issue Feb 1, 2024

Add more helpful error message when metadata is malformed, xref pypa#558

5a388a7

pypa#626

timkpaine added a commit to timkpaine/build that referenced this issue Feb 21, 2024

Add more helpful error message when metadata is malformed, xref pypa#558

09f97cf

pypa#626

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType' #626

pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType' #626

Torxed commented Jun 25, 2023 •

edited by layday

Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 •

edited

Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 •

edited

Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 •

edited

Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 •

edited

Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023

pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType' #626

pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType' #626

Comments

Torxed commented Jun 25, 2023 • edited by layday Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 • edited Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 • edited Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 • edited Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023 • edited Loading

layday commented Jun 25, 2023

Torxed commented Jun 25, 2023

Torxed commented Jun 25, 2023 •

edited by layday

Loading

Torxed commented Jun 25, 2023 •

edited

Loading

Torxed commented Jun 25, 2023 •

edited

Loading

Torxed commented Jun 25, 2023 •

edited

Loading

Torxed commented Jun 25, 2023 •

edited

Loading