Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Argument 'bases' has incorrect type #7689

Open
5 of 15 tasks
joostlek opened this issue Sep 28, 2023 · 24 comments
Open
5 of 15 tasks

Argument 'bases' has incorrect type #7689

joostlek opened this issue Sep 28, 2023 · 24 comments
Assignees
Labels
bug V1 Bug related to Pydantic V1.X

Comments

@joostlek
Copy link

Initial Checks

  • I have searched GitHub for a duplicate issue and I'm sure this is something new
  • I have searched Google & StackOverflow for a solution and couldn't find anything
  • I have read and followed the docs and still think this is a bug
  • I am confident that the issue is with pydantic (not my code, or another library in the ecosystem like FastAPI or mypy)

Description

So last night 1.10.13 got released. And apparently it breaks on runtime,

The context is that there are some integrations in HomeAssistant that use pydantic, and they all seem to raise this issue when 1.10.13 is installed. This isn't raised in 1.10.12 (home-assistant/core#101044). Linked issue on the HA repo (home-assistant/core#101042).

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/src/homeassistant/homeassistant/components/radio_browser/__init__.py", line 5, in <module>
    from radios import RadioBrowser, RadioBrowserError
  File "/usr/local/lib/python3.11/site-packages/radios/__init__.py", line 8, in <module>
    from .models import Country, Language, Station, Stats, Tag
  File "/usr/local/lib/python3.11/site-packages/radios/models.py", line 12, in <module>
    class Stats(BaseModel):
  File "pydantic/main.py", line 186, in pydantic.main.ModelMetaclass.__new__
TypeError: Argument 'bases' has incorrect type (expected list, got tuple)

Some of the libs used are https://github.com/frenck/python-vehicle and https://github.com/frenck/python-radios

Example Code

No response

Python, Pydantic & OS Version

Python 3.11
Pydantic 1.10.13
Linux

Affected Components

@joostlek joostlek added bug V1 Bug related to Pydantic V1.X pending Awaiting a response / confirmation labels Sep 28, 2023
@Kludex
Copy link
Member

Kludex commented Sep 28, 2023

Do you have a minimal, reproducible example? We didn't change much: v1.10.12...v1.10.13

@joostlek
Copy link
Author

joostlek commented Sep 28, 2023

I will give it a shot. As context, these are the libs in HA:

Warning!! Cyclic dependencies found:
* gps3 => gps3
------------------------------------------------------------------------
pydantic==1.10.13
├── aiolivisi==0.0.19 [requires: pydantic]
├── aionotion==2023.5.5 [requires: pydantic>=1.10.7,<2.0.0]
├── aioopenexchangerates==0.4.0 [requires: pydantic>=1.9,<2.0]
├── aiopurpleair==2022.12.1 [requires: pydantic>=1.10.2,<2.0.0]
├── aiowaqi==0.2.1 [requires: pydantic>=1.10.8]
├── demetriek==0.4.0 [requires: pydantic>=1.9.0,<2.0.0]
├── elgato==4.0.1 [requires: pydantic>=1.8.0,<2.0.0]
├── gcal-sync==4.1.4 [requires: pydantic>=1.9.0,<2.0a]
├── google-nest-sdm==3.0.2 [requires: pydantic>=1.10.4]
├── ical==5.0.1 [requires: pydantic>=1.9.1]
│   ├── gcal-sync==4.1.4 [requires: ical>=4.2.5]
│   └── pyrainbird==4.0.0 [requires: ical>=4.2.9]
├── inflect==7.0.0 [requires: pydantic>=1.9.1]
│   ├── jaraco.itertools==6.4.1 [requires: inflect]
│   │   └── jaraco.abode==3.3.0 [requires: jaraco.itertools]
│   └── jaraco.text==3.11.1 [requires: inflect]
│       ├── jaraco.collections==4.3.0 [requires: jaraco.text]
│       │   ├── jaraco.abode==3.3.0 [requires: jaraco.collections]
│       │   ├── jaraco.email==3.1.0 [requires: jaraco.collections]
│       │   │   └── jaraco.net==9.3.1 [requires: jaraco.email]
│       │   │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       │   └── jaraco.net==9.3.1 [requires: jaraco.collections]
│       │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       ├── jaraco.email==3.1.0 [requires: jaraco.text>=1.3]
│       │   └── jaraco.net==9.3.1 [requires: jaraco.email]
│       │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       └── jaraco.net==9.3.1 [requires: jaraco.text]
│           └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
├── intellifire4py==2.2.2 [requires: pydantic]
├── lacrosse-view==1.0.1 [requires: pydantic>=1.9.0]
├── open-meteo==0.2.1 [requires: pydantic>=1.8.0,<2.0.0]
├── peco==0.0.29 [requires: pydantic>=1.9.0]
├── pvo==1.0.0 [requires: pydantic>=1.8.0,<2.0.0]
├── pyaussiebb==0.0.15 [requires: pydantic>=1.9.0,<2.0.0]
├── pyrainbird==4.0.0 [requires: pydantic>=1.10.4]
├── python-bsblan==0.5.16 [requires: pydantic>=1.9.0]
├── python-kasa==0.5.3 [requires: pydantic>=1,<2]
├── python-opensky==0.2.0 [requires: pydantic>=1.10.8]
├── pytraccar==1.0.0 [requires: pydantic>=1,<2]
├── pyunifiprotect==4.20.0 [requires: pydantic!=1.9.1]
├── radios==0.1.1 [requires: pydantic>=1.9,<2.0]
├── sfrbox-api==0.0.6 [requires: pydantic>=1.10.2]
├── systembridgeconnector==3.8.2 [requires: pydantic>=1.9.0]
├── tailscale==0.2.0 [requires: pydantic>=1.8.0,<2.0.0]
├── vehicle==1.0.1 [requires: pydantic>=1.8.0,<2.0.0]
├── withings-api==2.4.0 [requires: pydantic>=1.7.2,<2.0.0]
├── xbox-webapi==2.0.11 [requires: pydantic]
├── yolink-api==0.3.1 [requires: pydantic>=1.9.0]
├── youtubeaio==1.1.5 [requires: pydantic>=1.10.8]
└── zwave-js-server-python==0.52.0 [requires: pydantic>=1.10.0]

We can already confirm from reports that these are failing:

  • zwave-js-server-python
  • pyunifiprotect
  • gcal-sync
  • ical
  • radios
  • vehicle
  • xbox-webapi
  • python-kasa
  • inflect
  • pytraccar
  • google-nest-sdm

I know this isn't an example, but all these libs raise problems when used with 1.10.13, and not with 1.10.12.

I'll get back with an example

@joostlek
Copy link
Author

We also looked at the changelog and were puzzled on why this suddenly fails

@joostlek
Copy link
Author

Apparently running tests of vehicle works on my windows machine with 1.10.13 installed. Also tried python-opensky.

@bdraco
Copy link

bdraco commented Sep 28, 2023

Seems to only affect the systems we have binary wheels for ie i686/x86_64

previous report langchain-ai/langchain#8361

@bdraco
Copy link

bdraco commented Sep 28, 2023

pydantic-1.10.12 doesn't seem to have the problem, but I think they were built with cython 0.29.x

Maybe cython 3.x issue?

@samuelcolvin
Copy link
Member

pydantic v1 should be built with cython 0.29:

Cython==0.29.32;sys_platform!='win32'

@bdraco
Copy link

bdraco commented Sep 28, 2023

I think that won't get used until its in build-system

https://peps.python.org/pep-0518/#build-system-table

@bdraco
Copy link

bdraco commented Sep 28, 2023

If I create a pyproject.toml

pydantic % cat pyproject.toml 
[build-system]
requires = ["setuptools", "wheel", "Cython==0.29.32;sys_platform!='win32'"]

It uses cython 0.29.32

@bdraco
Copy link

bdraco commented Sep 28, 2023

Without the pyproject.toml its too late since setup.py loads cython and you get the system installed version instead of the one to build with

@samuelcolvin
Copy link
Member

makes sense. @hramezani could you work on another patch release using the right cython when building.

@bdraco
Copy link

bdraco commented Sep 28, 2023

#7695

Edit: saw above comment after I opened the PR. Closed the pr

@hramezani hramezani removed the pending Awaiting a response / confirmation label Sep 29, 2023
@samuelcolvin
Copy link
Member

Hi, I've tried to reproduce this using and x64_64 instruction set using gitpod (I'm on an M1 mac today), and I can't reproduce it.

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.10.13
            pydantic compiled: True
                 install path: /workspace/.pyenv_mirror/user/current/lib/python3.11/site-packages/pydantic
               python version: 3.11.5 (main, Sep 15 2023, 13:46:24) [GCC 11.4.0]
                     platform: Linux-6.1.54-060154-generic-x86_64-with-glibc2.35
     optional deps. installed: ['typing-extensions']

I just tried creating a model and doing some validation and it ran fine - but the exception occurs on this line which is just creating a model.


I'd really like to be able to reproduce this, then confirm it's fixed before creating a new release.

@samuelcolvin
Copy link
Member

I also tried running pydantic 1.10.13 inside alpine inside gitpod to use musl binaries, and again - no error.

@rkoehler2017
Copy link

rkoehler2017 commented Oct 3, 2023

same here, tried do install dietpi's octoprint on a pi 2 zero

Screen outputTraceback (most recent call last): File "/mnt/dietpi_userdata/octoprint/.local/bin/octoprint", line 8, in <module> sys.exit(main()) ^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/__init__.py", line 1008, in main octo(args=args, prog_name="octoprint", auto_envvar_prefix="OCTOPRINT") File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/cli/common.py", line 37, in invoke return self._impl.invoke(ctx) ^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1685, in invoke super().invoke(ctx) File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/cli/config.py", line 57, in cli ctx.obj.settings = init_settings( ^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/__init__.py", line 203, in init_settings from octoprint.settings import InvalidSettings, settings File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/settings/__init__.py", line 35, in <module> from octoprint.schema.config import Config File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/schema/config/__init__.py", line 9, in <module> from .access_control import AccessControlConfig File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/schema/config/access_control.py", line 11, in <module> class AccessControlConfig(BaseModel): File "pydantic/main.py", line 186, in pydantic.main.ModelMetaclass.__new__ TypeError: Argument 'bases' has incorrect type (expected list, got tuple) root@DietPi:/tmp/DietPi-Software# sudo -u octoprint /mnt/dietpi_userdata/octoprint/.local/bin/octoprint config set server.port 5001^C

@ewenmcneill
Copy link

ewenmcneill commented Oct 9, 2023

In case it helps since (a) I spotted this issue reading the OctoPrint release notes, and (b) I just spent a some time debugging a mysterious issue with importlib and TypeError that was hard to reproduce in another Python project (GlasgowEmbedded/glasgow#422): early Python 3.9 / especially early Python 3.10 don't have some fixes to importlib that got backported, eg, around March 2022 (eg, python/cpython#91160).

In the case of the problem I was debugging (metadata extras) the problem was a re.Match() was being returned, instead of a string, it affected Python 3.9 <= 3.9.10 and Python 3.10 <= 3.10.2. Your issue is clearly slightly different (tuple instead of list, and bases) but I'd suggest considering "maybe this issue got fixed in a Python importlib backport", and looking for a potential backport fix (I found the one affecting me via git annotate on the importlib function being called).

In the Glasgow project case I initially worked around it by catching the return type received was the "wrong one" and fixing it up before using it in the Glasgow project code; but the final fix merged ended up monkeypatching in the newer code for importlib metadata extras only on the old affected Python minor versions (see GlasgowEmbedded/glasgow#436). (in the tuple / list case, you could fairly easily unconditionally force it to be a list before using it.)

Ewen

TojikCZ added a commit to prusa3d/Prusa-Link that referenced this issue Oct 10, 2023
TojikCZ added a commit to prusa3d/Prusa-Link that referenced this issue Oct 10, 2023
@foosel
Copy link

foosel commented Oct 12, 2023

It gets weirder.

After pinning pydantic to 1.10.12 in my project (OctoPrint) to solve this problem, and confirming it against various deployment scenarios, I got reports that even that won't work on RaspberryPi OS Bookworm with Python 3.11.2.

So I ran some tests:

  • Python 3.11.4 on x86_64 (my laptop): works
  • CI (GitHub Actions) with Python 3.11.2 on x86_64: works
  • RPiOS Buster with Python 3.9 on armv7: works
  • armv7 Docker with stock Debian Bookworm packages and Python 3.11.2: works
  • armv7 Docker with RPiOS Bookworm packages and Python 3.11.2: broken

I had to downgrade to pydantic 1.10.10 in the last scenario to make things work there again.

At this point I have to say that I'm beyond confused by this particular issue but hope that the above findings can help in any way to get to the bottom of this.

@joostlek
Copy link
Author

We have also seen reports of 1.10.12 not working for some reason. In our case that could've been caused by custom integrations (which have their own dependency), but in earlier tests, installing 1.10.12 would fix the issue, but in some cases it just doesn't

@foosel
Copy link

foosel commented Oct 12, 2023

Well, if anything at least it fits spooky season 🎃👻

Currently running some more tests to collect some more data points.

@foosel
Copy link

foosel commented Oct 12, 2023

I got some more results. On a whim I tried a stock Debian Bookworm image with an added config to use piwheels.org as additional index for pip, and THAT resulted in a reproduction.

My test setup can be found here: https://gist.github.com/foosel/80b22908c008c063b85c819aee12c5ad

I'm testing with a really minimal test.py:

from pydantic import BaseModel


class TestModel(BaseModel):
    foo: str = "bar"


if __name__ == "__main__":
    test = TestModel()
    print(test)

And here's the results against Debian Bookworm with Python 3.11.2 on arm32v7:

pydantic pypi piwheels
1.10.10
1.10.11
1.10.12
1.10.13

So the issue as observed for pydantic 1.10.11 through 1.10.13 on Raspberry Pi OS Bookworm with Python 3.11.2 seems to stem from the binary wheels provided by the piwheels project. Piwheels comes preconfigured as additional index URL on RPi OS, and telling pip to use piwheels als additional index via /etc/pip.conf results on failure for pydantic 1.10.11 through 1.10.13 even on a stock Debian Bookworm under arm32v7.

A look at the package page reveals that piwheels only has wheels available from 1.10.11 onward, which means 1.10.10 would be taken from pypi, so something is apparently up with the builds on piwheels against Python 3.11.

I wonder if the packages compiled by piwheels also ran into the aforementioned problem of accidentally being compiled against Cython 3?

edit In any case, someone from the pydantic team with the proper authority of a maintainer should maybe request removal of the affected packages from piwheels as described in the FAQ:

image

edit I took the liberty to at least report the problem to piwheels: piwheels/packages#396

@CaseyHaralson
Copy link

I also tried running pydantic 1.10.13 inside alpine inside gitpod to use musl binaries, and again - no error.

Updating from 1.10.12 to 1.10.13 fixed the problem for me.

@mtelka
Copy link

mtelka commented Nov 9, 2023

I see this issue while testing maison.

@mtelka
Copy link

mtelka commented Nov 28, 2023

Testing of jaraco.text 3.12.0 with pydantic 1.10.13 failed:

________________ ERROR collecting jaraco/text/show-newlines.py _________________
/usr/lib/python3.9/vendor-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
/usr/lib/python3.9/vendor-packages/_pytest/runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/usr/lib/python3.9/vendor-packages/_pytest/doctest.py:567: in collect
    module = import_path(
/usr/lib/python3.9/vendor-packages/_pytest/pathlib.py:567: in import_path
    importlib.import_module(module_name)
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1030: in _gcd_import
    ???
<frozen importlib._bootstrap>:1007: in _find_and_load
    ???
<frozen importlib._bootstrap>:986: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:680: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:850: in exec_module
    ???
<frozen importlib._bootstrap>:228: in _call_with_frames_removed
    ???
jaraco/text/show-newlines.py:2: in <module>
    import inflect
/usr/lib/python3.9/vendor-packages/inflect/__init__.py:2027: in <module>
    class engine:
/usr/lib/python3.9/vendor-packages/inflect/__init__.py:2049: in engine
    def defnoun(self, singular: Optional[Word], plural: Optional[Word]) -> int:
pydantic/decorator.py:49: in pydantic.decorator.validate_arguments
    ???
pydantic/decorator.py:36: in pydantic.decorator.validate_arguments.validate
    ???
pydantic/decorator.py:126: in pydantic.decorator.ValidatedFunction.__init__
    ???
pydantic/decorator.py:264: in pydantic.decorator.ValidatedFunction.create_model
    ???
pydantic/main.py:1026: in pydantic.main.create_model
    ???
pydantic/main.py:186: in pydantic.main.ModelMetaclass.__new__
    ???
E   TypeError: Argument 'bases' has incorrect type (expected list, got tuple)

@johntamplin
Copy link

While the smart people are working on the proper fix for this, is there a simple workaround that we can use?

anschweitzer added a commit to thegridelectric/gridworks-protocol that referenced this issue Apr 23, 2024
anschweitzer added a commit to thegridelectric/gw-scada-spaceheat-python that referenced this issue Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug V1 Bug related to Pydantic V1.X
Projects
None yet
Development

No branches or pull requests