Undocumented different behaviors of handling generic types in v1 and v2 #6994
Comments
haoyun
added
bug V2
|
I could replace the type alias to get rid of the type checker error and runtime error, I must explicitly write all the concrete types. |
haoyun
changed the title
|
I'm having the same issue. Did you find a workaround that didn't break your type checker? Related SO question: https://stackoverflow.com/questions/76965613/there-are-no-type-variables-left-after-union-of-pydantic-models |
|
Here is a minimal reproducible example: from pydantic import BaseModel
from typing import Generic, TypeVar
T = TypeVar("T")
class Model(BaseModel, Generic[T]):
data: T
GenericAlias = Model[T] | None
def f() -> GenericAlias[int]: ...Note: This causes different errors depending on the Python minor version used! Error with Python Error with Python My system in both cases just with varying Python versions: So far I was unable to figure out where that error is actually coming from, but suffice it to say the code is valid in terms of typing, passes Mypy and does not raise that error, if The fact that the stacktrace is so short could indicate that the error is emitted at the CPython level, but this is just conjecture. But if that is true, I would assume that Pydantic does something to the generic class that makes it appear non-generic at runtime. I found a possibly interesting, somewhat related discussion about implicitly generic type aliases. |
|
Python 3.11.4 (main, Jul 25 2023, 17:36:13) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin I am using FastAPI to write an api and need to return this general data structure. Unlike the above, I currently have no inheritance behavior, so it seems that generic types are not supported? I use the generic type as the return type to display a more complete data structure in the OpenAPI UI
from dataclasses import dataclass
from typing import Generic, TypeVar
from fastapi import FastAPI
app = FastAPI()
app.debug = True
T = TypeVar("T")
@dataclass
class UserData:
id: int
name: str
age: int
avatar: str
@dataclass
class BookData:
id: int
name: str
use: int
@dataclass
class RespList(Generic[T]):
code: int
data: list[T]
@app.get("/api/getUser")
def index():
return RespList[UserData](
code=200,
data=[
UserData(
id=i,
name=f"test_{i}",
age=i,
avatar=f"https://avatars.dicebear.com/v2/avataaars/{i}.svg",
).__dict__
for i in range(10)
],
)
# run it> uvicorn main:app --reloadBelow is the traceback ➜ ~ uvicorn main:app --reload
INFO: Will watch for changes in these directories: ['/Users/alan/me/recatpy_']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [43575] using WatchFiles
Process SpawnProcess-1:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/homebrew/lib/python3.11/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/opt/homebrew/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
File "/opt/homebrew/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve
config.load()
File "/opt/homebrew/lib/python3.11/site-packages/uvicorn/config.py", line 467, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string
module = importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/alan/me/recatpy_/main.py", line 33, in <module>
@app.get("/api/getUser")
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/fastapi/routing.py", line 704, in decorator
self.add_api_route(
File "/opt/homebrew/lib/python3.11/site-packages/fastapi/routing.py", line 643, in add_api_route
route = route_class(
^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/fastapi/routing.py", line 448, in __init__
self.response_field = create_response_field(
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/fastapi/utils.py", line 99, in create_response_field
return ModelField(**kwargs) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^
File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare
File "pydantic/fields.py", line 755, in pydantic.fields.ModelField._type_analysis
TypeError: Fields of type "<class 'main.RespList'>" are not supported.
@app.get("/api/getUser")
def index() -> dict[str, int | list[dict[str, int | str]]]:
return dict(
code=200,
data=[
dict(
id=i,
name=f"test_{i}",
age=i,
avatar=f"https://avatars.dicebear.com/v2/avataaars/{i}.svg",
)
for i in range(10)
],
)
I can redeclare the class structure, but it's too much trouble. |
When I write this question to end... |
I'm not haoyun, but I can repro on 2.3.0 with his original example. @daniil-berg has a more minimal example that also errors on 2.3.0. Indeed I get the same error even if I install from the |
|
A relevant observation: listing the generic as the first parent of the class removes the buggy behavior. T = TypeVar("T")
# exhibits the buggy behavior
class BuggyModel(BaseModel, Generic[T]):
data: T
# does not!
class FixedModel(Generic[T], BaseModel):
data: T |
|
The problem here is that, for various reasons, the value returned by The problem is that when the typing module tries to parametrize the generic parametrized I believe this is fixable through some reworking of the I know it may be unsatisfying, but my suggestion would be to make an alias for A[T, str] instead of B = list[A[T, str]]I would use B = A[T, str]
# Instead of B[int], you now do list[B[int]]Hopefully we can get this fixed properly one day so your snippet "just works", but I think it will take some time. |
|
Also, I think detecting this behavior in |
|
we need to document the behaviour, and add a warning when the parents are used in the "wrong" order. |
Hmm yes, but currently there's no right order, yeah? class GenericFirst(Generic[T], BaseModel):
"""Produces a _GenericAlias object, not a submodel"""
pass
class ModelFirst(BaseModel, Generic[T]):
"""Produce a submodel, but exposes the bug described in this ticket"""
pass
# namely:
T1 = ModelFirst | str | int
T2 = GenericFirst | str | int
T1[list] # fails, but shouldn't!
T2[list] # succeeds the way the previous line should! |
I opened #7845 (before even seeing this) and started working on it. A few tests specifically related to parents in the wrong order started failing, and digging into the history led me here.
Yes, specifically there are places where it could pick up One even has a comment: I was able to work around this somewhat with some monkeypatching: import typing
from typing import Generic, TypeVar, List, Union
from pydantic import BaseModel
def patch_method(original):
def patched(self, args, *rest):
if not isinstance(args, tuple):
args = (args,)
args = tuple(
typing._GenericAlias(t, t.__parameters__)
if isinstance(t, type)
and issubclass(t, BaseModel)
and hasattr(t, "__parameters__")
else t
for t in args
)
return original(self, args, *rest)
return patched
typing._GenericAlias.__getitem__ = typing._tp_cache(
patch_method(typing._GenericAlias.__getitem__)
)
typing._GenericAlias._make_substitution = patch_method(
typing._GenericAlias._make_substitution
)
typing._SpecialGenericAlias.__getitem__ = typing._tp_cache(
patch_method(typing._SpecialGenericAlias.__getitem__)
)
T = TypeVar("T")
class A(BaseModel, Generic[T]):
pass
L = List[A[T]]
print(L[int])
# typing.List[__main__.A[int]]
U = Union[list[T], A[T]]
print(U[str])
# typing.Union[list[str], __main__.A[str]]
U2 = List[T] | A[T]
print(U2[float])
# typing.Union[typing.List[float], __main__.A[float]]Note that this code isn't exhaustive and it may not even be correct, but I think it demonstrates the point. However it doesn't work when the https://github.com/python/cpython/blob/main/Objects/genericaliasobject.c#L252
It seems that a full solution supporting
In the meantime, this comment seems fair:
I'd like to know if inheriting from |
Initial Checks
Description
I am creating a model
Athat has two type varsTandS.I am trying to define a type as
list[A[T, str], however withpydantic v2I got error:which does not happen with
pydantic v1.Example Code
Python, Pydantic & OS Version
Selected Assignee: @lig
The text was updated successfully, but these errors were encountered: