Mypy type issues with model_validator()

[plannigan]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

I am trying to port a project to pydantic v2 that uses model validators. I am seeing a number of errors for mypy that make me not sure about what is the intended API versus what might be potential bugs in the type annotations exposed to mypy. (The validators documentation page, still has a warning that the page still needs to be updated, but seems to have had some updates. If these docs were in a known state, I might have a better idea of where the error might be.)

Based on the example code, it seems like:

• both "before" and "after" support being called as an instance method or a classmethod (even without @classmethod). Returning self or data, respectively, does not cause any runtime errors.
• before with data annotated as a Foo: actual type is dict. NO mypy error ❌
• before with data annotated as a dict: actual type matches, no mypy ✔️
• before without data argument: actual self type is dict. mypy error that function signature is not correct and return type is wrong ❓
• after with data annotated as a Foo: actual type matches. mypy error that function signature is not correct ❌
• after with data annotated as a dict: actual type is Foo. mypy error that function signature is not correct ✔️
• after without data argument: actual self type is Foo. mypy error that function signature is not correct and return type is wrong ❓

mypy output:

mypy --version; mypy scratch/pydantic_model_validator_investigation.py 
mypy 1.4.1 (compiled: yes)
scratch/pydantic_model_validator_investigation.py:31: error: Argument 1 has incompatible type "Callable[[type[Foo]], Foo]"; expected
"ModelBeforeValidator | ModelBeforeValidatorWithoutInfo"  [arg-type]
        @model_validator(mode="before")
         ^
scratch/pydantic_model_validator_investigation.py:35: error: Incompatible return value type (got "type[Foo]", expected "Foo")  [return-value]
            return self
                   ^~~~
scratch/pydantic_model_validator_investigation.py:37: error: Argument 1 has incompatible type "Callable[[type[Foo], Foo], Foo]"; expected
"Callable[[<nothing>, ValidationInfo], <nothing>] | Callable[[<nothing>], <nothing>]"  [arg-type]
        @model_validator(mode="after")
         ^
scratch/pydantic_model_validator_investigation.py:44: error: Argument 1 has incompatible type
"Callable[[type[Foo], dict[str, object]], dict[str, object]]"; expected
"Callable[[<nothing>, ValidationInfo], <nothing>] | Callable[[<nothing>], <nothing>]"  [arg-type]
        @model_validator(mode="after")
         ^
scratch/pydantic_model_validator_investigation.py:51: error: Cannot infer function type argument  [misc]
        @model_validator(mode="after")
         ^
scratch/pydantic_model_validator_investigation.py:55: error: Incompatible return value type (got "type[Foo]", expected "Foo")  [return-value]
            return self
                   ^~~~
Found 6 errors in 1 file (checked 1 source file)

pyproject.toml:

[tool.mypy]
files = "hyper_bump_it"
show_error_codes = true
warn_unused_configs = true
pretty = true
strict = true
disallow_any_explicit = true
warn_unreachable = true
plugins = [
  "pydantic.mypy"
]

[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true

Example Code

from pydantic import model_validator, BaseModel, ConfigDict, __version__

print("pydantic version:", __version__)


def display(name: str, x: object) -> None:
    print(name, "type:", type(x), "value:", x)


class Foo(BaseModel):
    model_config = ConfigDict(
        extra="forbid", str_min_length=1, frozen=True, strict=True
    )
    a: int = 1
    b: bool

    @model_validator(mode="before")
    def _before_annotated_as_class(cls, data: "Foo") -> "Foo":
        print("\n_before_annotated_as_class()")
        display("cls", cls)
        display("data", data)
        return data

    @model_validator(mode="before")
    def _before_annotated_as_dict(cls, data: dict[str, object]) -> dict[str, object]:
        print("\n_before_annotated_as_dict()")
        display("cls", cls)
        display("data", data)
        return data

    @model_validator(mode="before")
    def _before_annotated_no_data(self) -> "Foo":
        print("\n_before_annotated_no_data()")
        display("self", self)
        return self

    @model_validator(mode="after")
    def _after_annotated_as_class(cls, data: "Foo") -> "Foo":
        print("\n_after_annotated_as_class()")
        display("cls", cls)
        display("data", data)
        return data

    @model_validator(mode="after")
    def _after_annotated_as_dict(cls, data: dict[str, object]) -> dict[str, object]:
        print("\n_after_annotated_as_dict()")
        display("cls", cls)
        display("data", data)
        return data

    @model_validator(mode="after")
    def _after_annotated_no_data(self) -> "Foo":
        print("\n_after_annotated_no_data()")
        display("self", self)
        return self


foo = Foo(a=2, b=True)
print("\nresult")
display("foo", foo)

Python, Pydantic & OS Version

pydantic version: 2.0.3
        pydantic-core version: 2.3.0 release build profile
                 install path: /home/patrick/code/.pydantic_venv/lib/python3.11/site-packages/pydantic
               python version: 3.11.0 (main, Oct 26 2022, 05:13:43) [GCC 9.4.0]
                     platform: Linux-5.4.0-153-generic-x86_64-with-glibc2.31
     optional deps. installed: ['typing-extensions']

Selected Assignee: @adriangb

[adriangb]

I would add @classmethod when you use cls.

before with data annotated as a Foo: actual type is dict. NO mypy error

But it can be Foo!: Foo.model_validate(Foo.model_construct(a=2, b=True))

Some of these other ones are questionable, e.g. after with the signature (cls, Foo) is equivalent to just (Foo) i.e. (self) as far as we are concerned, but there's no reason to use it that way.

The most correct signatures to use are (cls, data: Any) (using isinstance(data, dict) and such to narrow the type) for before and (self) for after.

[adriangb]

Closing since we don't plan on making any changes (aside from maybe tweaking the docs)

[plannigan]

Thank you for clarifying the intended API for before. With the current state of the documentation, it wasn't clear that data could be any arbitrary object (A TypeError would be raised in the validator if UserModel.model_validate() is passed an object that isn't a child of BaseModel).

As for after, mypy still raises an error for the intended API.

from pydantic import model_validator, BaseModel, ConfigDict

class Foo(BaseModel):
    model_config = ConfigDict(
        extra="forbid", str_min_length=1, frozen=True, strict=True
    )
    a: int = 1
    b: bool

    @model_validator(mode="after")
    def _after_annotated_return_self(self) -> "Foo":
        print("\n_after_annotated_return_self()")
        return self


foo = Foo(a=2, b=True)

mypy scratch/pydantic_experimentaiton/model_validator_after.py 
scratch/pydantic_experimentaiton/model_validator_after.py:10: error: Cannot infer function type argument  [misc]
        @model_validator(mode="after")
         ^
scratch/pydantic_experimentaiton/model_validator_after.py:13: error: Incompatible return value type (got "type[Foo]", expected "Foo")  [return-value]
            return self
                   ^~~~
Found 2 errors in 1 file (checked 1 source file)

mypy thinks self is type[Foo]. I'm guessing there is something that makes it think any @model_validator() decorated methods is a @classmethod. PyCharm also warns "Usually first parameter of such methods is named 'cls'" for the self argument to the method.

mypy also rejects an explicit annotation of self.

scratch/pydantic_experimentaiton/model_validator_after.py:11: error: The erased type of self "model_validator_after.Foo" is not a supertype of its
class "Type[model_validator_after.Foo]"  [misc]
        def _after_annotated_return_self(self: "Foo") -> "Foo":

There is also the "Cannot infer function type argument" error. I saw python/mypy#15620. Maybe this means that the Callable workaround doesn't completely resolve the issue.

[plannigan]

#6709 captures the issue I posted about above as I am also using the pydantic mypy plugin.