mypy 1.7.0 results in new [misc] type errors with pydantic usage

[jessemyers-lettuce]

I was asked to file a bug with more details; it's possible that the behavior is not as much a bug as increased strictness, but given that it impacts a widely used library, I wanted to share (and ask for recommendations on how proceed since manually ignoring this issue across many projects is going to be tedious).

Minimal program:

Minimal setup:

% python --version
Python 3.12.0

% python3 -m venv .venv
% source .venv/bin/activate
% pip install -U pip mypy pydantic
% pip freeze
annotated-types==0.6.0
mypy==1.7.0
mypy-extensions==1.0.0
pydantic==2.4.2
pydantic_core==2.10.1
typing_extensions==4.8.0

% echo "from pydantic import BaseModel" >> example.py
% echo >> example.py
% echo "class Foo(BaseModel):" >> example.py
% echo "    pass" >> example.py

% mypy --disallow-any-explicit --disallow-any-decorated example.py
example.py:3: error: Explicit "Any" is not allowed  [misc]
example.py:3: error: Type of decorated function contains type "Any" ("Callable[[DefaultNamedArg(dict[str, Any] | None, '__pydantic_extra__'), DefaultNamedArg(set[str], '__pydantic_fields_set__'), DefaultNamedArg(dict[str, Any] | None, '__pydantic_private__')], None]")  [misc]
Found 2 errors in 1 file (checked 1 source file)

These errors are not reproducible with mypy 1.6.1

[hauntsaninja]

Thanks for the repro! Other than --disallow-any-generics and --disallow-subclassing-any, I think the --disallow-any* are not too widely used.

This bisects to #15915. It's a little unfortunate, whether it's a bug seems arguable since you can get Any via dataclasses.replace cc @ikonst in case you have an opinion

[jessemyers-lettuce]

I think the --disallow-any* are not too widely used.

I know. My most recent projects are the first time it was even conceivable to use it because enough of the community projects have valid types that it's actually sane to enforce. Having done similar things with TypeScript, I find a lot of value in not allowing explicit use of Any, but it's really tricky to make this play nice with third-party systems. We haven't decided yet how to deal with this change in our systems and are currently holding back the latest mypy release.

I only raised this issue because pydantic is widely used enough that it might impact others. I understand if you don't find that compelling.

[JelleZijlstra]

It feels wrong that we're raising an "explicit Any" error here when the user didn't directly use Any, it's inside pydantic's code.

[jessemyers-lettuce]

Is it plausible to give more control over the "explicit Any" behavior so we can more easily differentiate user code vs dependency code? Because that would be amazing.

[ikonst]

Type of decorated function contains type "Any"

The internal __mypy_replace is deemed "decorated" because it's a static method, so it has a @staticmethod decorator.

We could:

-        self.check_untyped_after_decorator(sig, e.func)
+        if e.decorators:
+            self.check_untyped_after_decorator(sig, e.func)

This is circumstantial (we're still confusingly checking a "synthetic" function) but would get rid of this error.

Explicit "Any" is not allowed

The root cause is that __mypy_replaces' args can contain Anys from the dataclasses' attr types.

Would it make sense to skip the synthetic FuncDef entirely in check_func_def? Unfortunately FuncDef doesn't have any "is synthetic" attr that we could check.

[ikonst]

"Type of decorated function contains type "Any"" is somewhat of a red herring here since @classmethod does not change the function, i.e. it didn't add more Anys that were there in the first place: https://mypy-play.net/?mypy=latest&python=3.11&flags=disallow-any-decorated%2Cdisallow-any-explicit&gist=4933e60b808558971aa1bb4967d01db3

I considered maybe when synthesizing __mypy_replace we should somehow rewrite AnyTypes to be something other than TypeOfAny.explicit, but there's no better type, and it doesn't even address the "Type of decorated function" error since it's not limited to explicit Anys.

A mechanism to skip generated functions is probably the most reasonable fix.