mypy error on default_factory=list in pydantic 1.10

[tapple]

Initial Checks

• I have searched GitHub for a duplicate issue and I'm sure this is something new
• I have searched Google & StackOverflow for a solution and couldn't find anything
• I have read and followed the docs and still think this is a bug
• I am confident that the issue is with pydantic (not my code, or another library in the ecosystem like FastAPI or mypy)

Description

mypy complains about default_factory = list in pydantic 1.10

But using [] works fine. I think this inconsistency should be fixed

$ mypy bug.py
bug.py:6: error: Incompatible types in assignment (expression has type "List[_T]", variable has type "List[str]")
Found 1 error in 1 file (checked 1 source file)

Also, I don't think the documentation indicates that pydantic deep-copies mutable default values. I found that out at https://stackoverflow.com/questions/63793662/how-to-give-a-pydantic-list-field-a-default-value

Example Code

from typing import List
from pydantic import BaseModel, Field


class Model(BaseModel):
    l1: List[str] = Field(default_factory=list)
    l2: List[str] = Field([])
    l3: List[str] = []

Python, Pydantic & OS Version

pydantic version: 1.10.1
            pydantic compiled: True
                 install path: /home/mfulmer/miniconda3/envs/rk310/lib/python3.10/site-packages/pydantic
               python version: 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0]
                     platform: Linux-5.4.0-124-generic-x86_64-with-glibc2.31
     optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']

Affected Components

• Compatibility between releases
• Data validation/parsing
• Data serialization - .dict() and .json()
• JSON Schema
• Dataclasses
• Model Config
• Field Types - adding or changing a particular data type
• Function validation decorator
• Generic Models
• Other Model behaviour - construct(), pickling, private attributes, ORM mode
• Settings Management
• Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

[Pentusha]

Same with dict.

parameters: Dict[str, str] = Field(default_factory=dict)

error: Incompatible types in assignment (expression has type "Dict[_KT, _VT]", variable has type "Dict[str, str]")

[samuelcolvin]

Could be related to #4086, @richardxia any idea how this can be fixed?

---

By the way, @tapple @Pentusha it would be much easier for you and me if problems like this got caught before releasing v1.10.

In future, please consider testing pydantic pre-releases.

We had three pre-releases over 2 weeks and i talked about them a lot on twitter.

[richardxia]

Could be related to #4086, @richardxia any idea how this can be fixed?

I don't think it's related to #4086; that PR should have only affected whether the mypy plugin thinks a given field is required or not. I also tested reverting that commit locally and testing the example code in this ticket, but I still see the same error message show up.

I did a git bisect, and it looks like 460f858 (#3430) is the first commit that fails to type check the example code in this ticket. Maybe @klaa97 could take a look?

[klaa97]

@richardxia Thank you for the tag and sorry for this 🤦 I am pretty sure this comes from my commit, here in particular: https://github.com/klaa97/pydantic/blob/13e0a25cc8c3c8ec5a264f6c9635350ea6b71f3e/pydantic/mypy.py#L165 .

In particular, it comes from the default_factory mypy check. Mypygives back a CallableType which return is builtins.list[_T], and unfortunately I am not finding an easy way to "block" this behaviour. It works by specifying the type in the factory (for example a function returning list[int], but in this case it fails.

I wish we'd find it sooner :(, but I'm afraid I cannot find much more documentation on how to tackle this error therefore I'd say reverting the change might be the easiest fix.

[samuelcolvin]

Thanks so much for looking into this.

Maybe we could comment out that check for now and ask on the mypy repo if there's a work around?

[klaa97]

I completely agree, it looks like a plan! I could open a PR by end of the day to remove the check on default_factory, and proceed with the investigation on the mypy repo.

[samuelcolvin]

Please tag me in the mypy discussion, I'd be interested to see what they say.

[samuelcolvin]

I think I have a fix for this that doesn't involve removing the check, see #4471. @klaa97 would be great if you could review.