`_pydantic_post_init()` reassigns `__dict__` and removes properties added to the object, for example in `__new__()`, breaking integration with SQLAlchemy and other libraries

[DomWeldon]

Checks

• [ x] I added a descriptive title to this issue
• [ x] I have searched (google, github) for similar issues and couldn't find anything
• [ x] I have read and followed the docs and still think this is a bug

Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.8.2
            pydantic compiled: True
                 install path: /somewhere/on/doms/filesystem/.venv/lib/python3.9/site-packages/pydantic
               python version: 3.9.5 (default, Jun 13 2021, 13:20:20)  [GCC 9.3.0]
                     platform: Linux-5.8.0-59-generic-x86_64-with-glibc2.31
     optional deps. installed: ['typing-extensions']

Related issues: #1089 #2924 .

Summary of Problem

The pydantic dataclass decorator calls object.__setattr__() on self.__dict__ for a dataclass post init, using only the output of validate_model(), which removes any other attributes in the object's namespace, causing many issues including those well described for SQLAlchemy.

Full code example

"""Comparison: pydantic basemodels, dataclasses, and stdlib dataclasses"""
# Standard Library
import dataclasses
import typing

# Third Party Libraries
import pydantic


class StandardClass:
    """Class which modifies instance creation."""
    a: str

    def __new__(cls, *args, **kwargs):
        instance = super().__new__(cls)

        instance._special_property = 1

        return instance


StandardLibDataclass = dataclasses.dataclass(StandardClass)
PydanticDataclass = pydantic.dataclasses.dataclass(StandardClass)

class PydanticModel(pydantic.BaseModel):
    a: str

clases_to_test = {
    "StandardLibDataclass": StandardLibDataclass,
    "PydanticDataclass": PydanticDataclass,
}


test_string = "string"
for name, class_ in clases_to_test.items():
    instance = class_(a=test_string)
    if not hasattr(instance, "_special_property"):
        print(f"⚠️ Something's up with {name}")
        print(instance.__dict__)
    assert instance._special_property == 1
    assert instance.a == test_string
    print(f"✔️ {name}")

Output

✔️ StandardLibDataclass
⚠️ Something's up with PydanticDataclass
{'a': 'string', '__initialised__': True}
Traceback (most recent call last):
  File "/somewhere/on/doms/filesystem/dataclasses_vs_pydantic.py", line 40, in <module>
    assert instance._special_property == 1
AttributeError: 'StandardClass' object has no attribute '_special_property'

As you can see from the example above, this behaviour is particular to pydantic's dataclass implementation, and does not match with standard library dataclasses.

The fix for this could be very simple: simply calling self.__dict__.update() with the validated values, rather than re-assigning __dict__. This fixed the issue in my minimal test case for SQLAlchemy integration, but I am not sure of the wider consequences of the fix yet.

I also realise there may be a philosophy behind the decision to use this approach (and tests with pydantic.BaseModel-derived classes are similar), so appreciate any input if this is the case. However, based on PEP 557 I believe that a dataclass shouldn't alter class/instance behaviour in this way.

I'm working on a PR suggesting this fix, but it's the first time I've checked out this repo and am still getting the dev environment setup.

[DomWeldon]

Updated the issue above to correct an issue with the example code.

[zulrang]

@DomWeldon I've created a PR to fix this and added your test, but full credit should go to you.

[DomWeldon]

Thanks @zulrang - although lots of the investigative work was done by @zzzeek who deserves the credit here!