feat: make pydantic `dataclass` decorator support built-in `dataclass` #1817

PrettyWood · 2020-08-11T10:39:18Z

Change Summary

It was not possible to use pydantic.dataclasses.dataclass decorator directly on a built-in dataclass.
Now it should work.
It allows to easily support nested dataclasses and add validation for them. This is why it's also part of this PR

ℹ️ I'll add some doc if the behaviour is validated and no extra use cases are added

Related issue number

closes #744
closes #1743

Checklist

Unit tests for the changes exist
Tests pass on CI and coverage remains at 100%
Documentation reflects the changes where applicable
changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)

codecov · 2020-08-11T10:41:56Z

Codecov Report

Merging #1817 into master will increase coverage by 0.09%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           master     #1817      +/-   ##
===========================================
+ Coverage   99.90%   100.00%   +0.09%     
===========================================
  Files          21        21              
  Lines        4051      4094      +43     
  Branches      807       821      +14     
===========================================
+ Hits         4047      4094      +47     
+ Misses          3         0       -3     
+ Partials        1         0       -1

Impacted Files	Coverage Δ
pydantic/types.py	`100.00% <ø> (ø)`
pydantic/dataclasses.py	`100.00% <100.00%> (ø)`
pydantic/schema.py	`100.00% <100.00%> (ø)`
pydantic/utils.py	`100.00% <100.00%> (ø)`
pydantic/validators.py	`100.00% <100.00%> (ø)`
pydantic/env_settings.py	`100.00% <0.00%> (ø)`
pydantic/typing.py	`100.00% <0.00%> (+3.14%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5288920...e9d14eb. Read the comment docs.

closes pydantic#744

closes pydantic#1743

dylrich · 2020-09-09T22:57:37Z

This is great!

hawkaa · 2020-10-06T19:39:34Z

This is amazing! Need a hand with the last details before this is merged to master?

PrettyWood · 2020-10-06T20:59:07Z

@hawkaa I'll update the doc once the behaviour is fully validated. If you want to help, you could review the code and ensure the behaviour is the right one even for nested dataclasses. I'm not sure I thought about everything there may probably be some edge cases to handle.
Thanks for the feedback and for the help! 🙂

samuelcolvin

This looks awesome, I really want it.

Mostly small questions or tweaks.

samuelcolvin · 2020-10-09T08:22:00Z

pydantic/dataclasses.py

        return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen, config)

    if _cls is None:
        return wrap

    return wrap(_cls)
+
+
+def make_dataclass_validator(_cls: Type[Any], **kwargs: Any) -> 'CallableGenerator':


docstring please to explain to what this does.

samuelcolvin · 2020-10-09T08:23:11Z

pydantic/dataclasses.py

@@ -1,23 +1,32 @@
-from typing import TYPE_CHECKING, Any, Callable, Dict, Generator, Optional, Type, TypeVar, Union
+import dataclasses


I have a suspicion that importing dataclasses globally had a big effect on overall import time.

I think easiest to move it back to imports in functions, we could benchmark to see the performance of making this global.

Yes I know I remember when you changed some imports to be local in order to improve overall import time.
I forgot to import it back to all functions.

samuelcolvin · 2020-10-09T08:25:21Z

pydantic/dataclasses.py

@@ -128,18 +203,23 @@ def dataclass(
    unsafe_hash: bool = False,
    frozen: bool = False,
    config: Type[Any] = None,
-) -> Union[Callable[[Type[Any]], 'DataclassType'], 'DataclassType']:
+) -> Union[Callable[[Type[Any]], Type['DataclassType']], Type['DataclassType']]:


Wait, surely this signature is now wrong, you have double Type and the type of a type is always just type.

Maybe DataclassType is defined wrongly?

The name was missleading as DataclassType was refering to an instance of a pydantic dataclass. I renamed it to Dataclass.

samuelcolvin · 2020-10-09T08:25:56Z

pydantic/dataclasses.py

+    return not hasattr(_cls, '__processed__') and dataclasses.is_dataclass(_cls)
+
+
+def _dataclass_with_validation(


not sure it's worth this being a separate function?

samuelcolvin · 2020-10-09T08:26:57Z

pydantic/validators.py

@@ -565,6 +567,9 @@ def find_validators(  # noqa: C901 (ignore complexity)
    if is_literal_type(type_):
        yield make_literal_validator(type_)
        return
+    if is_builtin_dataclass(type_):
+        yield from make_dataclass_validator(type_)


👍 looks good.

tests/test_dataclasses.py

The name `DataclassType` was missleading as it's not a `Type` per say.

pydantic import time was improved in pydantic#1132 by keeping `dataclass` import local. So let's keep it that way!

PrettyWood · 2020-10-10T12:10:25Z

@samuelcolvin Thanks for the review! I took all your remarks into account. I still need to write some documentation about this but first I really want to make sure the whole behaviour is right. If you think it is please let me know!

samuelcolvin

This looks great, just a small question, and documentation.

samuelcolvin · 2020-10-25T16:05:17Z

pydantic/dataclasses.py


    return cls


+@overload


No biggy, but make these should be in TYPE_CHECKING?

I tried but doesn't seem to work
And then I saw that it was not in TYPE_CHECKING for root_validator

PrettyWood · 2020-10-26T11:03:57Z

@samuelcolvin Documentation has been added 👍 Do not hesitate to tweak it if need be

samuelcolvin · 2020-10-26T11:10:26Z

this is great, thanks.

PrettyWood mentioned this pull request Aug 11, 2020

Validating nested dataclasses without class ownership #1743

Closed

PrettyWood changed the title ~~feat: pydantic dataclass decorator now support built-in dataclass~~ feat: make pydantic dataclass decorator support built-in dataclass Aug 11, 2020

PrettyWood added 4 commits August 12, 2020 13:08

feat: pydantic dataclasses support built-in ones

5be6898

closes pydantic#744

feat: improve dataclass typing

d991cdf

feat: add support for nested dataclasses

55b6314

closes pydantic#1743

feat: support dataclass schema with nested dataclasses

2d076e1

PrettyWood force-pushed the builtin-dataclass-to-pydantic-dataclass branch from 6203b3d to 2d076e1 Compare August 12, 2020 11:35

PrettyWood mentioned this pull request Oct 6, 2020

Creating Model from standard dataclass #1967

Closed

samuelcolvin reviewed Oct 9, 2020

View reviewed changes

PrettyWood added 6 commits October 10, 2020 13:05

Merge branch 'master' into builtin-dataclass-to-pydantic-dataclass

d8af433

refactor: remove _dataclass_with_validation function

8ad29d6

docs: add docstring for make_dataclass_validator

e05e544

refactor: rename DataclassType into Dataclass

906033e

The name `DataclassType` was missleading as it's not a `Type` per say.

refactor: change global dataclass import to local

184138c

pydantic import time was improved in pydantic#1132 by keeping `dataclass` import local. So let's keep it that way!

test: add extra nested case with BaseModel

11397ec

PrettyWood force-pushed the builtin-dataclass-to-pydantic-dataclass branch from 748f3d7 to 11397ec Compare October 10, 2020 12:07

PrettyWood mentioned this pull request Oct 10, 2020

support make_dataclass for pydantic.dataclasses #1954

Closed

Merge branch 'master' into builtin-dataclass-to-pydantic-dataclass

b075dd5

samuelcolvin reviewed Oct 25, 2020

View reviewed changes

Merge branch 'master' into builtin-dataclass-to-pydantic-dataclass

b076ae2

PrettyWood mentioned this pull request Oct 26, 2020

build history and uprev version #2040

Merged

chore: s/pydantic/_pydantic_/g

432e1ce

PrettyWood force-pushed the builtin-dataclass-to-pydantic-dataclass branch from 81cc288 to 5b0b889 Compare October 26, 2020 10:59

docs: add some documentation

e9d14eb

PrettyWood force-pushed the builtin-dataclass-to-pydantic-dataclass branch from 5b0b889 to e9d14eb Compare October 26, 2020 11:01

samuelcolvin merged commit c83156d into pydantic:master Oct 26, 2020

PrettyWood deleted the builtin-dataclass-to-pydantic-dataclass branch October 26, 2020 11:10

PrettyWood mentioned this pull request Oct 27, 2020

fix: pydantic dataclass can inherit from stdlib dataclass and arbitrary_types_allowed is supported #2051

Merged

4 tasks

aimestereo mentioned this pull request Nov 10, 2020

support for built-in dataclass breaks pickle #2111

Closed

3 tasks

PrettyWood mentioned this pull request Nov 23, 2020

Fix #2111: support pickle for built-in dataclasses #2114

Merged

4 tasks

samuelcolvin mentioned this pull request Dec 12, 2020

Does pydantic.dataclasses.dataclass support classic mapping in SQLAlchemy? #1089

Closed

This was referenced Aug 12, 2022

Support cross-process pickling of Pydantic-wrapped built-in dataclasses #4084

Closed

dataclass conversion breaks multiprocessing pickling #3453

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: make pydantic `dataclass` decorator support built-in `dataclass` #1817

feat: make pydantic `dataclass` decorator support built-in `dataclass` #1817

PrettyWood commented Aug 11, 2020 •

edited

codecov bot commented Aug 11, 2020 •

edited

dylrich commented Sep 9, 2020

hawkaa commented Oct 6, 2020

PrettyWood commented Oct 6, 2020 •

edited

samuelcolvin left a comment

samuelcolvin Oct 9, 2020

samuelcolvin Oct 9, 2020

PrettyWood Oct 10, 2020

samuelcolvin Oct 9, 2020

PrettyWood Oct 10, 2020

samuelcolvin Oct 9, 2020

samuelcolvin Oct 9, 2020

PrettyWood commented Oct 10, 2020

samuelcolvin left a comment

samuelcolvin Oct 25, 2020

PrettyWood Oct 26, 2020

PrettyWood commented Oct 26, 2020

samuelcolvin commented Oct 26, 2020

		@@ -1,23 +1,32 @@
		from typing import TYPE_CHECKING, Any, Callable, Dict, Generator, Optional, Type, TypeVar, Union
		import dataclasses

		return not hasattr(_cls, '__processed__') and dataclasses.is_dataclass(_cls)


		def _dataclass_with_validation(

feat: make pydantic dataclass decorator support built-in dataclass #1817

feat: make pydantic dataclass decorator support built-in dataclass #1817

Conversation

PrettyWood commented Aug 11, 2020 • edited

Change Summary

Related issue number

Checklist

codecov bot commented Aug 11, 2020 • edited

Codecov Report

dylrich commented Sep 9, 2020

hawkaa commented Oct 6, 2020

PrettyWood commented Oct 6, 2020 • edited

samuelcolvin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PrettyWood commented Oct 10, 2020

samuelcolvin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PrettyWood commented Oct 26, 2020

samuelcolvin commented Oct 26, 2020

feat: make pydantic `dataclass` decorator support built-in `dataclass` #1817

feat: make pydantic `dataclass` decorator support built-in `dataclass` #1817

PrettyWood commented Aug 11, 2020 •

edited

codecov bot commented Aug 11, 2020 •

edited

PrettyWood commented Oct 6, 2020 •

edited