Dynamic default value #1210

PrettyWood · 2020-02-06T01:13:27Z

Change Summary

Following #155 and #866, here is a proposal of implementation to simplify dynamic default values

Right now, to have a dynamic default value, we need to use validators like

class DynamicValueModel(BaseModel):
    ts: datetime = None

    @validator('ts', pre=True, always=True)
        def set_ts_now(cls, v):
            return v or datetime.now()

Now, we can use default_factory (same name as dataclasses.to be consistent)

class DynamicValueModel(BaseModel):
    ts: datetime = Field(default_factory=datetime.now)

Related issue number

#155 and #866

Checklist

Unit tests for the changes exist
Tests pass on CI and coverage remains at 100%
Documentation reflects the changes where applicable
changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)

codecov · 2020-02-06T01:21:47Z

Codecov Report

Merging #1210 into master will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1210   +/-   ##
=======================================
  Coverage   99.89%   99.89%           
=======================================
  Files          21       21           
  Lines        3689     3706   +17     
  Branches      726      730    +4     
=======================================
+ Hits         3685     3702   +17     
  Misses          2        2           
  Partials        2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f94861...606449f. Read the comment docs.

Bobronium · 2020-02-06T19:20:46Z

Discussed in #866

So I would like to see this:

from datetime import datetime

from pydantic import BaseModel

class Record(BaseModel):
    name: str
    description: str
    ts: datetime = datetime.now

record = Record(name='Foo', description='Bar and Baz also.')
print(record.ts)
#> 2020-02-06 22:11:43.454720

However if type of field is callable, it must remain untouched:

from random import randint
from typing import Callable

from pydantic import BaseModel

class IntGenerator(BaseModel):
    fabric: Callable[..., int] = randint

generator = IntGenerator()
print(generator.fabric(0, 1))
#> 1

@samuelcolvin, any thoughts?

PrettyWood · 2020-02-06T20:00:48Z

@MrMrRobat Thanks ! I thought about the implementation with Callable but I'm afraid it would maybe break some existing behaviour. I personally thought it was the most intuitive one at first but the more I think about it, the more I fear there are some cases where it won't be very clear and we may have some bad surprises.
This is why I chose to use an extra field. I will rename it to default_factory to be consistent with native dataclasses.
If the Callable approach is chosen, this PR can be closed

samuelcolvin

Mostly looks goods, needs tests fixing and docs.

samuelcolvin · 2020-02-07T11:17:42Z

pydantic/fields.py

@@ -76,8 +77,12 @@ class FieldInfo(Representation):
        'extra',
    )

-    def __init__(self, default: Any, **kwargs: Any) -> None:
+    def __init__(self, default: Any = Undefined, *, default_factory: Any = None, **kwargs: Any) -> None:
+        if default is not Undefined and default_factory is not None:


perhaps better to move this check into Fieldm then you can use pop() as is done elsewhere.

samuelcolvin · 2020-02-07T11:18:04Z

pydantic/fields.py

    *,
+    default_factory: Any = None,


Suggested change

default_factory: Any = None,

default_factory: Optional[Callable[[], Any]] = None,

or something

samuelcolvin · 2020-02-07T11:26:06Z

I'm keen on the approach of

from datetime import datetime

from pydantic import BaseModel

class Record(BaseModel):
    ts: datetime = datetime.now

However I think it could cause subtle changes in behaviour in edge cases so let's wait for v2.

Getting Field(default_factory=...) to work now is a great first step.

PrettyWood · 2020-02-08T20:16:34Z

tests/test_main.py

+    # With a callable: we still should be able to set callables as defaults
+    class FunctionModel(BaseModel):
+        a: int = 1
+        uid: Callable[[], UUID] = Field(uuid4)


Had to use = Field(uuid) instead of = uuid4 as functions are ignored when creating fields and the default is hence not properly set
I think this behaviour should be fixed but in another PR (maybe for v2 with the other syntax for dynamic values)

PrettyWood · 2020-02-08T20:18:29Z

pydantic/fields.py

    *,
+    default_factory: Optional['Callable[[], Any]'] = None,


Had to use 'Callable...' because it would raise
TypeError: 'ABCMeta' object is not subscriptable

How is that possible? It already used in this module without an issues. That could be the case for collections.Callable, but it's a different thing.

Ah right ! I didnt' see the Callable came from .typing. Thanks !

samuelcolvin

There's a chance we might need to change the signature or behaviour of default_factory in future.

Therefore might be useful to comment in the documentation that this feature is provisional and might change in future minor releases? See #1179 for an example.

Otherwise looking good.

samuelcolvin · 2020-02-10T11:51:30Z

changes/1210-prettywood.md

@@ -0,0 +1 @@
+Add `default_factory` `Field` argument to create a dynamic default value by passing a zero-argument callable.


Suggested change

Add `default_factory` `Field` argument to create a dynamic default value by passing a zero-argument callable.

Add `default_factory` argument to `Field` to create a dynamic default value by passing a zero-argument callable.

samuelcolvin · 2020-02-10T12:00:52Z

pydantic/fields.py

@@ -3,6 +3,7 @@
 from typing import (
    TYPE_CHECKING,
    Any,
+    Callable,


if the below is TypingCallable, what is this? Maybe good to give this an alias.

I decided to add a NoArgAnyCallable directly in the local typing module. Makes things clearer imo

samuelcolvin · 2020-02-10T12:03:06Z

pydantic/fields.py

@@ -307,8 +321,9 @@ def prepare(self) -> None:
        Note: this method is **not** idempotent (because _type_analysis is not idempotent),
        e.g. calling it it multiple times may modify the field and configure it incorrectly.
        """
-        if self.default is not None and self.type_ is None:
-            self.type_ = type(self.default)
+        default_value = self.default_factory() if self.default_factory is not None else self.default


perhaps useful to have a get_default() function on this class which is used here and in main.py and perhaps deals with deepcopy?

closes pydantic#866

PrettyWood · 2020-02-10T14:30:57Z

@samuelcolvin Thanks for the review ! Changes done

samuelcolvin

otherwise looks good, sorry I missed this before.

pydantic/fields.py

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

jacobsvante · 2020-03-25T08:33:11Z

@samuelcolvin When do you think a release will be ready with this PR? Looking forward to it!

samuelcolvin · 2020-03-27T18:52:52Z

@jmagnusson see #1341 for details on release date.

PrettyWood force-pushed the dynamic-default-values branch 2 times, most recently from ba4603a to 5f7fc5d Compare February 7, 2020 03:00

samuelcolvin reviewed Feb 7, 2020

View reviewed changes

samuelcolvin mentioned this pull request Feb 7, 2020

Simplify setting dynamic default values #866

Closed

PrettyWood force-pushed the dynamic-default-values branch 2 times, most recently from 6b24d27 to 065cf78 Compare February 8, 2020 17:37

PrettyWood commented Feb 8, 2020

View reviewed changes

PrettyWood force-pushed the dynamic-default-values branch 2 times, most recently from 96fa43c to 5b08aef Compare February 10, 2020 10:09

samuelcolvin reviewed Feb 10, 2020

View reviewed changes

PrettyWood added 2 commits February 10, 2020 15:29

Simplify dynamic default values

02af626

closes pydantic#866

Add documentation and change

3a01cd5

PrettyWood force-pushed the dynamic-default-values branch from 5b08aef to 3802985 Compare February 10, 2020 14:31

Add ModelField.get_default method

fce7c10

PrettyWood force-pushed the dynamic-default-values branch from 3802985 to fce7c10 Compare February 10, 2020 14:46

Merge branch 'master' into dynamic-default-values

160aeda

samuelcolvin reviewed Mar 1, 2020

View reviewed changes

pydantic/fields.py Outdated Show resolved Hide resolved

Fix default value in infer

606449f

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

samuelcolvin merged commit 78a3f42 into pydantic:master Mar 4, 2020

PrettyWood deleted the dynamic-default-values branch March 25, 2020 08:56

samuelcolvin mentioned this pull request Mar 27, 2020

Issue with dynamic defaults #1346

Closed

PrettyWood mentioned this pull request Apr 22, 2020

Fix :: avoid mutating Field default value #1413

Merged

4 tasks

CasperWA mentioned this pull request Jun 12, 2020

Add usage of default_factory to models Materials-Consortia/optimade-python-tools#335

Open

keu mentioned this pull request Oct 1, 2021

Marcos/test pr 6054 airbytehq/airbyte#6581

Merged

38 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dynamic default value #1210

Dynamic default value #1210

PrettyWood commented Feb 6, 2020 •

edited

codecov bot commented Feb 6, 2020 •

edited

Bobronium commented Feb 6, 2020 •

edited

PrettyWood commented Feb 6, 2020 •

edited

samuelcolvin left a comment

samuelcolvin Feb 7, 2020

samuelcolvin Feb 7, 2020

samuelcolvin commented Feb 7, 2020

PrettyWood Feb 8, 2020

PrettyWood Feb 8, 2020

Bobronium Feb 8, 2020

PrettyWood Feb 10, 2020

samuelcolvin left a comment

samuelcolvin Feb 10, 2020

samuelcolvin Feb 10, 2020

PrettyWood Feb 10, 2020 •

edited

samuelcolvin Feb 10, 2020

PrettyWood commented Feb 10, 2020

samuelcolvin left a comment

jacobsvante commented Mar 25, 2020

samuelcolvin commented Mar 27, 2020

	default_factory: Any = None,
	default_factory: Optional[Callable[[], Any]] = None,

		@@ -0,0 +1 @@
		Add `default_factory` `Field` argument to create a dynamic default value by passing a zero-argument callable.

	Add `default_factory` `Field` argument to create a dynamic default value by passing a zero-argument callable.
	Add `default_factory` argument to `Field` to create a dynamic default value by passing a zero-argument callable.

Dynamic default value #1210

Dynamic default value #1210

Conversation

PrettyWood commented Feb 6, 2020 • edited

Change Summary

Related issue number

Checklist

codecov bot commented Feb 6, 2020 • edited

Codecov Report

Bobronium commented Feb 6, 2020 • edited

PrettyWood commented Feb 6, 2020 • edited

samuelcolvin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samuelcolvin commented Feb 7, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samuelcolvin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PrettyWood Feb 10, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PrettyWood commented Feb 10, 2020

samuelcolvin left a comment

Choose a reason for hiding this comment

jacobsvante commented Mar 25, 2020

samuelcolvin commented Mar 27, 2020

PrettyWood commented Feb 6, 2020 •

edited

codecov bot commented Feb 6, 2020 •

edited

Bobronium commented Feb 6, 2020 •

edited

PrettyWood commented Feb 6, 2020 •

edited

PrettyWood Feb 10, 2020 •

edited