Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add skip_none argument #587

Closed
JakeSummers opened this issue Jun 10, 2019 · 4 comments · Fixed by #1003
Closed

add skip_none argument #587

JakeSummers opened this issue Jun 10, 2019 · 4 comments · Fixed by #1003

Comments

@JakeSummers
Copy link

@JakeSummers JakeSummers commented Jun 10, 2019

Question / Feature Request

Suppose that I have a class like this:

class Jake(BaseModel):
    foo: Optional[str]

When I serialize it, I only want to serialize optional values when they are not null. The skip_defaults flag does something similar to what I am looking for but has this weird case:

>>> j = Jake()
>>> 
>>> # This is the behavior I expect: 
... j.dict(skip_defaults=True)
{}
>>> 
>>> # Set the foo property: 
>>> j.foo = "baz"
>>> 
>>> # This is also the behaviour I expect: 
... j.dict(skip_defaults=True)
{'foo': 'baz'}
>>> 
>>> 
>>> # Unfortunately, this is not the behavior that I expect:
... j.foo = None
>>> j.dict(skip_defaults=True)
{'foo': None}

Right now after I serialize to dict, I walk the dict and remove all of the None values recursively, but I would rather not have to do this. Is this built into Pydantic? Is the behavior that I am seeing a bug?

@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Jun 11, 2019

This behavior is actually not a bug (I found it a little confusing at first too) -- it is intended so that you can tell precisely which fields were set on the specific object instance, rather than taking the default value based on the class definition. (I've included an example below to show why this might be useful.)

Removing None from the dicts seems most commonly relevant to serialization logic; based on this comment this seems explicitly outside the scope of pydantic.

For what it's worth, FastAPI has a function (fastapi.encoders.jsonable_encoder) that is specifically intended for converting (nested) pydantic models to dicts, and it has an include_none parameter that does what you want. If you want this capability in the context of serialization, you might find this function useful, even if only as a reference implementation (it handles a number of other serialization edge cases as well).


Here's a concrete example of how the skip_defaults=True logic can be useful in the case of a REST resource with optional properties:

Imagine you want to store a user's username, and optionally an email.

class User(BaseModel):
    user_id: uuid.UUID
    username: str
    email: Optional[str]

A clean way to implement updates is to have a corresponding model where all fields are optional. Imagine you have the following update function and model:

def update_user(user_id: uuid.UUID, **update_kwargs):
    for key, value in update_kwargs.items():
        set_user_attribute(user_id, key, value)

class UserUpdate(BaseModel):
    user_id: uuid.UUID
    username: Optional[str]
    email: Optional[str]

Then

user_update = UserUpdate(user_id=<user_id>, email="a@b.com")
update_user(**user_update.dict(skip_defaults=True))

makes sure that you only update the fields that were specified; in this case, only the email address. (This is a common pattern for the HTTP PATCH method.)

If the skip_defaults=True logic always removed None, then it would no longer be possible to specify that you want to change the email attribute from a non-None value back to None.

@samuelcolvin

This comment has been minimized.

Copy link
Owner

@samuelcolvin samuelcolvin commented Jun 11, 2019

great answer @dmontagu

@JakeSummers

This comment has been minimized.

Copy link
Author

@JakeSummers JakeSummers commented Jun 11, 2019

@samuelcolvin - Is this the kind of functionality that you would consider for Pydantic? Could we leave this open as a feature request?

(Thanks @dmontagu great explanation of that behaviour!)

@samuelcolvin samuelcolvin changed the title Do not serialize keys with optional values when optional is absent add skip_none argument Jun 12, 2019
@samuelcolvin samuelcolvin reopened this Jun 12, 2019
@samuelcolvin

This comment has been minimized.

Copy link
Owner

@samuelcolvin samuelcolvin commented Jun 12, 2019

I'd accept skip_none keyword argument to dict() and friends.

@niknetniko niknetniko mentioned this issue Nov 17, 2019
4 of 4 tasks complete
samuelcolvin added a commit that referenced this issue Nov 18, 2019
* add `exclude_none` option (#587)

* run formatter

* Apply @samuelcolvin's suggestions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.