Add a config to make all fields Optional

[christophelec]

Discussed in #3089

^{Originally posted by christophelec August 11, 2021}

Proposal : adding a Config key to set all fields of a model as Optional

Context :

In FastAPI, the OpenAPI spec is defined via Pydantic models.

To create a partial update endpoint in FastAPI, I'd like to be able to create a Model derived from another, but with all fields Optional, without having to manually duplicate all fields.

For example, let's say I have an API dealing with user, and creating a user takes as input :

class User(BaseModel):
  first_name: str
  last_name: str
  age: int
  address: str

If I want to have an endpoint to be able to partially update the user (e.g. change only the first_name), I'd like an endpoint where I can send :

{"first_name": "NEW"}

To have this behavior, currently I have to either :

• Create a specific model for patching, with all fields set as Optional :

class PatchedUser(BaseModel):
  first_name: Optional[str]
  last_name: Optional[str]
  age: Optional[int]
  address: Optional[str]

• Set all fields as Optional in the first model, and then add back manual validation down the road in the context of a user creation (where the fields are mandatory) (https://fastapi.tiangolo.com/tutorial/body-updates/#partial-updates-with-patch)

The first solution results in a lot of duplication, and therefore a lot of potential bugs.
The second one makes the schema less clear, and validation less easy.

Proposed solution

I would like to be able to derive a model from this base model, but switching all the fields to Optional. A possible usage could be :

class PartialUser(User):
  class Config:
    all_optional = True

This way, the PartialUser would keep all the fields from User, but setting them as Optional.

The result in FastAPI would look like this :

@router.post("/users/")
async def create_user(
    user: User,
):
    # All fields are mandatory
    return create_user(user)

@router.patch("/users/{user_id}/")
async def update_user(
    user_id: int,
    user: PartialUser,
):
  # Only fields to modify will be kept
  return update_user(user.dict(exclude_unset=True))

I'm willing to open a PR if you think this is a good idea.

[PrettyWood]

Hello @christophelec
In the meantime you can see #2272 (comment).
For feature requests, better use the discussions section btw

Edit: just saw you also opened the discussion! Can we close this issue in the meantime?

[christophelec]

Hello @PrettyWood, I opened the discussion last week but it did not trigger much discussion except a 👍, so I went ahead and opened the issue, as well as started a PR as it would be really useful on my side.

The PR is here : #3121, and I just saw your comment about class kwargs, which indeed would be nice : I'll update accordingly.

I'm ok to close the issue

[mrlubos]

Per discussion in #3179, the current advice is to wait for v2

[adriangb]

I don't think we'll have a special solution for this in v2. This is just not possible to express to Python's type system. The best I could come up with is the following, but I think it would work in v1 as well so it's nothing new:

from copy import deepcopy
from typing import Any, Optional, Tuple, Type, TypeVar

from pydantic import BaseModel, create_model
from pydantic.fields import FieldInfo


class User(BaseModel):
  first_name: str
  last_name: str


def make_field_optional(field: FieldInfo, default: Any = None) -> Tuple[Any, FieldInfo]:
  new = deepcopy(field)
  new.default = default
  new.annotation = Optional[field.annotation]  # type: ignore
  return (new.annotation, new)


BaseModelT = TypeVar('BaseModelT', bound=BaseModel)

def make_partial_model(model: Type[BaseModelT]) -> Type[BaseModelT]:
  return create_model(  # type: ignore
    f'Partial{model.__name__}',
    __base__=User,
    __module__=User.__module__,
    **{
        field_name: make_field_optional(field_info)
        for field_name, field_info in User.model_fields.items()
    }
    )


PartialUser = make_partial_model(User)


print(PartialUser(first_name='Adrian'))
#> first_name='Adrian' last_name=None

Since we can't really do much better than that without the Python ecosystem reaching a consensus and adding typing support for a partial operator, I'm closing this as a wontfix.

[jd-solanki]

Can we reopen this for v2? In typescript, it's just Partial<User> 🤷🏻‍♂️

I read the issue and almost everyone has to repeat code. All of us were waiting for v2 but there's no luck yet 😢

Is it even planned?

[samuelcolvin]

We'll support it as soon as python has a type that allows us to express this in a way that type checkers can understand.

If you really want this, I could post on discuss.python.org advocating for this feature in the language, post the link here and I'll reply and support it.

You're best option might be to use a typeddict where you can set total=False, though I still don't think there's a way to "partialise" an existing model.

[jd-solanki]

Thanks @samuelcolvin for responding.

If you really want this

Yes, I honor the work you are doing (as I'm also the OSS dev) and if you can raise the query upstream (python) it will be a great addition to pydantic because people (mostly FastAPI devs) always wanted this. I'll do my best to support this query.

I tried typedDict with my knowledge but I failed because I wanted to "partialise" the class inherited from BaseModel just like you mentioned.

Best Regards,
JD

[sree1026]

I don't think we'll have a special solution for this in v2. This is just not possible to express to Python's type system. The best I could come up with is the following, but I think it would work in v1 as well so it's nothing new:

from copy import deepcopy
from typing import Any, Optional, Tuple, Type, TypeVar

from pydantic import BaseModel, create_model
from pydantic.fields import FieldInfo


class User(BaseModel):
  first_name: str
  last_name: str


def make_field_optional(field: FieldInfo, default: Any = None) -> Tuple[Any, FieldInfo]:
  new = deepcopy(field)
  new.default = default
  new.annotation = Optional[field.annotation]  # type: ignore
  return (new.annotation, new)


BaseModelT = TypeVar('BaseModelT', bound=BaseModel)

def make_partial_model(model: Type[BaseModelT]) -> Type[BaseModelT]:
  return create_model(  # type: ignore
    f'Partial{model.__name__}',
    __base__=User,
    __module__=User.__module__,
    **{
        field_name: make_field_optional(field_info)
        for field_name, field_info in User.model_fields.items()
    }
    )


PartialUser = make_partial_model(User)


print(PartialUser(first_name='Adrian'))
#> first_name='Adrian' last_name=None

Since we can't really do much better than that without the Python ecosystem reaching a consensus and adding typing support for a partial operator, I'm closing this as a wontfix.

@adriangb Thank you so much for the solution. Seems like Pydantic v1.10 throws TypeError: Object of type 'ModelField' is not JSON serializable when I try to callschema_json() on the PartialUser object.

However, when testing with Pydantic v2.0.3, the error is no longer thrown. I'm not sure why the older version of Pydantic was throwing the error. In any case, I plan to update to the latest version, so it's not a major concern for me. I just wanted to post the issue here so that anyone else facing the same error with older Pydantic versions will know that updating might resolve it.

[tobiasfeil]

Has anyone actually tried the code? I had to replace

    __base__=User,
    __module__=User.__module__,

with

    __base__=model,
    __module__=model.__module__,