Question: How to completely replace Pydantic models with ormar models?

[soderluk]

I know one could use the ormar models as is, for both Pydantic and database.

However, how would one go about if it's necessary to only include some of the model information when e.g. creating a record or updating one?

E.g. as in the FastAPI examples, there are schemas (Pydantic models) for ModelBase, ModelCreate and Model. The models inherit the ModelBase, and are then used as input models when creating (ModelCreate), as well as response models (Model). These models can have different parameters when input/output.

Is it possible to achieve with just the ormar models in some way? I'd like to get rid of the duplication that I have to do atm, since I don't know how to modify the response parameters/request parameters for the one model.

[collerek]

Hi, Can you provide some more concrete example what you want to do.
Like - exclude the fields in response, add additional fields into create model, remove/ add calculated properties etc.

Cause some things are possible already in ormar, some in fastapi, some in pydantic, some hera and there. And if something is missing I can try to implement it, that's I would like a specific sample of few models which you would like to replace with (one?) ormar Model.

[soderluk]

Of course, sorry, if I weren't clear. I'll give an example.

Assuming a database model User

class User(ormar.Model):
    class Meta:
        tablename: str = "users"

    id: int = ormar.Integer(primary_key=True, index=True)
    email: str = ormar.String(max_length=255, nullable=False)
    password: str = ormar.String(max_length=255, nullable=False)
    first_name: str = ormar.String(max_length=255, nullable=False)
    last_name: str = ormar.String(max_length=255, nullable=False)
    category: str = ormar.String(max_length=255, nullable=True)

In case of creating a new user, I'd not need to post the ID, but if using this model as input model, it requires the ID to be given in the body. Also I want to hide the password from the response, and just include the necessary information, such as ID, email, first/last names etc.

So this is how I'd like to use the ormar models:

from fastapi import APIRouter

router = APIRouter()

@router.post("/users/", response_model=User)
async def create_user(user: User):
    return await User.save()

But, currently I need to define multiple Pydantic models for the create input and the response output models (as in FastAPI docs)

import pydantic

class UserBase(pydantic.BaseModel)
    email: str
    first_name: str
    last_name: str

class UserCreateSchema(UserBase):
    password: str
    category: str

class UserSchema(UserBase)
    class Config:
        orm_mode = True

and need to use these Pydantic models in the routes:

from fastapi import APIRouter

router = APIRouter()

@router.post("/users/", response_model=UserSchema)
async def create_user(user: UserCreateSchema):
    stored_user =  await User(**user.dict()).save()
    return UserSchema(**stored_user.dict())

As you can see there's quite a lot of duplication going on here. And when there's models that have lots of properties, it becomes quite cumbersome to copy to Pydantic models all the time.

Hope this clarifies the issue a bit :)

[collerek]

Ok, so it might not be possible fully, but it really depends on a specific use cases. Note that in your example you use pydantic and still need 3 models, and ormar is build on top of pydantic.

But! There are some tricks and nuisances that you can use for your benefit.

Given a model like yours:

class User(ormar.Model):
    class Meta:
        tablename: str = "users"
        metadata = metadata
        database = database

    # dont have to set index on pk key -> pk column is indexed in all backends by default
    # Integer pks are also by default set to autoincrement and are optional (can skip in create)
    id: int = ormar.Integer(primary_key=True)
    email: str = ormar.String(max_length=255, nullable=False)
    # if you want to exclude some field and do not create new models field has to be nullable
    password: str = ormar.String(max_length=255, nullable=True)
    first_name: str = ormar.String(max_length=255, nullable=False)
    last_name: str = ormar.String(max_length=255, nullable=False)
    category: str = ormar.String(max_length=255, nullable=True)

If the field is nullable you don't have to include it in payload during creation as well as in response, so given example above you can:

Response

1. response_model_exclude

Fastapi has response_model_exclude that accepts a set or a list - so that has it's limitation as pydantic accepts also dictionary in which you can set exclude/include columns also on nested models (more on this below)

@app.post("/users/", response_model=User, response_model_exclude={"password"})
async def create_user(user: User):
    return await user.save()

In above example you can pass data like follow:

client = TestClient(app)
    with client as client:
        # note there is no pk
        user = {
            "email": "test@domain.com",
            "password": "^*^%A*DA*IAAA",
            "first_name": "John",
            "last_name": "Doe",
        }
        response = client.post("/users/", json=user)
        created_user = User(**response.json())
        # note pk is populated by autoincrement
        assert created_user.pk is not None
        # note that password is missing
        assert created_user.password is None

2. exclude in Model's dict()

Alternatively you can just return a dict from ormar model. Like this you can also set exclude/include as dict and exclude fields on nested models

@app.post("/users2/", response_model=User)
async def create_user2(user: User):
    user = await user.save()
    return user.dict(exclude={'password'})
    # could be also something like return user.dict(exclude={'category': {'name'}}) to exclude category name

3. don't use response_model and exclude in Model's dict()

Alternatively you can just return a dict from ormar model. Like this you can also set exclude/include as dict and exclude fields on nested models. In theory you loose validation here but since you operate on ormar Models anyway you have already validated your data.

In your example you unnecessarily duplicate this step. Or it's rather even triple checked:

1. ormar.User() model call provides validation
2. your manual call to UserSchema does the validation one more time but on just subset of columns
3. the same call like 2 is done on response_model by fastapi

@router.post("/users/", response_model=UserSchema)
async def create_user(user: UserCreateSchema):
    stored_user =  await User(**user.dict()).save()
    # this is redundant if you provide same response_model as this will be done by fastapi
    # you can just return stored_user
    return UserSchema(**stored_user.dict()) 

So if you skip response_model altogether you can do something like this:

class User2(ormar.Model):
    class Meta:
        tablename: str = "users2"
        metadata = metadata
        database = database

    id: int = ormar.Integer(primary_key=True)
    email: str = ormar.String(max_length=255, nullable=False)
    # note how now password is required
    password: str = ormar.String(max_length=255, nullable=False)
    first_name: str = ormar.String(max_length=255, nullable=False)
    last_name: str = ormar.String(max_length=255, nullable=False)
    category: str = ormar.String(max_length=255, nullable=True)


@app.post("/users4/")
async def create_user4(user: User):
    user = await user.save()
    return user.dict(exclude={'password'})

But in order to:

• exclude password in response
• but still make it required during creation
• and still use response_model

you need two models -> there is really no way around this that I can think of (or even how it would have to work, would require some parameter from fastapi passed to pydantic to mark if we are in request params or in response params. And despite the close cooperation of samuelcolvin and tiangolo those two libraries are separated and doubt that pydantic creators would allow it)

You can freely combine ormar and pydantic models. So if you want to keep fields in ormar required, but still exclude them in response you need just one more model that will exclude those fields so:

# you new pydantic model with just the fields you want
class UserBase(pydantic.BaseModel):
    class Config:
        orm_mode = True

    email: str
    first_name: str
    last_name: str

# note that it's now can use ormar Model User2 with required password
@app.post("/users3/", response_model=UserBase) # use pydantic model here
async def create_user3(user: User2): #use ormar model here
    return await user.save()

Request

Note that the same (need for additional) applies if you want to pass less fields as request parameters but keep them as required on ormar.Model. This is more rare situation, cause it means that you will get the fields value from somewhere else than request (as you not pass them).

That usually means that you can pass server_default to ormar Fields and that will fill the value in sql, or you can use default ormar Fields parameter and pass either static value or a function (with no args) that will fill this field for you. If you pass default or server_default ormar/pydantic field becomes optional and you can use the same model in request and ormar.

In sample below only last_name is required

def gen_pass():
    choices = string.ascii_letters + string.digits + "!@#$%^&*()".split()
    return "".join(random.choice(choices) for _ in range(20))


class RandomModel(ormar.Model):
    class Meta:
        tablename: str = "users"
        metadata = metadata
        database = database

    id: int = ormar.Integer(primary_key=True)
    password: str = ormar.String(max_length=255, default=gen_pass)
    first_name: str = ormar.String(max_length=255, default='John')
    # note that in ormar by default if you not provide autoincrement, default or server_default the field is required
    # so nullable=False - you do not need to provide it for each field
    last_name: str = ormar.String(max_length=255)
    created_date: str = ormar.DateTime(server_default=sqlalchemy.func.now())

# that way only last_name is required and you will get "random" password etc.
# so you can still use ormar model in Request param.
@app.post("/random/", response_model=RandomModel)
async def create_user5(user: RandomModel):
    return await user.save()

# you can pass only last_name in payload but still get the data persisted in db
user3 = {
            'last_name': 'Test'
        }
response = client.post("/random/", json=user3)
assert list(response.json().keys()) == ['id', 'password', 'first_name', 'last_name', 'created_date']

But if you cannot set default you will need additional pydantic Model.

I strongly advise going through whole fastapi tutorial as well as read pydantic documentation as they both have a lot of options and features.

P.S. Probably again gonna include at least some part of this answer in docs 😄

BTW. There was a bug in ormar dict() method which was fixed in 0.5.3. today. So please update before proceeding

[soderluk]

Thank you for the thorough answer again!

I just need to play around to get the behavior I want. I know I've probably done way too much duplication in the endpoints, and I need to clean these up.

I've gone through the FastAPI documentation, but that was a long time ago. Probably need to refresh.

Thanks for your time! You can probably just close this issue, I'll test out your suggestions.

[collerek]

In the so called mean time I had one idea I want to check, but need to check some fastapi internals and performance to see if this is gonna be possible.

If yes, this would allow to exclude different fields in request and response, but not sure if possible yet.

[collerek]

Ok, so in #64 I fixed a bug with pydantic_only field and introduced @property_field decorator that exposes a function from model as a field in results. Updated docs in this regard.

Both can be used to add fields to requests/responses in fastapi - either with validation or not.

Still doing some investigation as of excluding different fields in response/request. Should be possible but in a kind of hacky way (implementation of python specific) so will check if I include this in the end, but let's see.

[hidaris]

wonderful！

[collerek]

Another tool in the toolset for replacing pydantic is Mixins and Models inheritance introduced in 0.8.0.
Check the docs to see what is required and how this works.

[collerek]

Since I enabled discussions I think this issue belongs here ;)

[collerek]

In 0.10.10 i added two new features helping with usage of ormar in fastapi (and not only). Also expanded the documentation in regards of request and response usage with some tricks and tips.

Let me know if i forgot about anything.