model_validate: exclude

[mustafa0x]

Initial Checks

• I have searched Google & GitHub for similar requests and couldn't find anything
• I have read and followed the docs and still think this feature is missing

Description

Lazy loaded orm fields are loaded inadvertently by model_validate. I'd like to be able to exclude fields from model_validate.

Previously discussed here #6861.

With the release of pydantic v2, is it possible to load a model and exclude certain fields when loading using the new model_validate method? I was hoping the new context parameter was able to achieve this, but it looks this does not quite do what I was thinking.

Affected Components

• Compatibility between releases
• Data validation/parsing
• Data serialization - .model_dump() and .model_dump_json()
• JSON Schema
• Dataclasses
• Model Config
• Field Types - adding or changing a particular data type
• Function validation decorator
• Generic Models
• Other Model behaviour - model_construct(), pickling, private attributes, ORM mode
• Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

[sydney-runkle]

@mustafa0x,

Could you use the extra='ignore' config setting? https://docs.pydantic.dev/latest/api/config/#pydantic.config.ConfigDict.extra

[mustafa0x]

Thanks for the reply @sydney-runkle. That seems to be for a different use case.

#6861 explains the intent quite well.

My use case is very similar to this issue #1806, when loading models from an ORM object, if there are related objects within the main ORM object, then they would be lazily loaded. I want to be be able to exclude certain fields from being lazily loaded.

fastapi's response_model_exclude also makes plain the goal here.

You can also use the path operation decorator parameters response_model_include and response_model_exclude.

They take a set of str with the name of the attributes to include (omitting the rest) or to exclude (including the rest).

This can be used as a quick shortcut if you have only one Pydantic model and want to remove some data from the output.

(emphasis mine)

[axelmukwena]

@mustafa0x I had the same issue too and here is how I solved it. I used noload to explicitly tell sqlalchemy to not trigger a relationship load when getting many items, however do it when getting one item

class RoleOrm(OrmBase):
    __tablename__ = "roles"

    # Columns
    uid: Mapped[str] = mapped_column(
        default=lambda: str(uuid4().hex),
        primary_key=True,
        index=True,
        nullable=False,
        unique=True,
    )

    # Relationships
    role_permissions: Mapped[
        list["RolePermissionOrm"]  # noqa: F821 # type: ignore
    ] = relationship(cascade="all, delete-orphan", back_populates="role", lazy="joined")

Role Model

class Role(RoleBase, RoleBlame, RoleAutoGenerated):
    role_permissions: list[RolePermission] | None = Field(
        default=None,
        description="The permissions that the role has",
    )

In Controller/Repository

from fastapi import HTTPException
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import noload

class RoleRepository:
    @staticmethod
    async def get_by_uid(db: AsyncSession, uid: str) -> Role:
        statement = select(RoleOrm).where(RoleOrm.uid == uid)
        results = await db.execute(statement)
        role_orm = results.unique().scalar_one_or_none()
        if not role_orm:
            raise HTTPException(
                status_code=404,
                detail="Role with given id does not exist",
            )
        role = Role.model_validate(role_orm)
        return role

    @staticmethod
    async def get_many(
        db: AsyncSession, skip: int = 0, limit: int = 20
    ) -> tuple[list[Role], str]:
        count = await db.scalar(select(func.count(RoleOrm.uid)))
        if not count:
            return [], "0"
        statement = (
            select(RoleOrm)
            .options(noload(RoleOrm.role_permissions))
            .offset(skip)
            .limit(limit)
        )
        results = await db.execute(statement)
        roles = [Role.model_validate(orm) for orm in results.unique().scalars().all()]
        return roles, str(count)

The result on get_many now just gives me

"role_permissions": []

[mustafa0x]

Thanks @axelmukwena! It seems like that would work, but I prefer to not have to resort to that. I'm hoping pydantic v2 can add back this v1 feature. :)

[ZackPlauche]

I really hope they add this one! I can't believe there's no standard way of handling this!

[NicolaiLolansen]

+1 for this functionality!

[pedrorjbr]

+1

[hmarcuzzo]

+1 for this functionality!