OpenAPI discriminator field disapeared with pydantic v2.5

[BaptisteSaves]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

Hello,

We are using pydantic V2 to generate the openapi schema.
We bumped pydantic from 2.3.0 to 2.5.2 and the discriminator field disappeared in the openapi.yaml.

2.3.0 openapi.yaml

components:
  schemas:
    CreateCustomerOrderDto:
      properties:
        items:
          items:
            oneOf:
              - $ref: '#/components/schemas/Item1'
              - $ref: '#/components/schemas/Item2'
            discriminator:
              propertyName: type
              mapping:
                item1: '#/components/schemas/Item1'
                item2: '#/components/schemas/Item2'

2.5.2 openapi.yaml

components:
  schemas:
    CreateCustomerOrderDto:
      properties:
        items:
          items:
            oneOf:
              - $ref: '#/components/schemas/Item1'
              - $ref: '#/components/schemas/Item2

--> the discriminator tag is not there anymore

Can you tell me if it is normal this field disappeared ?

Thanks a lot for your help

Example Code

class Item1(BaseModel):
    item_type: Annotated[Literal[temType.ITEM1], Field(alias="type")]
    XXX 


class List(BaseModel):
    items: list[
        Annotated[
            Union[
                Item1,
                Item2,
            ],
            Field(discriminator="item_type"),
        ]
    ]

Python, Pydantic & OS Version

pydantic version: 2.5.2
        pydantic-core version: 2.14.5
          pydantic-core build: profile=release pgo=true
                 install path: /home/XXX/.cache/pypoetry/virtualenvs/XXX-TXFlUrBt-py3.11/lib/python3.11/site-packages/pydantic
               python version: 3.11.5 (main, Sep  2 2023, 14:16:33) [GCC 13.2.1 20230801]
                     platform: Linux-6.6.1-arch1-1-x86_64-with-glibc2.38
             related packages: typing_extensions-4.8.0 pydantic-settings-2.1.0 fastapi-0.104.1 mypy-1.7.1

[sydney-runkle]

@BaptisteSaves,

Thanks for reporting this. Looks like a bug! Marking with good first issue and help wanted in case anyone wants to pick this up to help with our 2.6.0 release!

[StrawHatDrag0n]

@sydney-runkle I would like to take this issue. Can you point me to where I should start looking ?

[sydney-runkle]

@StrawHatDrag0n,

Fantastic 😄! My first thought is that if this was changed in 2.5, it might have had to do with the changes we made to discriminator - ex, adding CallableDiscriminator support.

Additionally, you can probably focus on the JSON schema generation logic for a given model. Lmk if this helps, happy to elaborate more if needed!

[StrawHatDrag0n]

@BaptisteSaves I am getting the discriminator field using the example you provided. I had to modify the example to make it work.

Please check the example and output below, and let me know if you are still getting the issue.

Modified Example Code

class Item1(BaseModel):
    item_type: Annotated[Literal[1], Field(alias="type")] 
class Item2(BaseModel):
    item_type: Annotated[Literal[2], Field(alias="type")]
class List(BaseModel):
    items: list[
        Annotated[
            Union[
                Item1,
                Item2,
            ],
            Field(discriminator="item_type"),
        ]
    ]

Output from List.model_json_schema()

{
  "$defs": {
    "Item1": {
      "properties": {
        "type": {
          "const": 1,
          "title": "Type"
        }
      },
      "required": [
        "type"
      ],
      "title": "Item1",
      "type": "object"
    },
    "Item2": {
      "properties": {
        "type": {
          "const": 2,
          "title": "Type"
        }
      },
      "required": [
        "type"
      ],
      "title": "Item2",
      "type": "object"
    }
  },
  "properties": {
    "items": {
      "items": {
        "discriminator": {
          "mapping": {
            "1": "#/$defs/Item1",
            "2": "#/$defs/Item2"
          },
          "propertyName": "type"
        },
        "oneOf": [
          {
            "$ref": "#/$defs/Item1"
          },
          {
            "$ref": "#/$defs/Item2"
          }
        ]
      },
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "List",
  "type": "object"
}

[BaptisteSaves]

@StrawHatDrag0n

I realize I forgot to mention the impacted generated yaml is the one we get from FastAPI, sorry.

I managed to reproduce the problem with a simple example.

The discriminator field in the generated OpenAPI only disapears if the examples field is present in Body.
If I use the deprecated example, I will find discriminator in the OpenApi.

This change of behavior happens when I change the Pydantic version from 2.3.0 to 2.5.0 without changing the FastApi version

At this point, It seems examples is directly sent to Pydantic FieldInfo but tell me if I should open the ticket on FastApi side.

Thanks for your help 🙏

fastapi = "0.104.1"
pydantic = "2.5.0"
pydantic-settings = "2.1.0"

from enum import StrEnum
from enum import auto
from typing import Annotated
from typing import Literal
from typing import Union

from fastapi import APIRouter
from fastapi import Body
from pydantic import BaseModel as BaseModelPydantic
from pydantic import Field

my_router = APIRouter(prefix="/api/create")


class ItemType(StrEnum):
    ITEM1 = auto()
    ITEM2 = auto()


class CreateItem1(BaseModelPydantic):
    item_type: Annotated[Literal[ItemType.ITEM1], Field(alias="type")]
    id: int


class CreateItem2(BaseModelPydantic):
    item_type: Annotated[Literal[ItemType.ITEM2], Field(alias="type")]
    id: int


class CreateObjectDto(BaseModelPydantic):
    id: int
    items: list[
        Annotated[
            Union[
                CreateItem1,
                CreateItem2,
            ],
            Field(discriminator="item_type"),
        ]
    ]


@my_router.post(
    "",
    status_code=201,
)
def post_object(
    create_object_dto: Annotated[
        CreateObjectDto,
        Body(examples=[{"id": 1, "items": [{"id": 3, "type": "ITEM1"}]}]),
    ],
) -> None:
    pass

[sydney-runkle]

@StrawHatDrag0n @BaptisteSaves,

I think this is indeed an issue with pydantic then if the version bump on our end caused the issue. @StrawHatDrag0n, you'll probably have to dig a bit into the fastapi code to see how they're generating the schema for the pydantic models to see where the discriminator is falling through the gaps.

[StrawHatDrag0n]

@sydney-runkle Do you have a guess where this code might be in fastAPI ?

[sydney-runkle]

I'm not sure.

@Kludex, any ideas?

[StrawHatDrag0n]

@BaptisteSaves @sydney-runkle I am trying to create a toy project using the example code provided but I am not able to get the doc page for the OpenAPI spec, for the url 127.0.0.1:8000/docs I am getting Not Found

Below is the example code:

from enum import StrEnum
from enum import auto
from typing import Annotated
from typing import Literal
from typing import Union

from fastapi import APIRouter
from fastapi import Body
from pydantic import BaseModel as BaseModelPydantic
from pydantic import Field

my_router = APIRouter()


class ItemType(StrEnum):
    ITEM1 = auto()
    ITEM2 = auto()


class CreateItem1(BaseModelPydantic):
    item_type: Annotated[Literal[ItemType.ITEM1], Field(alias="type")]
    id: int


class CreateItem2(BaseModelPydantic):
    item_type: Annotated[Literal[ItemType.ITEM2], Field(alias="type")]
    id: int


class CreateObjectDto(BaseModelPydantic):
    id: int
    items: list[
        Annotated[
            Union[
                CreateItem1,
                CreateItem2,
            ],
            Field(discriminator="item_type"),
        ]
    ]


@my_router.post(
    "/api/create",
    status_code=201,
)
def post_object(
    create_object_dto: Annotated[
        CreateObjectDto,
        Body(examples=[{"id": 1, "items": [{"id": 3, "type": "ITEM1"}]}]),
    ],
) -> None:
    return {"Hello": "Hello"}

@my_router.get(
    path="/Hello"
)
def hello():
    return {"Hello": "First Page"}

any ideas ?

Edit: Ignore, just noticed that I was using APIRouter instead of FastAPI

[StrawHatDrag0n]

@BaptisteSaves Can you show me how you create the YAML ?

[BaptisteSaves]

@StrawHatDrag0n Here is the file and env, I ran it with uvicorn main:app --reload then going to http://127.0.0.1:8000/openapi.json

from enum import StrEnum, auto
from typing import Annotated, Union, Literal

from fastapi import FastAPI, Body
from pydantic import BaseModel as BaseModelPydantic, Field

app = FastAPI()


class ItemType(StrEnum):
    ITEM1 = auto()
    ITEM2 = auto()


class CreateItem1(BaseModelPydantic):
    item_type: Annotated[Literal[ItemType.ITEM1], Field(alias="type")]
    id: int


class CreateItem2(BaseModelPydantic):
    item_type: Annotated[Literal[ItemType.ITEM2], Field(alias="type")]
    id: int


class CreateObjectDto(BaseModelPydantic):
    id: int
    items: list[
        Annotated[
            Union[
                CreateItem1,
                CreateItem2,
            ],
            Field(discriminator="item_type"),
        ]
    ]


@app.post(
    "/api/create",
    status_code=201,
)
def post_object(
        create_object_dto: Annotated[
            CreateObjectDto,
            Body(examples=[{"id": 1, "items": [{"id": 3, "type": "ITEM1"}]}]),
        ],
) -> None:
    return {"Hello": "Hello"}

pyproject.toml

[tool.poetry]
name = "fast-api-pydantic"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = ">3.11"
fastapi = "0.105.0"
uvicorn = { version = "0.24.0.post1", extras = ["standard"] }
pydantic = "2.5.2"

[StrawHatDrag0n]

@sydney-runkle I think I have found the issue. I see that when we create core_schema here, we generate the schema here.

I think the issue is where we generate the schema here we add the definition of the parent schema before all the children schemas which the current parent schema depends on.

For the example code: definition of CreateObjectDto added before the definition of CreateItem1 and CreateItem2. This causes issues here, which causes the definition of CreateObjectDto to be processed first. Hence when we try to create the json for openapi we don't have access to definitions of CreateItem1 and CreateItem2 which causes the discriminator [here] field to be empty hence it does not show up in the yaml.

I think the solution could be to delay the addition of root schema definition till all the children schemas are added to the core_schema.

Let me know if this makes sense.

PS: Can you assign me this issue ?

[balint-backmaker]

Hello, until folks fix the deeper issue, I found a workaround if you can afford to have a base class.

When you introduce the field that you'll discriminate on via a parent class, the discriminator seems to be retained(pydantic 2.5.3, fastapi 0.108.0)

class SomeAnimal(BaseModel):
    animal_kind: str


class Cat(SomeAnimal):
    animal_kind: Literal["cat"] = "cat"
    cat_name: str


cat_example = Cat(cat_name="Garfield")


class Dog(SomeAnimal):
    animal_kind: Literal["dog"] = "dog"
    dog_name: str

class AnimalRequest(BaseModel):
    animals: list[Annotated[Cat | Dog, Field(discriminator="animal_kind")]]

[StrawHatDrag0n]

@sydney-runkle I am not sure what the unit test for this change might be ? Should I just check against the the correct OpenAPI json ? If yes, then where should I add the unit test ?

[sydney-runkle]

@StrawHatDrag0n,

Thanks for the fix! Yep, if you could check against the correct JSON, that'd be great. The test_discriminated_union.py file would be best. I'll review tomorrow morning once you add the test. Thanks!!

[sydney-runkle]

Going to reopen this. The fix used for this ended up breaking more important things, so we're going to revert that change and will need to find another workaround for this issue.

[StrawHatDrag0n]

Hi @sydney-runkle! Can you point me to where this change was causing issues? I will try to look for a work around.

[sydney-runkle]

@StrawHatDrag0n,

See this issue: #8689. Thanks!

[BaptisteSaves]

Hello @sydney-runkle @StrawHatDrag0n,

Do you have any news for a possible fix on this regression 🙏

[sydney-runkle]

@BaptisteSaves,

Not yet, though I'll take a closer look this week. This looks pretty related: #8628

[sydney-runkle]

@StrawHatDrag0n,

Could you please verify that this issue is fixed on the fix-discriminators-for-dataclasses branch?

[sydney-runkle]

Ah actually I think this isn't fixed by that change, though maybe that gets us a good start!

[sydney-runkle]

@BaptisteSaves @StrawHatDrag0n,

Is there a purely pydantic reproduction of this issue?

[BaptisteSaves]

@sydney-runkle I am not sure.
What I tested is that model_json_schema() has the same result in both versions

Also the issue is linked to the usage of alias on the discriminator field. Without alias, the resultant OpenApi.yaml is correct