openapi.json fails to be generated for nested models

[LKay]

Describe the bug
Fails to autogenerate docs.

To Reproduce

1. Create a response model with other nested models and try to generate openapi.json or open autogenerated docs.
2. Application throws an error

Expected behavior
Generate openapi.json and open autogenerated docs without problems.

Environment:

• OS: macOS
• FastAPI Version: 0.30.0
• Python version: 3.7.3

Additional context

My models look like this:

class PaginatedItems(Generic[Item], BaseModel, abc.ABC):
    items: List[Any]
    has_after: bool = False
    has_before: bool = False

class OrderUser(BaseModel):
    name: Optional[str] = None
    email: EmailStr


class Order(BaseModel):
    order_id: UUID
    amount: Decimal
    currency: str

    # Users part
    merchants: List[OrderUser]
    handlers: List[OrderUser]

class PaginatedOrders(PaginatedItems[Order]):
    items: List[Order] = list()

My route definition looks like that:

@router.get(orders_uri,
            response_model=PaginatedOrders
async def list():
	...

This results in the following exception when trying to open autogenerated docs:

pydantic.error_wrappers.ValidationError: 4 validation errors
schemas -> OrderUser
  value is not a valid dict (type=type_error.dict)
schemas -> OrderUser
  value is not a valid dict (type=type_error.dict)
components -> schemas
  value is not none (type=type_error.none.allowed)
components
  value is not none (type=type_error.none.allowed)

When I comment out the response_model=PaginatedOrders part from route definition everything works, but docs obviously miss response type.

[dmontagu]

I believe this was very recently fixed in the most recent version of pydantic (not yet supported by FastAPI, if I recall correctly). I'm currently using the following as a workaround:

# my_fast_api.py

from typing import Any, Dict, Sequence, Set, Type

from fastapi import FastAPI, routing
from fastapi.encoders import jsonable_encoder
from fastapi.openapi.constants import REF_PREFIX
from fastapi.openapi.models import OpenAPI
from fastapi.openapi.utils import get_openapi_path
from fastapi.utils import get_flat_models_from_routes
from pydantic import BaseModel
from pydantic.schema import get_model_name_map, model_process_schema
from starlette.routing import BaseRoute


def get_model_definitions(
    *, flat_models: Set[Type[BaseModel]], model_name_map: Dict[Type[BaseModel], str]
) -> Dict[str, Any]:
    definitions: Dict[str, Dict[Any, Any]] = {}
    for model in flat_models:
        m_schema, m_definitions = model_process_schema(model, model_name_map=model_name_map, ref_prefix=REF_PREFIX)
        # definitions.update(m_definitions)
        model_name = model_name_map[model]
        definitions[model_name] = m_schema
    return definitions


def get_openapi(
    *,
    title: str,
    version: str,
    openapi_version: str = "3.0.2",
    description: str = None,
    routes: Sequence[BaseRoute],
    openapi_prefix: str = "",
) -> Dict[str, Any]:
    info = {"title": title, "version": version}
    if description:
        info["description"] = description
    output = {"openapi": openapi_version, "info": info}
    components: Dict[str, Dict[str, Any]] = {}
    paths: Dict[str, Dict[str, Any]] = {}
    flat_models = get_flat_models_from_routes(routes)
    model_name_map = get_model_name_map(flat_models)
    definitions = get_model_definitions(flat_models=flat_models, model_name_map=model_name_map)
    for route in routes:
        if isinstance(route, routing.APIRoute):
            result = get_openapi_path(route=route, model_name_map=model_name_map)
            if result:
                path, security_schemes, path_definitions = result
                if path:
                    paths.setdefault(openapi_prefix + route.path_format, {}).update(path)
                if security_schemes:
                    components.setdefault("securitySchemes", {}).update(security_schemes)
                if path_definitions:
                    definitions.update(path_definitions)
    if definitions:
        components.setdefault("schemas", {}).update(definitions)
    if components:
        output["components"] = components
    output["paths"] = paths
    openapi = OpenAPI(**output)
    return jsonable_encoder(openapi, by_alias=True, include_none=False)


class MyFastAPI(FastAPI):
    def openapi(self) -> Dict[str, Any]:
        if not self.openapi_schema:
            self.openapi_schema = get_openapi(
                title=self.title,
                version=self.version,
                openapi_version=self.openapi_version,
                description=self.description,
                routes=self.routes,
                openapi_prefix=self.openapi_prefix,
            )
        return self.openapi_schema

I then import MyFastAPI as FastAPI; I'll swap it out once this is fixed again.

(Basically the only change is the commented line in get_model_definitions which is currently overwriting previously computed schemas with None in some situations involving nested models.)

I am not 100% sure that this handles all edge cases properly, but it generated everything correctly in my api (many models of varied shape and nesting).

[tiangolo]

Thanks for the help @dmontagu!

The latest Pydantic is now included/supported in the latest FastAPI 0.33.0.

Can you update and check?

[dmontagu]

@tiangolo yes, I’m happy to say I’ve removed the above workaround from my projects. Thanks!

[LKay]

I can confirm it now works fine! Thanks for fixing it so quickly!

[unography]

@tiangolo I am facing a similar issue,

my code is currently like this -

class SimilarProducts(BaseModel):
    count: int
    productIds: List[str]

@router.get("/api/v1/similar/products/{user_id}/{product_id}")
async def get_similar_products(
    user_id: str = Path(..., title="userid"),
    product_id: str = Path(..., title="productid"),
    response_model=SimilarProducts, 
):
    try:
        dummy_result = {
            "count": 1,
            "productIds": [product_id]
        }
        return dummy_result
    except Exception as e:
        raise HTTPException(status_code=202, detail="Exception: {}".format(e))

When trying to open /docs I get this error -

TypeError: Object of type 'MetaModel' is not JSON serializable

I'm using fastapi version 0.33.0
pydantic version 0.30

[euri10]

what is MetaModel

…

On Tue, Jul 23, 2019 at 10:42 AM Dhruv Karan ***@***.***> wrote: @tiangolo <https://github.com/tiangolo> I am facing a similar issue, my code is currently like this - class SimilarProducts(BaseModel): count: int productIds: List[str] @router.get("/api/v1/similar/products/{user_id}/{product_id}") async def get_similar_products( user_id: str = Path(..., title="userid"), product_id: str = Path(..., title="productid"), response_model=SimilarProducts, ): try: dummy_result = { "count": 1, "productIds": [product_id] } return dummy_result except Exception as e: raise HTTPException(status_code=202, detail="Exception: {}".format(e)) When trying to open /docs I get this error - TypeError: Object of type 'MetaModel' is not JSON serializable — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#383>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAINSPXWMX55R3KMDZTTURDQA276LANCNFSM4ICBLKDA> .

-- benoit barthelet http://pgp.mit.edu/pks/lookup?op=get&search=0xF150E01A72F6D2EE

[unography]

@euri10 it's not there in my code, I thought it was something inside Pydantic?

[euri10]

oups sorry I think your mistake is putting response_model=SimilarProducts, in the wrong spot, it's in the @router part

[unography]

oh yes, it works, my bad.
thanks! @euri10

[tiangolo]

Thanks @dmontagu and @LKay for reporting back (and closing the issue).

Thanks @euri10 for your help! I'm glad it works now @unography.

[yeus]

I feel like reopening this bug. I get the same erros as above by using a nested model like this:

Dict[str, List[Tuple[str, float]]]

I am on:
fastapi version: '0.68.1', python 3.8.10, ubuntu 20.04.

this code here:

from typing import List, Dict, Tuple

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Answers(BaseModel):
    answers: Dict[str, List[Tuple[str, float]]] = {}

@app.get("/qa", response_model=Answers)
async def qatask(request: str):
    return Answers()

if __name__ == "__main__":
    import uvicorn
    uvicorn.run("bugfind:app", host="0.0.0.0", port=5000, reload=True,
                workers=1)  # , debug=True)

causes the same error to appear.

Traceback (most recent call last):
  File "//home/name/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "//home/name/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "//home/name/.local/lib/python3.8/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "//home/name/.local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "//home/name/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "//home/name/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "//home/name/.local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "//home/name/.local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "//home/name/.local/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "//home/name/.local/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "//home/name/.local/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
    response = await func(request)
  File "//home/name/.local/lib/python3.8/site-packages/fastapi/applications.py", line 161, in openapi
    return JSONResponse(self.openapi())
  File "//home/name/.local/lib/python3.8/site-packages/fastapi/applications.py", line 136, in openapi
    self.openapi_schema = get_openapi(
  File "//home/name/.local/lib/python3.8/site-packages/fastapi/openapi/utils.py", line 410, in get_openapi
    return jsonable_encoder(OpenAPI(**output), by_alias=True, exclude_none=True)  # type: ignore
  File "pydantic/main.py", line 406, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 4 validation errors for OpenAPI
components -> schemas -> Answers -> properties -> answers -> additionalProperties -> items -> items
  value is not a valid dict (type=type_error.dict)
components -> schemas -> Answers -> properties -> answers -> additionalProperties -> $ref
  field required (type=value_error.missing)
components -> schemas -> Answers -> properties -> answers -> additionalProperties
  value could not be parsed to a boolean (type=type_error.bool)
components -> schemas -> Answers -> $ref
  field required (type=value_error.missing)

[yeus]

not sure, if I should open a new issue for this?

[Nicholas-Schaub]

I have similar code structure to @yeus and also am running into this issue. I only have issues with response_model values that have nested iterable types (i.e. List, Tuple, Dict). I did some additional testing that made me think this was a Windows vs Linux issue, but upon further investigation I found that something became broken in the most recent version of fastapi.

This code will generate proper documentation when using fastapi[standard]==0.65.2 but does not generate proper documentation in fastapi[standard]==0.68.1.

# Obviously skipping some imports

class DataResponse(BaseModel):
    data: typing.List[int]
    limits: typing.List[typing.Tuple[float,float]]
    counts: typing.Tuple[int,int]

@app.post("/bins",
          status_code=status.HTTP_200_OK,
          tags=["data"],
          response_model=DataResponse)
def bins(input: DataSelection):

    return None

if __name__ == "__main__":
    import uvicorn

    uvicorn.run("server:app", host="127.0.0.1", port=8001, log_level="info")

When the above is run using fastapi[standard]==0.65.2, I can access documentation at 127.0.0.1:8001/docs.

When the above is run using fastapi[standard]==0.68.1, I receive the following error:

  File "pydantic\main.py", line 406, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 3 validation errors for OpenAPI
components -> schemas -> DataResponse -> properties -> limits -> items -> items
  value is not a valid dict (type=type_error.dict)
components -> schemas -> DataResponse -> properties -> counts -> items
  value is not a valid dict (type=type_error.dict)
components -> schemas -> DataResponse -> $ref
  field required (type=value_error.missing)

For now, I am just downgrading fastapi so that documentation works.

[yeus]

@Nicholas-Schaub I decided to reopen a new issue then in order to handle this...

[Nicholas-Schaub]

@yeus I happened to stumble upon it by accident and reposted 👍