response processing pipeline has too much magic

[toppk]

test-fastapijson.py.txt

using the latest version from git, I notice some automagic in how response are modified that make it difficult to impact json encoding.

Here are my complaints:

1. it seems to control json encoding, you must call JSONResponse (or private implementation) yourself. using response_class won't work because the data has already been modified

2. types that cannot encode to json will be done itself. an object is something that shouldn't be able to encode to json without help, but somewhere there is some code that does it automatically.

The documentation talks about returning response directly or returning the data. It seems that there's more happening behind the scenes then is discussed.

My expectation is that if I return something that isn't json encodable, then I get that exception. For other reasons I'm not able to use response_model at the moment. I wonder if I were to use that, I'd have the ability to plug in the type->json converter there.

The sample code shows three functions, which show the behavior i'm exploring. I'm happy to explain further.

[dmontagu]

Another approach is to manually encode and return the result (e.g., using fastapi.encoders.jsonable_encoder).

I think there is some work happening in pydantic right now to support custom json encoders; I suspect once that is complete, it will make sense to integrate better support for custom encoders into FastAPI.

[dmontagu]

Related -- if you want to look in the source code to see precisely how the result of your endpoint call is being converted into a serialized response (the "magic"), the relevant functions to look at are:

• fastapi.routing.get_app,
• fastapi.routing.serialize_response (called in get_app), and
• fastapi.encoders.jsonable_encoder (called in serialize_response)

It actually isn't that magical under the hood; I think the bulk of the code is just handling common edge cases (e.g., uuid.UUID isn't json-encodable by json.dumps), or accommodating for pydantic capabilities.

[toppk]

Thanks for the pointers. I'm sure it will look less magically if I read it a bit more . With that, I was able to find a workaround:

pydantic.json.ENCODERS_BY_TYPE[Complex] = str

for reference, I was hitting this block in
fastapi.encoders.jsonable_encoder

`

  except KeyError as e:
    try:
        data = dict(obj)
    except Exception as e:
        errors.append(e)
        try:
            data = vars(obj)
        except Exception as e:
            errors.append(e)
            raise ValueError(errors)`

I think there should be at least a warning message when you hit this code path.

I also noticed that jsonable_encoder has a attribute called custom_encoder, that isn't plumbed back to where the user can use set it with a useful value. I'm happy to submit a patch to plumb it back, but maybe it's better just to set this globally in pydantic.json as I've did above.

[tiangolo]

Thanks @dmontagu for the help here!

I'm glad you solved your issue @toppk .

You might also be interested in Pydantic's Config encoder: https://pydantic-docs.helpmanual.io/usage/exporting_models/#modeljson as it probably does what you need.

[github-actions]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.