Global filter/enhancer for responses

[LKay]

First check

• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.

Description

Is it possible to somehow globally filter response that is returned from a route method? What I have in mind if there is any proper way to additionally filter response based on some context, or FastAPI dependencies (Dependency, Security etc.).

My use case is:

• I have some authorization mechanism with different roles (let's say admin and user for simplicity).
• Data is typical CRUD model stored in the database, and all records have boolean active column to handle "soft" delete and not remove records entirely from the database. If record was "soft" deleted the flag would be set to false.
• As user with admin role I can access all records, even if they were "soft" deleted, and user role can only access the ones that have been not "soft" deleted.
• Record from database is represented by pydantic model:

class MyModel(BaseModel):
    id: str
    name: str
    active: bool = True

• I have some route defined which fetches a model from database and returns it:

async def get_model(
    id: str = Path(...),
    current_user: User = Depends(get_current_user)
) -> MyModel:
    # The `current_user` will be a model representing currently authorized user. 
    # In case of admin user inactive record might also be returned here.
    return await fetch_model_from_db(id, user.is_admin())

• The output model will have the attribute active present in any case, but I want to only preserve it for admin user and hide it from regular user role (it can access only records with active flag as true anyway). In short I would like to additionally parse response based on current_user role.

What I found

First thing I thought about was excluding certain attributes as per docummentation https://fastapi.tiangolo.com/tutorial/response-model/#use-the-response_model_exclude_unset-parameter but this only allows excluding attributes explicitly without any additional context.

Then I thought about making custom middleware, which serves the purpose of processing response object before it is returned to the client, but I couldn't find a clear way to pass down the context as current_user, so I could use that to process response and exclude unwanted attributes from response body. This however would add additional overhead as the response body is already serialized at this stage and would need to be deserialized -> filtered -> serialized again.

Additional context

I could add additional function like this and call it as the last thing in every route, but it's way too verbose and I'd rather handle it more generic way:

def filter_admin_attributes(input: Dict, is_admin: bool, attributes: Set[str]) -> Dict:
   return {k: v for k, v in input.items() if not is_admin and k in attributes}

async def get_model(
    id: str = Path(...),
    current_user: User = Depends(get_current_user)
) -> Dict:
    # The `current_user` will be a model representing currently authorized user. 
    # In case of admin user inactive record might also be returned here.
    result = await fetch_model_from_db(id, user.is_admin())
	return filter_admin_attributes(result.dict(), user.is_admin(), {'active',})

[dmontagu]

I'm not sure there is a good way to handle this. There are many different ways this could be accomplished (the one you described for example), but I can't think of any off the top of my head that are both simple and clean.

I think part of the problem is that OpenAPI specs don't really have a way to have different specs for the same endpoint under different conditions (admin vs. not), which is what I think you are going for.

Since OpenAPI integration is a major goal and selling point of FastAPI, I think this may at least partially explain why implementing this pattern isn't more convenient.

---

Personally, if I wanted to implement something like this in my own projects (and I had control over the API design), I would probably create separate endpoints for admin access, and use the same logic in the admin and non-admin endpoint, but use a different pydantic model as the response_model for the admin endpoints.

But obviously that pattern may not work for you or may be otherwise undesirable to you, just something to consider!

[LKay]

Thank you for answering. Actually, I tried the approach in couple of my other projects (written in other languages and frameworks) and had set of endpoints for admin and non-admin users. This works fine for few basic resources but it adds a lot of overhead and redundant endpoints while the application grows. For this reason I'd rather handle that on request/response processing level providing extra context (auth in this case) and keep the number of endpoints to minimum. I was successful implementing that using other languages and frameworks, and just hoped this pattern could be easily ported to FastAPI.

It is possible to get ahead of handling Union types and define model as:

class MySimpleModel(BaseModel):
  id: str
  name: str

class MyModel(MySimpleModel):
  active: bool = True


async def get_model(
    id: str = Path(...),
    current_user: User = Depends(get_current_user)
) -> Union[MyModel, MySimpleModel]:
    ...

This would keep OpenAPI schema contain both admin and non-admin models, but would still require to format response on the fly somehow generic.

I could create a global middleware to handle authentication and overloading Request object along with the auth logic (to make passes AuthRequest enhanced with retrieved current user) as per https://fastapi.tiangolo.com/tutorial/custom-request-and-route/. This way I would have full control what's coming out, however I think it might be an overkill. Also I think this would probably require to define auth details in both middleware and dependencies (as per https://fastapi.tiangolo.com/tutorial/security/get-current-user/) so OpenAPI schema generator picks up everything correctly.

[euri10]

I'd take a step backwards and deal with that not on the application level but on the db level.
It's very simple, way cleaner and more secure, I know people just connect to db most of the time with superuser rights, I think it's just asking for troubles.
I'm doing something similar in one of my project

For that you create appropriate roles that would restrict column selection with something like
grant select (id, name) on shcema.table to role_user;
grant select on shcema.table to role_admin;

Your Depends then switch role accordingly, and you're set.

on top of that you can even use row level security after, never tested it so far but will eventually !

[dmontagu]

@euri10 I personally think that sounds way more complex than managing it in at the application level, but it would definitely be more secure!

But I don't think it resolves the issue if a response_model and OpenAPI spec are involved -- you'd still have to pick the right model for the admin vs. non-admin endpoints. I know you don't typically use a response_model so obviously this approach would work nicely if you can just json-encode the records straight from the db, but I think it might not work as nicely if the goal is to keep things pydantic-ish in the endpoint returns.

[dmontagu]

@LKay I also think you could make this work through the use of a decorator that modifies the endpoint to remove the sensitive fields from the response if the user isn't an admin. You'd have to write logic for how to convert the admin model/response-data to a non-admin model/response-data for each possible (sensitive) return type, but presumably you'd have to do that somewhere no matter what.

Writing the decorator might be a little bit tricky to make sure it does everything you need it to do for fastapi compatibility, but it is definitely possible. If that sounds like an interesting approach to you but you have trouble figuring it out how to make it work let me know.

[LKay]

@euri10 Having this active column was just an example, in actual project I have some more attributes which are only returned if admin user queries for it. Your solution to make permission on db level is bulletproof on data level, but adds much more complexity on the application level. Going with such approach would either hold two separate connection pools to database set up and ready to use (for admin and non admin querying) or what's worse, would require to establish db connection with different db users based on the authorization level from application. Database users and application users are different entities.

@dmontagu I already have some a bit verbose solution, but I wan't to make it more generic during a refactoring we want to do in the near future so will try it out and try to leave some follow up here.

[euri10]

Going with such approach would either hold two separate connection pools to database set up and ready to use (for admin and non admin querying) or what's worse, would require to establish db connection with different db users based on the authorization level from application. Database users and application users are different entities.

edit : ok I explained badly,

your db has 2 roles admin_role and user_role:

create role admin_role nologin;
create role user_role nologin;

you restrict columns for roles with something like:
grant select(id, active, name) on todos to admin_role;
grant select(id, name) on todos to admin_role;

your fastapi app connects to your db with let's say app role, that role can login

create role app with
	login
	encrypted password 'secret'
	in role admin_role, user_role;

now your Depends will connect with app to your db, and it will set role admin_role if he's an admin
there's absolutely no duplication

[tiangolo]

I think the options are what you already discussed here. Your custom function, a custom decorator, a custom APIRoute class, or a middleware.

The other thing I can think of is using validators in Pydantic models and using tricks to make them receive also the current user and use it to filter the data. But I would personally discard that option in favor of any of the other ones.

[github-actions[bot]]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.