TypeVar in route handler parameters recognized as strings not their constraint or bound types.

[dmajere]

Describe the bug
TypeVar in route handler parameters recognized as strings not their constraint or bound types.

To Reproduce

• create generic typevars
• use typevars as typehints for route handler parameters

from typing import TypeVar
from starlette.responses import PlainTextResponse

str_T = TypeVar('str_T', bound=str)
bool_T = TypeVar('bool_T', bound=bool)
int_T = TypeVar('int_T', bound=int)
const_T = TypeVar('const_T', int, float)

@router.get('/str_T/{elem}')
async def generic_str(elem: str_T):
    return PlainTextResponse(str(type(elem)))

@router.get('/bool_T/{elem}')
async def generic_bool(elem: bool_T):
    return PlainTextResponse(str(type(elem)))

@router.get('/int_T/{elem}')
async def generic_int(elem: int_T):
    return PlainTextResponse(str(type(elem)))

@router.get('/const_T/{elem}')
async def generic_const(elem: const_T):
    return PlainTextResponse(str(type(elem)))

curl http://localhost:8000/str_T/str
<class 'str'>%                                                                                                                                                                                               

curl http://localhost:8000/int_T/1
<class 'str'>%

curl http://localhost:8000/bool_T/True 
<class 'str'>%

curl http://localhost:8000/const_T/4
<class 'str'>%

Expected behavior
When for generics defined with constraints type casting is probably would be impossible, since
we have a list of possible types, for bounded typevars fastapi should type cast to bounded type.

[dmontagu]

Is there a desired use case for this? I'm having a hard time imagining a concrete scenario where supporting bounded typevars is more appropriate than just placing the bound itself in the annotation. (Not to say a reasonable application doesn't exist, just that I can't think of one.)

The typing API is not especially clean, and as a result I think supporting this feature might have the potential to add serious backwards-compatibility pain over time. As a result, I think it would be better for fastapi to not support this feature unless it enabled a capability that is currently missing or difficult to achieve cleanly.

[dmajere]

No, I would say that there is no strong desire.
It's more for documentation-like purposes, usually It gives you more information on what code is doing when you see domain-specific types.
However, yes, this is more of a nice-to-have than a necessity.

[dmontagu]

@dmajere if you want domain-specific types, you might try typing.NewType (which is supported by fastapi currently). Based on your description it sounds like this may actually be more appropriate than a TypeVar anyway.

[dmajere]

@dmontagu
This works perfectly, thanks.

[dmontagu]

Glad to hear it! I use typing.NewType extensively in my own projects.

Especially for object IDs -- this makes it much easier to prevent silly mistakes where you accidentally call a function expecting an ID of one type of object with an ID of another type. If everything is an integer or a UUID, it can be very painful to debug the problem, but using typing.NewType means mypy will catch it for you without any effort.

[euri10]

I'm a mypy newb and didn;t get the subtlety of this, care giving a little snippet @dmontagu ?

…

On Wed, Sep 18, 2019 at 6:30 AM dmontagu ***@***.***> wrote: Glad to hear it! I use typing.NewType extensively in my own projects. Especially for object IDs -- this makes it much easier to prevent silly mistakes where you accidentally call a function expecting an ID of one type of object with an ID of another type. If everything is an integer or a UUID, it can be very painful to debug the problem, but using typing.NewType means mypy will catch it for you without any effort. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#533>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAINSPT6N2OPP6GITZULKRLQKGVFDANCNFSM4IW4EZSA> .

-- benoit barthelet http://pgp.mit.edu/pks/lookup?op=get&search=0xF150E01A72F6D2EE

[dmontagu]

Here's an example that demonstrates the value:

Imagine you have a database of products, and have an API for marking a product as "saved" for a particular user. Without NewType, some of the functions/classes involved might look like:

import uuid

from pydantic import BaseModel

class Product(BaseModel):
    product_id: uuid.UUID
    title: str

class SavedProductRecord(BaseModel):
    saved_product_id: uuid.UUID
    product_id: uuid.UUID
    user_id: uuid.UUID

# "CRUD" functions
def read_saved_product(saved_product_id: uuid.UUID) -> SavedProductRecord:
    pass

def read_product(product_id: uuid.UUID) -> Product:
    pass

Now pretend you get a SavedProductRecord instance from somewhere (e.g., after loading all of the records for a particular user), and then want to load the associated product. You go to call read_product, but your mind slips for a second and you pass the function the saved_product_id, rather than the product_id:

record: SavedProductRecord = ...  # pretend this came from somewhere
read_product(record.saved_product_id)

You won't be able to find the matching product because you gave the wrong id (should have been product_id, not saved_product_id), but there's nothing in the code here that could be detected as a bug.

Since both product_id and saved_product_id have the same type and similar names, this error is hard to detect either algorithmically or as a human. But in an intuitive sense, the product_id and saved_product_id are a totally different "types" of things -- they should never be used interchangeably.

typing.NewType gives a way to formalize this idea in a way that works with type checkers.

Here's what the above code would look like with NewType:

from typing import NewType
import uuid

from pydantic import BaseModel

SavedProductID = NewType("SavedProductID", uuid.UUID)
ProductID = NewType("ProductID", uuid.UUID)
UserID = NewType("UserID", uuid.UUID)

class Product(BaseModel):
    product_id: ProductID
    title: str


class SavedProductRecord(BaseModel):
    saved_product_id: SavedProductID
    product_id: ProductID
    user_id: UserID


def read_saved_product(saved_product_id: SavedProductID) -> SavedProductRecord:
    pass


def read_product(product_id: ProductID) -> Product:
    pass


record: SavedProductRecord = ...  # pretend this came from somewhere
read_product(record.saved_product_id)

If you call mypy on this now, you'll get:

error: Argument 1 to "read_product" has incompatible type "SavedProductID"; expected "ProductID"

Also, PyCharm can detect the error:

With the pydantic mypy and/or pycharm plugins, you can also get ID type checking in BaseModel init signatures (very useful when converting one model to another).

---

This same idea is useful well beyond just ID types; any time you can imagine a domain-specific "type", but are representing the data as a bare str, int, etc., you can use NewType to get type checking for the data. An example of detecting this type of error might look like:

Bad:

class UserRegistrationRequest(BaseModel):
    username: str
    password: str

def create_user(password: str, username: str):
    ...

@app.post("/register")
def register(registration_request: UserRegistrationRequest) -> None:
    # whoops, wrong order
    create_user(registration_request.username, registration_request.password)  

Good:

RawPassword = NewType("RawPassword", str)
class UserRegistrationRequest(BaseModel):
    username: str
    password: RawPassword

def create_user(password: RawPassword, username: str):
    ...

@app.post("/register")
def register(registration_request: UserRegistrationRequest) -> None:
    # the error gets detected by mypy since registration_request.username is not a RawPassword
    create_user(registration_request.username, registration_request.password)

---

If you need to convert an instance of a type to a NewType-version, you just do:

RawPassword = NewType("RawPassword", str)
my_string = "helloworld"  # mypy thinks this is a string
my_password = RawPassword(my_string)  # mypy thinks this is a RawPassword

NewTypes are treated as subclasses of their "origin" type, so you could pass a RawPassword anywhere a str would be accepted. At runtime, calling RawPassword(my_string) actually does nothing -- at runtime RawPassword is essentially the same as lambda x: x (this ensures minimal performance impact). But mypy treats it differently.

[euri10]

ok I already told you @dmontagu but that's an amazing way of explaining clearly a feature a lot of people would not even notice it exists, just because you explains very well the problem it solves at first, it doesn't come out of the blue, it's pragmatic.

I just read again the mypy docs about NewType and I was debating if I found the git man pages more or less convoluted...

edit: that should go to pydantic docs !

[tiangolo]

Thanks for the help here everyone! 👏 🙇

Thanks for reporting back and closing the issue @dmajere 👍

[emilhdiaz]

One use case where it would be helpful to have support for this is when using class based router views from the fastapi_utils package: https://fastapi-utils.davidmontague.xyz/user-guide/class-based-views/

With a generic base class for all my routers:

ModelReadType = TypeVar("ModelReadType")
ModelCreateType = TypeVar("ModelCreateType")

class AppRouter(Generic[ModelReadType, ModelCreateType]):
    session: Session = Depends(GetSession)

    @router.post('/', response_model=ModelReadType)
    async def _create(self, user: ModelCreateType = Body(embed=False)):
        ....

    ... remaining CRUD operations

I could quickly inherit all of this CRUD functionality in my actual routes and even override route methods if I wanted to:

class UserRouter(AppRouter[UserReadModel, UserCreateModel]):
    ....

    @router.read('/', response_model=ModelReadType)
    async def _get(self, id: str):
        ....

Currently, this allows me to use the endpoints, but without any type inference or validation on those input models.

[github-actions[bot]]

Assuming the original need was handled, this will be automatically closed now. But feel free to add more comments or create new issues or PRs.

[aclementev]

Another use case for this is to build an "API factory", as a function that returns a FastAPI with some stardard endpoints pre-configured.

from fastapi import Body, FastAPI
from pydantic import BaseModel

T = TypeVar("T", bound=BaseModel)
def factory(schema: T) -> FastAPI:

    app = FastAPI()
    
    @app.post("/foo")
    def foo(input: Annotated[T, Body()]):
        return {"input": input}

    return app

You can then use this factory with a user provided schema (as a pydantic BaseModel).
(NOTE: I had to explicitly use the Body to make sure that FastAPI extracted the data from the request, but I guess that a separate issue).

[redb0]

I faced the same problem.

I want to dynamically create an API with a type set via TypeVar. But FastAPI puts the parameter in the query, not in the body

from typing import Generic, TypeVar
from fastapi import APIRouter, FastAPI
import uvicorn
from sqlmodel import SQLModel

T_ = TypeVar('T_', bound=SQLModel)


class CreateAPI(Generic[T_]):
    def __init__(self, model: type[T_]):
        self.model = model

    def create_api(self, router: APIRouter) -> None:
        def foo(obj: T_) -> None:
            print(obj)

        router.add_api_route(
            '/foo',
            foo,
            description='create',
            name='create',
            methods=['POST'],
        )


class Foo(SQLModel):
    a: int = 1
    b: str = 'qwe'

app = FastAPI()
router = APIRouter(prefix='/api')
CreateAPI(Foo).create_api(router)
app.include_router(router)

uvicorn.run(app)