How to define empty query parameters as missing?

[levchik]

First check

• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.

Description

How can I make empty query parameters to be interpreted as missing value (and therefore default value is used) instead of empty string? For example, if I were to use optional int query param, empty string would find its way as value being passed and validation error for invalid int would be raised.

Additional context

Sample code:

from fastapi import APIRouter

router = APIRouter()


@router.get("/test/", tags=["test"])
def test(q: int = None):
    return {"q": q}

GET http://URL/test/?q=

or

GET http://URL/test/?q

results in:

{
  "detail": [
    {
      "loc": [
        "query",
        "q"
      ],
      "msg": "value is not a valid integer",
      "type": "type_error.integer"
    }
  ]
}

[phy25]

Interesting, I can reproduce this with the latest fastapi, but not with fastapi~=0.49.0.

[levchik]

@phy25 I think it is somehow connected with this PR and therefore with versions of Starlette

[gasabr]

same question here, any updates on this?:)

[tiangolo]

Hmm, yeah, it probably has to do with Kludex/starlette#672.

I think this would be a request for Starlette. To parse ?q and ?q= as q = None instead of q = "".

Do you wanna request it there?

I'm seeing in Starlette that this is handled by Python's standard urlib.parse: https://docs.python.org/3/library/urllib.parse.html#urllib.parse.parse_qsl

---

And I'm actually not sure that reading that as None would be the correct behavior, as the URL is actually all a string.

It would make sense that something not provided would be None, and/or take its default value, but some query parameter that is passed in the URL, but without value, would kinda technically be an empty string.

I could think that not passing the parameter at all could make it have a default value, but passing the query parameter would set some value... The thing is that as URLs are strings, there's no simple notion of a None type.

I'm not really sure how to approach it here because I'm not totally convinced about any of the 2 approaches, but I think I'm inclined to think this "was kind of a bug" that is now "sort of fixed" 🤔

[levchik]

@tiangolo thanks for the reply!

Actually, I'm not sure which approach is better either, but I think that there should be a way to handle this in both use-cases. Since use-case "empty query parameter (?q or ?q=) has value of empty string" is the default one, I am curios about how to achieve "empty query parameter (?q or ?q=) is missing (e.g. None & default value is used)"?

Appreciate your help, thanks!

[Dectinc]

@tiangolo thanks for the reply!

Actually, I'm not sure which approach is better either, but I think that there should be a way to handle this in both use-cases. Since use-case "empty query parameter (?q or ?q=) has value of empty string" is the default one, I am curios about how to achieve "empty query parameter (?q or ?q=) is missing (e.g. None & default value is used)"?

Appreciate your help, thanks!

Same requirements here, and handled with a middleware at last.

We can just drop those fields with blank values, which are empty strings or strings contain only whitespaces, default values will be handled by Query models.

# Add blank query parameters processing middleware
@app.middleware('http')
async def drop_blank_query_params_middleware(request: Request, call_next):
    _scope = request.scope
    if request.method != 'GET':
        return await call_next(request)
    if not _scope or not _scope.get('query_string'):
        return await call_next(request)

    def _process_query_params(query_params):
        _query_params = QueryParams(query_params)

        from urllib.parse import urlencode
        return urlencode([
            # using `_query_params.items()` will mistakenly process list parameters
            (k, v) for k, v in _query_params._list if v and v.strip()  # noqa
        ])

    _scope['query_string'] = _process_query_params(_scope['query_string']).encode('latin-1')
    return await call_next(Request(_scope, request.receive, request._send))  # noqa

However, as @tiangolo commented, this approach may not be correct, maybe it's not recommended. :)

[github-actions]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.

[xoelop]

@tiangolo thanks for the reply!
Actually, I'm not sure which approach is better either, but I think that there should be a way to handle this in both use-cases. Since use-case "empty query parameter (?q or ?q=) has value of empty string" is the default one, I am curios about how to achieve "empty query parameter (?q or ?q=) is missing (e.g. None & default value is used)"?
Appreciate your help, thanks!

Same requirements here, and handled with a middleware at last.

We can just drop those fields with blank values, which are empty strings or strings contain only whitespaces, default values will be handled by Query models.

# Add blank query parameters processing middleware
@app.middleware('http')
async def drop_blank_query_params_middleware(request: Request, call_next):
    _scope = request.scope
    if request.method != 'GET':
        return await call_next(request)
    if not _scope or not _scope.get('query_string'):
        return await call_next(request)

    def _process_query_params(query_params):
        _query_params = QueryParams(query_params)

        from urllib.parse import urlencode
        return urlencode([
            # using `_query_params.items()` will mistakenly process list parameters
            (k, v) for k, v in _query_params._list if v and v.strip()  # noqa
        ])

    _scope['query_string'] = _process_query_params(_scope['query_string']).encode('latin-1')
    return await call_next(Request(_scope, request.receive, request._send))  # noqa

However, as @tiangolo commented, this approach may not be correct, maybe it's not recommended. :)

Note that QueryParams comes from starlette.datastructures. For it to work add from starlette.datastructures import QueryParams.

Works like a charm, btw. Thanks!

[mrchi]

@tiangolo thanks for the reply!
Actually, I'm not sure which approach is better either, but I think that there should be a way to handle this in both use-cases. Since use-case "empty query parameter (?q or ?q=) has value of empty string" is the default one, I am curios about how to achieve "empty query parameter (?q or ?q=) is missing (e.g. None & default value is used)"?
Appreciate your help, thanks!

Same requirements here, and handled with a middleware at last.
We can just drop those fields with blank values, which are empty strings or strings contain only whitespaces, default values will be handled by Query models.

# Add blank query parameters processing middleware
@app.middleware('http')
async def drop_blank_query_params_middleware(request: Request, call_next):
    _scope = request.scope
    if request.method != 'GET':
        return await call_next(request)
    if not _scope or not _scope.get('query_string'):
        return await call_next(request)

    def _process_query_params(query_params):
        _query_params = QueryParams(query_params)

        from urllib.parse import urlencode
        return urlencode([
            # using `_query_params.items()` will mistakenly process list parameters
            (k, v) for k, v in _query_params._list if v and v.strip()  # noqa
        ])

    _scope['query_string'] = _process_query_params(_scope['query_string']).encode('latin-1')
    return await call_next(Request(_scope, request.receive, request._send))  # noqa

However, as @tiangolo commented, this approach may not be correct, maybe it's not recommended. :)

Note that QueryParams comes from starlette.datastructures. For it to work add from starlette.datastructures import QueryParams.

Works like a charm, btw. Thanks!

Thanks for the answer. I met this issue when I use amis with FastAPI.

According to above code snippet, I make a litte optimization to simplify code.

from typing import Callable
from urllib.parse import parse_qsl, urlencode

from fastapi import Request

@app.middleware("http")
async def filter_blank_query_params(request: Request, call_next: Callable):
    scope = request.scope
    if scope and scope.get("query_string"):
        filtered_query_params = parse_qsl(
            qs=scope["query_string"].decode("latin-1"),
            keep_blank_values=False,
        )
        scope["query_string"] = urlencode(filtered_query_params).encode("latin-1")
    return await call_next(request)