Custom Request Parsers

[Kludex]

There were a lot of discussions in the past around:

• Custom request class
• Custom parser depending on the content type
• A way to change the json module used for Request and WebSocket classes

There's an extensive list of related issues around it:

• Possible to parse uploaded files while they are being uploaded? #849
• Proposal: Custom Request Classes using type annotations #930
• Add option to overwrite UploadFile impl #697
• MultipartParser: option to write files to temporary directory on file upload #388
• Using Custom JSON Encoders and Decoders? #715
• Customizable request class #875

This discussion is for us to think about an API that would satisfy all those issues, and that it doesn't bring a huge complexity to the source code.

Suggestions

request_class parameter

Having a request_class parameter on the Starlette, Router and Route classes.

from starlette.requests import Request
from starlette.applications import Starlette

class CustomRequestClass(Request):
    ...

app = Starlette(routes=..., request_class=CustomRequestClass)

request_parsers parameter

This idea is motivated from some comments that @tomchristie made around, that I guess are inspired on DRF: create an API for Starlette to use custom parsers that are used depending on the content type.

Since Request class is an ASGI application by itself, adding the parsers parameter over there would break this interface, so I guess we'd have some global configuration like we have for convertors:

https://github.com/encode/starlette/blob/33f46a13625bcca4b7520e33be299a23b2e2b26c/starlette/convertors.py#L86-L87

from starlette.requests import register_request_parser

class JSONParser:
    def parse(self, content: bytes) -> dict:
        ...

register_request_parser(content_type="application/json", parser=JSONParser)

I don't like this. Is there an alternative API we can create here?

Add json_loads parameter and parsers parameters to methods

We have the following methods:

• Request.json()
• Request.form()
• WebSocket.receive_json()

The idea would be to add json_loads parameter to Request.json() and WebSocket.receive_json(), and parsers parameter to Request.form().

To discuss

I actually touched superficially the subject here, I'd like for us to discuss it. cc @encode/maintainers

[alex-oleshkevich]

This is a very good initiative.
After a long time of looking for a solution for a custom request class, I stopped at inspecting the function signature in Route class and injecting a user-define request object:

class MyRequest(Request): ...

def index(request: MyRequest): ...

routes = [
   Route('/', index)
]

This is not a breaking solution and does not affect the performance because you extract class type once in the Route.
A note: we have to copy _body each time because the middleware can read and cache the body.

[Kludex]

I wouldn't like for Starlette to have this typing inference logic. 🤔

[alex-oleshkevich]

yeah, but this is true low-hanging fruit ;)

[alex-oleshkevich]

request_parsers can be a property of the application itself. Then in Request class, we can get them from self.app.request_parsers. No need in introducing a new API. It is the same way we check for debug property.

app = Starlette(request_parsers=[json_parser, form_parser, ...])

# Request
async def parse(self) -> Any:
    for parser in self.app.request_parsers:
        if parser.supports(self.headers['content-type']):
            return await parser.parse(...)
    raise 

async def json(self) -> Any:
   parser = get_json_parser(self.app.request_parsers)
   return await parser.parse(...)

See https://www.django-rest-framework.org/api-guide/content-negotiation/

[alex-oleshkevich]

1. The solution in the comment does not require any mapping. In form and json we explicitly pass a correct parser. User-facing API - async def parse(self, parser) where users specify any callable they need.
2. Not sure that WS needs it. All fine there IMO.

[Kludex]

I'm wondering since the solutions @tomchristie suggested previously were based on the content-type.

Not sure that WS needs it. All fine there IMO.

I mean, there's a json() method there. Would we just be ignoring it?

[adriangb]

What is the advantage of having a Request.parse(self, parser) method as opposed to just having a Parser that accepts a Request instance? Something like:

from foo_parser_library import FooParser

async def endpoint(request: Request) -> Response:
    foo = await FooParser(request).parse()
    raise NotImplementedError

It's 1 extra line of code and avoids any sort of introspection, additional APIs in Starlette, works with type checkers and does not introduce a conceptual dependance of Request on Parser. I guess one could argue that Request already has the concept of a parser in the form of json() and form(), maybe it's just because those are the most common types of parsers or maybe we should have never included that, but I'm not sure that Request should depend on the concept of a Parser.

[alex-oleshkevich]

@adriangb it is no longer one-liner when your endpoint responds with different formats depending on request content type.
Example: A CRM that serves HTML tables for HTML and responds with JSON or CSV when called from CLI or API.

[adriangb]

I think that'd be better suited for some sort of router, which is a primitive we already have.