New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Client middleware API #295
Comments
Trying to answer a few other questions I saw in the various discussions we've had… FAQWhat about the It seems the proposed middleware APIs encompass the current auth interface, which can be trivially implemented as a middleware that modifies the Shouldn't redirect stay part of the client? Maybe! We already have an (More precisely, this flag controls whether we automatically handle redirections. We still do some processing to store the Besides, if we think of middleware as a stack, I think we want redirection to always be handled first — e.g. we don't want auth to fail because it actually needed to call However, the various PRs have shown that it is possible to implement redirection handling as a middleware. So to strike a balance, we can make it so that the redirect middleware will always be added first by the client, and be wrapped by any extra (and potentially user-defined) middleware such as authentication. So we'll need to setup a middleware stack per request? It seems so: |
Pinging in here — anyone's got thoughts on this? Do we need the internal middleware at all? Is any of the proposed APIs potentially a good fit for a user-facing extensibility mechanism? |
Thanks for putting this together! :) You and @yeraydiazdiaz have been killing it with taking ownership of the middleware implementation. The callable middleware has my vote for simplicity and how close it resembles WSGI / ASGI. The calling itself recursively with a new request as a method of making a different request is smart too and solves the cache validation, digest auth, and redirect requirement. There's still the question of "what's the right way to access the As far as ordering I thought briefly about how we could take a page out of pyramid's book and allow saying "insert this middleware before X" on the register stage and then allow class-based middlewares to specify a default stage to be inserted at so that users don't have to think about it. I'm thinking only a few like: import httpx
import enum
class MiddlewareStage(enum.IntEnum):
AUTH = 1
CACHE = 2
REDIRECT = 3
client = httpx.Client()
client.add_middleware(MyCustomMiddleware(), before=MiddlewareStage.AUTH) As far as defining custom middlewares providing the type-hint for |
@sethmlarson Thanks for your input!
At first I thought, "what's the problem with storing anything we need to keep track of directly in middleware?". But then I remembered that middleware are re-created on each request, so we can't really persist anything there. But then, why should it be the middleware's responsibility to manage that kind of storage? I suppose the client can intercept responses that came back with cookies, and do its own thing to make sure those cookies are sent in subsequent requests to that host. Personally, I don't feel very comfortable discussing this because I don't have a clear vision yet of how one of these use cases could work. I think tackling caching would give us some insight on how to handle a concept of "client session" and its associated storage. Regarding middleware ordering — the enum idea is interesting and definitely worth digging into. Do you have any link to the Pyramid feature that gave you this idea? (I'm not very familiar with Pyramid.)
#268 already provides a Anyway, are we OK with proceeding with and merging #268 then? I'll ask for your review there, please let me know there if the |
I'm good with #268 proceeding. :)
I see that the redirect and auth middlewares are being recreated per request but would custom middlewares be restricted by this? |
Currently, yes. I guess it depends on whether we make .add_middleware (or whatever it’s going to be) accept a middleware class (which can be instanciated on each request) or a middleware instance. For some applications like redirection, only per-request middleware make sense. Maybe we’ll end up introducing a notion of scope (I’m thinking of pytest fixtures here), to be defined on the middleware class? |
Great job on this @florimondmanca! I think #268 is a great first implementation of this, chances are we'll start noticing pain points as we start implementing more middlewares but at this point in the project it's something we need to go through. I'll repurpose the Digest auth and look forward to more features using middlewares 🎉 |
Super @yeraydiazdiaz, I’m excited about Digest Auth! |
There's been quite a few PRs trying to take a stab at making the
Client
/AsyncClient
extensible, mostly with a middleware approach whose idea seems to have originated in #136 (comment) to help solve Digest Auth.As mentioned in #268 (comment) it might be useful to define and discuss in which direction we'd like to go.
Problem statement
A few requirements we may want to satisfy, gathered from looking at the various discussions:
None of the approaches attempt to solve 4) nor 5) — and that can definitely be dealt with in its own issue once this one is resolved. As a result, the scope of this issue is designing an internal API to solve 1), 2) and 3).
Existing implementation
A reminder of the features that HTTPX currently supports that may benefit from being refactored as as middleware:
Authorization
header based on ausername
andpassword
.req -> req
callable. (I personally think this might be too lose — we could just require the callable to return anAuthorization
header — but let's tighten the scope of this issue and keep things as they are.)These are currently implemented as part of the
BaseClient
itself. Them cluttering theBaseClient
is, as far as I understand, the main reason for refactoring them into loosely-coupled middleware.Proposed solutions
Here's a summary of the various APIs that were proposed up to now and their respective "no-op middleware" for the sake of example:
#176 : process_request / process_response
#267 : building on top of the dispatcher API
#268: callable middleware
I think the various PRs proved that all APIs tick 1), 2), and 3) as defined above, with various degrees of simplicity and boilerplate.
The callable middleware API seems to be the simplest one while offering enough flexibility. Making extra requests based on the initial response can be achieved by recursively calling the middleware, as in the RedirectMiddleware. As another PoC, a rough Digest-Auth implementation based on this snippet:
cc @tomchristie @yeraydiazdiaz @sethmlarson
The text was updated successfully, but these errors were encountered: