Performance issue

[davidbrochart]

I'm posting here although it's really a general Python/HTTP/ASGI/... issue. But since @Kludex maintains httpx2, uvicorn and starlette, I know I can have relevant feedback here :)
I've created https://github.com/davidbrochart/web-perf to show a scenario where we make lots of concurrent requests, and this seems to have catastrophic performances compared to sequential requests, as seen below:

It could be a mistake on my side, but do you see what could explain these numbers?

[davidbrochart]

To clarify, what I want to show is that a burst of requests on a server leads to a pretty bad response time for clients. I've updated the notebook to also show the total time it takes to handle all requests:

So it seems to me that there are two problems for the parallel case:

• some clients will see their response time as almost the total time it takes to process all requests.
• the total time to process all requests is significantly worse.

[Kludex]

@davidbrochart Can you check against #1075? It seems there was a long living PR in httpx itself about this.

[davidbrochart]

But the parallel case is still significantly worse than the sequential case.

[Kludex]

What about now?

[davidbrochart]

Much better! The parallel case is now faster than the sequential case overall (0.8s vs 1.1s), but the per-request response time still increases linearly. Wouldn't we want the response time to be constant for all requests, like for the sequental case? That could be a server issue though.

[tapetersen]

I think you will always see this as the test is set up as long as the time to receive a request is longer than it is to send it (reasonable that parsing is slower).

When you queue all test-tasks and then let them start the first thing they do is just record their time and send. When they get back you still have to process the responses and some tasks will simply have to wait since the synchronous handling of the response is fast but a bit slower than the sending and would show up as this difference.

[davidbrochart]

Yes that makes sense, I guess this is the best we can do then.

[Kludex]

I'll wait for @11kkw review on #1075, and I'll cut a release.