Question about connection pool

[jiacai2050]

Hi developers,

I meet a strange question when using reqwest in one of my codebase, I can reproduce it with this repo

• https://github.com/jiacai2050/tokio-debug

The demo will start two runtime:

• IO runtime
• CPU runtime, which task will sleep 200ms via blocking API

In theory, requests made in IO runtime should finish in 50ms, but there will be one or two requests take 1s to finish, it seems requests made in CPU runtime affect those in IO runtime.

If I disable connection pool via pool_max_idle_per_host(0), then all requests in IO runtime take 50ms to finish, so I wonder if there are some issue with current connection pool implementation.

[fabracht]

I read your article here, and came to look at your question.
I think you have an interesting problem there. I'm not sure what the solution to it, but I'll try to discuss the points I found interesting. First, here's what I think happens when you disable connection reuse by setting pool_max_idle_per_host to 0 in the reqwest client configuration.
In your setup with separate IO and CPU runtimes, resources like connections might be causing performance issues between the two runtimes. Disabling connection reuse ensures that slow or blocked tasks in one runtime don't delay the release or reuse of connections that could impact the other runtime. Although, establishing a new connection for each request incurs additional latency due to the handshake and TCP ramp-up times. You could try fine-tuning the pool size and idle timeouts based on performance measurements.
But even more interesting is the "why one runtime impacts the other" question. I think the problem is that there's no guarantee that the two thread pools are using different physical cores. Which means that the two runtimes might be competing for the same resources.
You could try and pin the two runtimes to different cores, but that comes with its own difficulties. Tokio doesn't provide (as far as I know) a way to pin a runtime to a specific core.
What do you think?