Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Improve client timeout and wait queue handling #1570
Thanks. This is a good start on a tough problem.
Am I correct that the timing out of requests is only going to happen when some request in the pool gets released and we look for the next one to unblock? If the pool is consumed by several slow connections, I don't think the waiting connections would time out in a timely fashion. Am I misreading this?
From the client perspective, do we care how long we wait for a connection from the pool, or do we care how long it takes from submission to getting a timeout response? Which wait are we trying to fix here?
If we're trying to get timeout responses in a timely fashion, then I think we want a scheduler, and need to account for responses whose timeout is partially but incompletely consumed while in the wait queue. (My second point). I think this is a positive, but it gets ugly when a non-idempotent request is in flight.
I removed the limitations from the scaladoc.
Note that our response header timeout is from when the request is submitted via the client, and not when the request goes over the wire. (This is different from the cloudflare document we've all been using as a discussion starter.)