New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve client timeout and wait queue handling #1570

Closed
wants to merge 0 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@eklavya
Contributor

eklavya commented Nov 28, 2017

No description provided.

@rossabaker

Thanks. This is a good start on a tough problem.

Am I correct that the timing out of requests is only going to happen when some request in the pool gets released and we look for the next one to unblock? If the pool is consumed by several slow connections, I don't think the waiting connections would time out in a timely fashion. Am I misreading this?

@eklavya

This comment has been minimized.

Show comment
Hide comment
@eklavya

eklavya Nov 30, 2017

Contributor

You are right, we do need to wait in that case, although we take care of the next expired ones immediately in "findFirstAllowedWaiter". But I can't think of an easy solution to this, ideas?

Contributor

eklavya commented Nov 30, 2017

You are right, we do need to wait in that case, although we take care of the next expired ones immediately in "findFirstAllowedWaiter". But I can't think of an easy solution to this, ideas?

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker Dec 1, 2017

Member

We could have a scheduler proactively clean out things caught in the wait queue. I don't know what semantics we should have for a request that's in flight. Imagine a non-idempotent POST that has been submitted but not yet responded.

Member

rossabaker commented Dec 1, 2017

We could have a scheduler proactively clean out things caught in the wait queue. I don't know what semantics we should have for a request that's in flight. Imagine a non-idempotent POST that has been submitted but not yet responded.

@eklavya

This comment has been minimized.

Show comment
Hide comment
@eklavya

eklavya Dec 1, 2017

Contributor

I thought about a scheduler but I thought it maybe overkill. Do we really want this?

I don’t understand the second point. If the request is in flight that means it’s not in wait queue, right?

Contributor

eklavya commented Dec 1, 2017

I thought about a scheduler but I thought it maybe overkill. Do we really want this?

I don’t understand the second point. If the request is in flight that means it’s not in wait queue, right?

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker Dec 3, 2017

Member

From the client perspective, do we care how long we wait for a connection from the pool, or do we care how long it takes from submission to getting a timeout response? Which wait are we trying to fix here?

If we're trying to get timeout responses in a timely fashion, then I think we want a scheduler, and need to account for responses whose timeout is partially but incompletely consumed while in the wait queue. (My second point). I think this is a positive, but it gets ugly when a non-idempotent request is in flight.

Member

rossabaker commented Dec 3, 2017

From the client perspective, do we care how long we wait for a connection from the pool, or do we care how long it takes from submission to getting a timeout response? Which wait are we trying to fix here?

If we're trying to get timeout responses in a timely fashion, then I think we want a scheduler, and need to account for responses whose timeout is partially but incompletely consumed while in the wait queue. (My second point). I think this is a positive, but it gets ugly when a non-idempotent request is in flight.

@rossabaker

It would also be good to document how the timeout gets reset when popped off the waiting queue. And then I think we can merge this as "better than before."

Show outdated Hide outdated client/src/test/scala/org/http4s/client/testroutes/GetRoutes.scala Outdated
@rossabaker

This LGTM as an interim step forward. As discussed on Gitter, going to try to integrate this fully without the resetting of the clock for 0.18.0.

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker Dec 22, 2017

Member

I think this is correct. We should take the nasty disclaimers out of the scaladoc.

I'd love to get an automated test around this, but am not yet sure the best way to do so.

Member

rossabaker commented Dec 22, 2017

I think this is correct. We should take the nasty disclaimers out of the scaladoc.

I'd love to get an automated test around this, but am not yet sure the best way to do so.

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker Dec 22, 2017

Member

I removed the limitations from the scaladoc.

Note that our response header timeout is from when the request is submitted via the client, and not when the request goes over the wire. (This is different from the cloudflare document we've all been using as a discussion starter.)

Member

rossabaker commented Dec 22, 2017

I removed the limitations from the scaladoc.

Note that our response header timeout is from when the request is submitted via the client, and not when the request goes over the wire. (This is different from the cloudflare document we've all been using as a discussion starter.)

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker Dec 22, 2017

Member

I think the formatting error in Travis is something I fixed in master in 8ed6be9

Member

rossabaker commented Dec 22, 2017

I think the formatting error in Travis is something I fixed in master in 8ed6be9

@rossabaker rossabaker changed the title from For #1532 Client timeout from time of request to Improve client timeout and wait queue handling Dec 23, 2017

@eklavya

This comment has been minimized.

Show comment
Hide comment
@eklavya

eklavya Dec 23, 2017

Contributor

Shouldn’t we also drain wait queue (fail all) while shutting down pool in PoolManager.scala? On mobile don’t really know if you have done it already.

Contributor

eklavya commented Dec 23, 2017

Shouldn’t we also drain wait queue (fail all) while shutting down pool in PoolManager.scala? On mobile don’t really know if you have done it already.

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker Dec 23, 2017

Member

I was aiming for a graceful shutdown: accept no new connections, but fulfill everything that was already queued. I think that's achieved in 54cfe4a.

Member

rossabaker commented Dec 23, 2017

I was aiming for a graceful shutdown: accept no new connections, but fulfill everything that was already queued. I think that's achieved in 54cfe4a.

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker Dec 23, 2017

Member

Sorry, I accidentally pushed my master to your master on merge, and then I couldn't fix it. This is all on #1613, minus the discussion. :(

Member

rossabaker commented Dec 23, 2017

Sorry, I accidentally pushed my master to your master on merge, and then I couldn't fix it. This is all on #1613, minus the discussion. :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment