-
Notifications
You must be signed in to change notification settings - Fork 178
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Notify client close #310
Notify client close #310
Conversation
@bertjwregeer Is there a chance that this PR will be accepted if I fix the tests and add new ones for the new feature? Or do you think our request is too exotic and the added complexity should be avoided? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not opposed to adding the feature, but not in this current form.
- Adding an extra lock for something most applications won't use adds extra overhead that is not necessary
- I'd want this to be a flag that is set in the configuration, so this needs an adjustment flag, so that app owners need to opt-in to this behaviour, I do not like the idea of this being a default
- I would be okay with a new key being added to the environment that is a callable that can check if a channel is now closed because of client disconnection. This is the suggestion that @mmerickel made.
The task already has access to the channel, and since WSGITask
is creating the environ
, adding a function that is:
def check_alive(self):
return self.channel.client_disconnected
would be simple.
And in handle_read()
we set client_disconnected
if the client has disconnected (i.e. the read returns 0). Then in Task
we add a check in finish()
to see if self.channel.client_disconnected
is true, to raise ClientDisconnected
so that the logic in HTTPChannel.service()
does the right thing to tear everything down.
Regarding number 2, I don't mind the extra knobs being available in the WSGI environment by default, although adjustments is available inside of Task
as well, so it could be added to the WSGI environment conditionally.
@bertjwregeer Thanks for the review. Regarding 3: I was thinking that A) would be easiest to deal with on the application side, especially if we are in a situation where we spawned another process or are currently waiting for an SQL database query to be performed. When triggered, we could then terminate the spawned process or the database worker process. However, this has the mentioned disadvantage that the handler interrupts the main thread of waitress. And maybe it also adds unnecessary complexity (maybe the additional lock was necessary due to this approach and is not required with C, but I am not sure yet). I will try to think about this a bit more. Regarding 1: Regarding 2: |
Addendum: Regarding the difference between strategies B and C, I guess with C we do not need to make sure to catch the environment of the currently running task as well as any possibly queued tasks. However, I guess we anyhow do not want to start any more tasks if the client disconnected, do we? In any case, C seems to be superior to B. (C was also the strategy that @mmerickel suggested). |
a9fa318
to
2982dd2
Compare
@bertjwregeer I tried to implement the requested changes. There is still a lot to be done regarding tests and documentation, but is the general idea now something that might be accepted? Now that I looked over it again, the additional lock I had in the original attempt was due to race conditions with |
@bertjwregeer I changed the implementation so However, this makes the difference to One change I am not sure about is the handling of the output buffers. Due to the change, they are flushed after each request separately. Before, if the client sent multiple short requests in a pipeline such that they would be picked up with a single I did some obvious adjustments to the tests. I also dropped There are two more failing tests that I will check next. And, of course, I need to add tests for the new functionality. |
@viktordick thanks for updating it, I haven't taken a look yet but I saw that @mmerickel provided some feedback. I've been busy moving, hopefully once things things calm down a little bit I'll get back to this and help figure out a best path forward. |
e4b7083
to
1f0022f
Compare
@bertjwregeer, @mmerickel So I tried to read through the code in order to get a feeling of when each variable is used and came up with the following:
|
Thanks @viktordick for going through this, I think I agree with most of your analysis. A channel is never reused so there is no concern about the buffer cleanup or From what I'm seeing, I noticed that there's a potential race condition in |
I could imagine some sort of graceful shutdown which might use
|
So there's a bunch of reasons to close that are triggered via client, wsgi app, or process:
This is sort of supported right now via the |
Also yes, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like where this is going. More graceful shutdown would be fantastic, and its something that I would like to add, so keeping that in mind while continuing down this path would be great.
3859888
to
f3176a8
Compare
I secured the writing into the I'd say this PR is ready for a re-review. One check seems to be stuck - I had this problem with some other PR on github once and it seemed to be necessary to close and re-open the PR to get the tests to run again. Should I do this here also? |
Maintainers have the ability to hit the re-run all jobs button. So no need to close/open this PR. |
Yes, that is what the code does now. If there are no other request currently being executed or queued to be executed, the 100-continue is sent as soon as the request that expects it finished submitting the header (as before). Only if it is a request that is pipelined after other requests which are still being executed, the 100-continue is delayed until all tasks before it have finished, which is necessary to keep the order of responses correct. So the 100-continue is always written as soon as it is possible to write it without messing up the order of the responses. |
b61399b
to
eedfedd
Compare
@viktordick this is awesome, in my mind the only thing missing is a changelog entry! |
This inserts a callable `waitress.client_disconnected` into the environment that allows the task to check if the client disconnected while waiting for the response at strategic points in the execution, allowing to cancel the operation. It requires setting the new adjustment `channel_request_lookahead` to a value larger than 0, which continues to read requests from a channel even if a request is already being processed on that channel, up to the given count, since a client disconnect is detected by reading from a readable socket and receiving an empty result.
94411f0
to
a514a47
Compare
@mmerickel Thanks! I wrote a changelog entry and squashed the commits. @bertjwregeer Are you also happy with the result? There is still a change request from you recorded on the PR, but I think everything mentioned there has been addressed in the mean time. |
Thanks for being patient and working through this. I want to read this critically one last time and make sure that I've convinced myself that there are no locking issues/race conditions. Thanks for all your hard work! |
Thanks so much @viktordick for your patience and working through this. This looks great, and I have adequately convinced myself that this won't break! |
Thanks a lot for the help and patience! Is a release already planned sometime soon? |
@viktordick https://pypi.org/project/waitress/2.0.0b0/ I need testers to validate this doesn't break anything, but there is now a released version of waitress you can try in your environment. |
Thanks.
In our |
I couldn't use it in waitress serve in version 2.0. I get a stack trace |
That's not how this feature works. You can't inject your own callable into the environ. |
Then how is it used? I couldn't find any example on it. |
When using waitress the wsgi environ is providing an extra key with a callable that can be invoked to see if the client is still connected. It is not a callback. You have to poll it in your request handling code. |
Implementation of the idea discussed in #308. The tests don't pass yet and I should also implement additional tests for this, but hopefully this helps with the discussion.