Increase size of buffer for signal forwarding with --sig-proxy #6508
Conversation
The chan struct used to forward signals to containers was one element only, which caused some signals to be dropped when many were being received. Increasing the size of the chan buffer makes this much less likely to happen. Docker-DCO-1.1-Signed-off-by: Matt Heon <mheon@redhat.com> (github: mheon)
|
With this fix does the signal test work? |
|
Depends on how many signals you wish to send. It's definitely still possible to drop signals - at least on my machine, Docker takes longer to forward the signal than it does for another process to send a new signal, so it's still possible to overflow the buffer with a continuous stream of signals. However, I don't know of a use case where this is necessary. The buffer is there in case a large number of signals arrive in a brief period of time. |
|
How are the signals being lost? they are just buffered and should be delivered as the chan is drained |
|
You can see the relevant code here: https://github.com/dotcloud/docker/blob/master/api/client/commands.go#L538 Signals arrive in the buffer, and are then pulled off by the goroutine which does a bit of processing, then forwards the signal. The issue arises when two or more signals arrive during that goroutine - as the chan was initialized with size 1, anything after the first one will be lost in this case. I can consistently make this happen with a simple Bash script calling kill in a loop - a few signals get through, but the vast majority are lost. Increasing the buffer size fixes this issue |
|
@crosbymichael: @mheon seems right:
We could either set the channel buffer size to a little-higher value and/or have (again) some option... (server-side I guess but one could argue it could also be on |
|
LGTM |
|
LGTM |
|
ping @crosbymichael |
|
LGTM |
Increase size of buffer for signal forwarding with --sig-proxy
Sending signals rapidly to a container (for example, through a script) frequently results in some signals being dropped. Root cause of this seems to be the chan for holding signals to be forwarded has a buffer size of 1, and if 2 or more new signals arrived while one was being forwarded, signals were lost. Increasing the buffer should fix this.
The value 128 here is somewhat arbitrary, but seems reasonable for normal use cases. Open to adjusting this up or down.