Hang when exiting a "docker exec" shell #24974
Comments
|
Potentially relevant log line: Docker-containerd version: Runc version: |
|
I have exact same problem I have only few different items to above configuration. Actually had the same problem with 1.12_rc3. My configuration: |
Is the container still running, or did it exit? You can only |
|
@thaJeztah Execed into a running container, the execed process exited & hangs, the container is still running. |
|
Reverting to docker-containerd 1b3a81545ca79456086dc2aa424357be98b962ee and docker-engine v1.12.0-rc3 fixes these issues. |
|
ping @mlaventure PTAL ^^ |
thaJeztah
added
area/runtime
kind/bug
Temporary revert until these issues are fixed: - moby/moby#24974 - moby/moby#24985 Signed-off-by: Christian Stewart <christian@paral.in>
|
@paralin I can't reproduce this issue. How did you start the initial container? Also, your docker version doesn't provide a git commit in its output. Here the version I used to try and reproduce the issue: |
|
@mlaventure Versions that work: docker info: The git commit for docker-engine is v.1.12.0-rc3 - the Git tag. |
Temporary revert until these issues are fixed: - moby/moby#24974 - moby/moby#24985 Signed-off-by: Christian Stewart <christian@paral.in>
|
@paralin it may be, but if I can't reproduce it, I can't fix it :) Could you share which base image you used, with what options you started it, and the daemon logs of the failure? atm, it works just fine on RC4 and Master for me. |
I can reproduce it with docker logs shows and |
Temporary revert until these issues are fixed: - moby/moby#24974 - moby/moby#24985 Signed-off-by: Christian Stewart <christian@paral.in>
|
@djarosz Unfortunately, I can't reproduce this either. the output for Also, can you confirm that the 3 git hash for Also, just in case, could you try shutting down your daemon, renaming From your log, the exit event is correctly received by the daemon which should have cause the connection to be released and the client to exit properly. |
Temporary revert until these issues are fixed: - moby/moby#24974 - moby/moby#24985 Signed-off-by: Christian Stewart <christian@paral.in>
Temporary revert until these issues are fixed: - moby/moby#24974 - moby/moby#24985 Signed-off-by: Christian Stewart <christian@paral.in>
|
@mlarcher Using: With a clean system. Still have the same error. |
|
Errors from that test: |
|
@paralin please post the exact steps you used to reproduce the error. Or are they the same as the one I've shown in #24974 (comment) ? Also, the full daemon logs would be useful. |
|
Unfortunately running docker after removing /var/lib/docker did not help. I cannot provide exect commit ids for runc and docker but it seams than gentoo packages download sources from following locations:
|
|
Exact steps to reproduce the error.
Obviously the odroid part is a tiny bit sarcastic, but that is exactly what you would have to do to reproduce this. As you can see here this commit reverting the patch IDs for docker-containerd and docker-engine fixes the problem. I'm glad @djarosz can also reproduce because he is operating in a bit of a more standard environment. |
|
I think a scorched-earth solution to this might be doing a git bisect on docker-containerd |
|
To reproduce I just I have just run the test witch docker, containerd and runc compiled from exactly the same git commits as pointed by @mlaventure in this comment So I've made test and issued this commands in following order. Numbers in brackets represent terminal windows And this is debug output of docker daemn during this time |
|
Forgot to add that was after Also I have tried commits which fix the issue for @paralin. But in my case they didn't work either. So maybe we are facing different issue after all. |
|
I got this error as well while using out of sync containerd/docker builds. Not just on exec, |
Temporary revert until these issues are fixed: - moby/moby#24974 - moby/moby#24985 Signed-off-by: Christian Stewart <christian@paral.in>
|
@djarosz Was the runc version important? Or just containerd / docker? |
|
|
|
No runc did not matter. For some of my tests I just switched between different commits of docker and containerd but stayed on the same commit of runc. Now I'm on 1.12.0_rc5 and works ok for me. |
|
I'll close this issue, because it looks resolved (and it's not a bug), but feel free to continue the discussion |
|
I have the steps to reproduce the issue; docker version: Server: step 1: Chose a image, and run cmd: docker run -d -it image:tag /bin/bash Can Open this issue???? @mlaventure |
|
@huangcuiyang This is fixed on newer versions of Docker... maybe even in 1.12.6... but likely you'll need to upgrade to 17.03 |
|
Hi, I got the same problem in version 17.03.1-ce I also tried restart/stop, nothing worked other information I gathered: |
|
@SerenaFeng this is a different issue altogether (something more like #32134) Hopefully 17.06 should resolve most if not all of those issues. |
|
Still seeing the same hang issue with exiting docker exec shell. My Docker version: 17.06.0-ce |
|
@Lax77 it's expected if your exec processes spawned children that are holding the IOs open |
|
in mycase on the problem system all I did was I didn't trigger in any special calls within the container when I exec into the container. All I did was exec into the container and get out. |
|
@Lax77 in that case open a new issue with the relevant data so we can debug this. If you can reproduce it, having the daemon in debug mode would be extremely helpful to try and have an idea of what's happening. |
|
Was a new issue created for this? I'm experiencing this as well. |
|
No, didn't get a chance to collect the debug logs of this issue. So didn't open a new bug for it. But this problem still keeps happening with 17.06, haven't tried upgrading to 17.09 though yet. |
|
Ah, thanks for the update. I'm also seeing it with 17.06. If I have some spare cycles I'll try to get some debug logs. Probably not till next week though |
|
I'm also seeing hangs exiting after running I think child processes are probably part of the problem, as I can exec to the same container a second time, and kill some previous processes to unhang my original shell. |
|
Maybe it worth trying -d flag for exec as workaround |
|
Hrmm, I'll give that a try.
…On Nov 8, 2017 18:43, "AndriiNikitin" ***@***.***> wrote:
does exec without -i (and -t) hang as well for you guys?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#24974 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AE2rYgAlVTiheyJN22zxnfRErZ-b5VFIks5s0kqlgaJpZM4JTeTX>
.
|
|
I am still seeing hangs using This doesn't happen as soon as the container is spawned, but sometime later after running a few |
|
i have same issue my docker version is |
|
I seem to understand what is causing the hang. When command executed by docker exec create child process, and this process still work after command exit. On next run command hung on exit. |
Hey @OleksiiStepanov , had the same issue. Fixed a few minutes ago :) with replacing mather OS. Looks like the cause of the issue is old kernel version + 18th docker. I have used Amazon Linux v1 with a docker 18+ version. The last of my hope was migration as I realized that issue in exit signals between mother OS and docker engine. As soon as I have migrated to Amazon Linux v2 with the 19th docker the issue is gone. I hope that it helps you as well. |
|
@eugeneievlev I no sure the problem with host OS i have same problem with different host OS. i made test with:
|
Docker info:
Version:
Any ideas?
The text was updated successfully, but these errors were encountered: