Skip to content

Commit

Permalink
Make container exit gracefully and with correct exit code
Browse files Browse the repository at this point in the history
While fiddling around with multiple child processes, and giving them
some nice "exit 1" error codes, I noticed that the container would
always exit with exit code "0". It was not until I placed an
"exit 2" above the 'trap "kill 0" EXIT' row that I started getting
the expected exit codes again.

I dived a bit deeper into this, and realized that this thing with
properly handling kill signals is pretty tedious. Nevertheless,
after reading a lot of articles (linked at the bottom) I finally
managed to create something which seems to work as intended.

The first trap will catch the SIGINT (Ctrl+C) and the SIGTERM
(docker stop ...) singnals, and make so that both of these trigger
a normal "exit" command in the script. The second trap is special
for bash (the 'EXIT' at the end of the trap command), since this
allows you to trigger a "cleanup script" before  exiting the main
program. I made so that it will run the function "clean_exit" before
terminating with the correct exit code.

The "clean_exit" script just sends a SIGTERM signal to the PIDs of
our two child processes (if they exists and have started), which
gives them a chance to exit gacefully.

By implementing this we can now exit the container in a controlled
and graceful manner, while also having it report the correct exit
code depending on what happened.

[1]: http://veithen.io/2014/11/16/sigterm-propagation.html
[2]: https://www.linuxjournal.com/content/bash-trap-command
[3]: https://blog.codeship.com/trapping-signals-in-docker-containers/
[4]: https://www.ctl.io/developers/blog/post/gracefully-stopping-docker-containers/
[5]: https://unix.stackexchange.com/a/444676
[6]: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
[7]: https://stackoverflow.com/a/35410993
[8]: https://unix.stackexchange.com/q/317492
  • Loading branch information
JonasAlfredsson committed Mar 7, 2020
1 parent b7074f6 commit 43dde6e
Showing 1 changed file with 16 additions and 3 deletions.
19 changes: 16 additions & 3 deletions src/scripts/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
#!/bin/bash

# When we get killed, kill all our children (o.O)
trap "exit" INT TERM
trap "kill 0" EXIT
# Helper function to gracefully shut down our child processes when we exit.
clean_exit() {
for PID in $NGINX_PID $CERTBOT_LOOP_PID; do
if kill -0 $PID 2>/dev/null; then
kill -SIGTERM "$PID"
wait "$PID"
fi
done
}

# Make bash listen to the SIGTERM and SIGINT kill signals, and make them trigger
# a normal "exit" command in this script. Then we tell bash to execute the
# "clean_exit" function, seen above, in the case an "exit" command is triggered.
# This is done to give the child processes a chance to exit gracefully.
trap "exit" TERM INT
trap "clean_exit" EXIT

# Source "util.sh" so we can have our nice tools.
. $(cd $(dirname $0); pwd)/util.sh
Expand Down

1 comment on commit 43dde6e

@JonasAlfredsson
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An extra note to this is that "kill 0" is not a totally wrong thing to call on as the "cleanup script" before exiting. The 0 here is a special case which signals all processes in the current process group, so that would mean us and all our children [1], [2].

However, after running that comand it seems like the new exit code will almost always become 0, since we will most certainly be able to kill the entrypoint.sh bash script without any problems. The bad thing about this is that it masks all the other error codes which may arise.

Using this new method, which is more verbose and less generic, is that it is possible to get an exit status like 130( = 128+2 -> Container terminated by Ctrl+C). This will greatly help if you are trying to do different things with the Docker containers depending on how they exit, or if you are tying to troubleshoot.

Please sign in to comment.