podman generate systemd yields file, that does not allow shutting down of a container without error
#11304
Comments
|
Thanks for reaching out, @polygamma!
What's happening is that the container doesn't shutdown on SIGTERM, so systemd is using the big SIGKILL hammer. We dropped the custom |
|
@vrothberg The problem is that systemd is running inside the container and systemd expects |
|
I share your opinion, @Luap99. Note that it also fails in simpler scenarios just running |
|
Can you expand on why we dropped the custom I think, if we have to choose between the two, it's a lot more important that user-initiated |
We wanted to get rid of using
I concur, |
Commit 9ac5267 changed the type of the generated systemd units from forking to notify. Parts of these changes was also removing the need to pass any information via the file system (e.g., PIDFILE, container ID). That in turn implies that systemd takes care of stopping the container. By default, systemd first sends a SIGTERM and after a certain timeout, it'll send a SIGKILL. That's pretty much what Podman is doing, unless the container was created with a custom stop signal which is the case when the --stop-signal flag was used or systemd is mounted. Account for that by using systemd's KillSignal option which allows for changing SIGTERM to another signal. Also make sure that we're using the correct timeout. Fixes: containers#11304 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
|
@polygamma, any chance you try rerunning with #11312? |
Commit 9ac5267 changed the type of the generated systemd units from forking to notify. Parts of these changes was also removing the need to pass any information via the file system (e.g., PIDFILE, container ID). That in turn implies that systemd takes care of stopping the container. By default, systemd first sends a SIGTERM and after a certain timeout, it'll send a SIGKILL. That's pretty much what Podman is doing, unless the container was created with a custom stop signal which is the case when the --stop-signal flag was used or systemd is mounted. Account for that by using systemd's KillSignal option which allows for changing SIGTERM to another signal. Also make sure that we're using the correct timeout. Fixes: containers#11304 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Commit 9ac5267 changed the type of the generated systemd units from forking to notify. Parts of these changes was also removing the need to pass any information via the file system (e.g., PIDFILE, container ID). That in turn implies that systemd takes care of stopping the container. By default, systemd first sends a SIGTERM and after a certain timeout, it'll send a SIGKILL. That's pretty much what Podman is doing, unless the container was created with a custom stop signal which is the case when the --stop-signal flag was used or systemd is mounted. Account for that by using systemd's KillSignal option which allows for changing SIGTERM to another signal. Also make sure that we're using the correct timeout for units generated with --new. Fixes: containers#11304 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Commit 9ac5267 changed the type of the generated systemd units from forking to notify. Parts of these changes was also removing the need to pass any information via the file system (e.g., PIDFILE, container ID). That in turn implies that systemd takes care of stopping the container. By default, systemd first sends a SIGTERM and after a certain timeout, it'll send a SIGKILL. That's pretty much what Podman is doing, unless the container was created with a custom stop signal which is the case when the --stop-signal flag was used or systemd is mounted. Account for that by using systemd's KillSignal option which allows for changing SIGTERM to another signal. Also make sure that we're using the correct timeout for units generated with --new. Fixes: containers#11304 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
It works "better", since the [jonny@jonny-work-pc ~]$ sudo systemctl status container-test
× container-test.service - Podman container-test.service
Loaded: loaded (/etc/systemd/system/container-test.service; disabled; vendor preset: disabled)
Active: failed (Result: signal) since Tue 2021-08-24 11:58:11 CEST; 42s ago
Docs: man:podman-generate-systemd(1)
Process: 3120 ExecStart=/usr/bin/podman run --cgroups=no-conmon --rm --sdnotify=conmon -d --replace -t --name test --privileged --network=host --ipc=host --systemd=true docker.io/archli>
Main PID: 3120 (code=killed, signal=RTMIN+3)
CPU: 167ms
Aug 24 11:57:59 jonny-work-pc podman[3081]: 2021-08-24 11:57:59.607832844 +0200 CEST m=+0.028930879 image pull docker.io/archlinux/archlinux:base
Aug 24 11:58:00 jonny-work-pc podman[3081]: 2021-08-24 11:58:00.183792866 +0200 CEST m=+0.604890952 container create 7f18f8b449233aa5f3a339b63285655f63a69009529afa570235cb5f2709ecdd (image=>
Aug 24 11:58:00 jonny-work-pc podman[3081]: 2021-08-24 11:58:00.548624061 +0200 CEST m=+0.969722140 container init 7f18f8b449233aa5f3a339b63285655f63a69009529afa570235cb5f2709ecdd (image=do>
Aug 24 11:58:00 jonny-work-pc systemd[1]: Started Podman container-test.service.
Aug 24 11:58:00 jonny-work-pc podman[3081]: 2021-08-24 11:58:00.642316377 +0200 CEST m=+1.063414461 container start 7f18f8b449233aa5f3a339b63285655f63a69009529afa570235cb5f2709ecdd (image=d>
Aug 24 11:58:00 jonny-work-pc podman[3081]: 7f18f8b449233aa5f3a339b63285655f63a69009529afa570235cb5f2709ecdd
Aug 24 11:58:11 jonny-work-pc systemd[1]: Stopping Podman container-test.service...
Aug 24 11:58:11 jonny-work-pc systemd[1]: container-test.service: Main process exited, code=killed, status=37/RTMIN+3
Aug 24 11:58:11 jonny-work-pc systemd[1]: container-test.service: Failed with result 'signal'.
Aug 24 11:58:11 jonny-work-pc systemd[1]: Stopped Podman container-test.service. |
Commit 9ac5267 changed the type of the generated systemd units from `forking` to `notify`. It further stopped using `--cidfile` and instead intended systemd to take care of stopping the container, which turned out to be a bad idea. Systemd will send the stop/kill signals to conmon which in turn may exit non-zero, depending on the signal, and ultimately breaking container cleanup. Hence, we need to use --cidfile again and let podman stop and remove the container to make sure that everything's in order. Fixes: containers#11304 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
|
#11315 will restore the previous behavior. This time for real :^) |
Works as intended now :) |
Commit 9ac5267 changed the type of the generated systemd units from `forking` to `notify`. It further stopped using `--cidfile` and instead intended systemd to take care of stopping the container, which turned out to be a bad idea. Systemd will send the stop/kill signals to conmon which in turn may exit non-zero, depending on the signal, and ultimately breaking container cleanup. Hence, we need to use --cidfile again and let podman stop and remove the container to make sure that everything's in order. Fixes: containers#11304 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Commit 9ac5267 changed the type of the generated systemd units from `forking` to `notify`. It further stopped using `--cidfile` and instead intended systemd to take care of stopping the container, which turned out to be a bad idea. Systemd will send the stop/kill signals to conmon which in turn may exit non-zero, depending on the signal, and ultimately breaking container cleanup. Hence, we need to use --cidfile again and let podman stop and remove the container to make sure that everything's in order. Backport of commit 74ab2aa. Fixes: containers#11304 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
github-actions
bot
added
the
locked - please file new issue/PR
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
With Podman 3.3.0 the
podman generate systemdcommand yields a file, that does not allow shutting down of a container without error.With Podman 3.2.3 everything works as expected.
Steps to reproduce the issue:
sudo podman container rm --force testPodman 3.2.3
Podman 3.3.0
Describe the results you received:
Status of the systemd service is
Active: failed (Result: timeout)Describe the results you expected:
Status of the systemd service should be
Active: inactive (dead)Output of
podman version:Output of
podman info --debug:Package info (e.g. output of
rpm -q podmanorapt list podman):Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)
Yes
The text was updated successfully, but these errors were encountered: