Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout option to docker run #1905

Closed
benbro opened this issue Sep 17, 2013 · 57 comments

Comments

Projects
None yet
@benbro
Copy link

commented Sep 17, 2013

This is a feature request to add a timeout option when calling the docker run command.
The option called be -to=5000 (5000 ms).

I'm using docker to run untrusted student code in a container (C, Python...).
It will be useful to be able to set a timeout to limit how much time the container is allowed to run before it docker stops it.

Thanks

@crosbymichael

This comment has been minimized.

Copy link
Member

commented Sep 17, 2013

@benbro Thanks for the idea. What other use cases for docker would this be needed for? I am trying to decide if this should be in the scope of docker itself or if this should be handled by an external tool.

@benbro

This comment has been minimized.

Copy link
Author

commented Sep 17, 2013

I think it is useful in any case you use a container as executable.

A user can wrap the call to docker run with a timeout in bash:
timeout 5 docker run...
but than he'll need to take care of the cleanup which might not be trivial.

When the -rm option will be added, the container might be left if docker run is stopped or exited.
If docker will handle a timeout, this issue might be minimized.

A timeout could also be useful with the remote API.
If you are executing a command using the API with an HTTP request, you probably want it to timeout after some time and cleanup.

@tianon

This comment has been minimized.

Copy link
Member

commented Sep 20, 2013

I've given this issue quite a bit of thought over the past several days, and here's the solution I've come up with that does the equivalent of a container timeout today, including cleanup, with no changes to the docker core: (I've got this script saved as docker-run-timeout.sh in my PATH.)

#!/bin/bash
set -e

to=$1
shift

cont=$(docker run -d "$@")
code=$(timeout "$to" docker wait "$cont" || true)
docker kill $cont &> /dev/null
echo -n 'status: '
if [ -z "$code" ]; then
    echo timeout
else
    echo exited: $code
fi

echo output:
# pipe to sed simply for pretty nice indentation
docker logs $cont | sed 's/^/\t/'

docker rm $cont &> /dev/null

Here's some example output from the script:

$ docker-run-timeout.sh 10s busybox sh -c 'echo start && sleep 30 && echo finish'
status: timeout
output:
    start
$ docker-run-timeout.sh 10s busybox sh -c 'echo start && sleep 5 && echo finish'
status: exited: 0
output:
    start
    finish
$ docker-run-timeout.sh 10s busybox sh -c 'echo start && sleep 5 && echo finish && exit 42'
status: exited: 42
output:
    start
    finish

Now, we don't get real-time output from the process as it runs this way, but if this were implemented somewhere outside the shell (such as in a Go library consuming the docker API proper), that should be easier to implement than it is here.

@crosbymichael

This comment has been minimized.

Copy link
Member

commented Nov 27, 2013

I still believe this is out of the scope of docker. Docker provides all the primitives required to do what is required.

Also the -rm option has been added now.

@andreisoare

This comment has been minimized.

Copy link

commented May 29, 2014

@benbro what about running timeout inside docker?

docker run --rm ubuntu timeout 2 sh -c 'echo start && sleep 30 && echo finish'

@benbro

This comment has been minimized.

Copy link
Author

commented May 29, 2014

@andreisoare thanks for the suggestion.
I think that a time in the host or in the container will work.
Having a timeout option as a parameter to docker run feels cleaner to me but I can live without it :)

@yoones

This comment has been minimized.

Copy link

commented Dec 17, 2014

+1

docker run --rm ubuntu timeout 2 sh -c 'echo start && sleep 30 && echo finish'

When you run untrusted code, a timeout inside your container cannot be trusted.

Consider the following:

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>

int main()
{
  int pid;

  if ((pid = fork()) == 0)
    {
      setsid();
      sleep(5);
      printf("Done here.\n");
    }
  else if (pid)
    wait(NULL);
  return (0);
}

In this case: timeout 2 ./a.out, the child process will stay alive after the timeout.
Only a timeout on the container itself can really be effective since it can't be affected by what happens inside the container.

@tianon

This comment has been minimized.

Copy link
Member

commented Dec 19, 2014

In the case you describe, the parent process would exit, and it would be pid 1, so the container would be destroyed and thus the child process also killed and reaped with the entire pid namespace.

@yoones

This comment has been minimized.

Copy link

commented Dec 19, 2014

@tianon The point is that you shouldn't trust anything that's happening inside your container when you run untrusted code inside it. That's why I would prefer being able to set a timeout on the container itself.

To give you another example:

$> cat ./test.sh

#!/bin/bash

pid=$(ps ax | grep timeout | grep -v grep | cut -d ' ' -f 2)
kill -19 $pid
sleep 5
echo "Done."
kill -18 $pid

$> timeout 2 ./test.sh

@imdreamrunner

This comment has been minimized.

Copy link
Contributor

commented Mar 18, 2015

Thanks for the discussing. I am looking for exactly the same thing discussed in this issue.

@jez jez referenced this issue Mar 25, 2015

Merged

Implementation of Docker vmms #45

11 of 12 tasks complete
@CrashyBang

This comment has been minimized.

Copy link

commented Jul 7, 2015

@tianon would it be possible to extend your script to work with infinite loops, at the moment the container gets stuck processing the request and is not killed.

http://stackoverflow.com/questions/31266529/killing-a-docker-container-running-an-infinite-process

ie:

while True:
   print "infinite loop"

Will get stuck.

@domino14

This comment has been minimized.

Copy link

commented Jul 8, 2015

@titpetric

This comment has been minimized.

Copy link

commented Nov 25, 2015

If you run a docker entrypoint, which is a cli program (and not a deamon), would a viable option of limiting execution time be with the available programs, like timelimit? ie:

timelimit -t 20 -T 10 docker run -it ubuntu bash ...?

I'm doing some long running processing from a docker image, and need to limit the execution time of each docker run. I could do it with timelimit inside (entrypoint) or outside of docker as suggested... only thing I'm worried about if some kind of cleanup (--rm=true) will work with signal 15/9 sent to docker.

@pjebs

This comment has been minimized.

Copy link

commented Feb 29, 2016

Could someone please reconsider implementing this feature. It is trivial for Docker to add this timeout option but would make our implementations more elegant and less error prone.

In understand that it is not strictly necessary as per @crosbymichael but it opens up possibilities like this: kubernetes/kubernetes#21925

@pjebs pjebs referenced this issue Feb 29, 2016

Closed

Temporary pod #21925

@domino14

This comment has been minimized.

Copy link

commented Feb 29, 2016

it's probably not trivial...

@pjebs

This comment has been minimized.

Copy link

commented Feb 29, 2016

It's hard to believe it wouldn't be trivial. It seems to be a policy decision that is preventing the team.

@thaJeztah

This comment has been minimized.

Copy link
Member

commented Feb 29, 2016

Perhaps think of something like a "killer" / garbage-collect container, containing a cron-job, that kills any container that's started with a specific label and has been running for xx seconds

@pjebs

This comment has been minimized.

Copy link

commented Feb 29, 2016

I'm aware of lots of work-arounds. But they all seem like overkill considering how much more elegant a -to option would be to docker run

@thaJeztah

This comment has been minimized.

Copy link
Member

commented Feb 29, 2016

Implementing it in docker would involve starting timers for these containers, and implementing logic to stop those containers; it may sound trivial, but can result in having to add a lot of moving parts that has to be supported and maintained.

@pjebs

This comment has been minimized.

Copy link

commented Mar 16, 2016

Firstly when you type in docker ps -a you can clearly see how long a container has been running for.
Secondly we know that the user can always type in docker rm -f XXXXX to force-destroy a container.

Therefore we know both essential functionality are already implemented.
Therefore it is TRIVIAL to implement a -to (timeout) parameter to the docker run command by combining the above-mentioned functionality.

@pjebs

This comment has been minimized.

Copy link

commented Mar 16, 2016

(worst-case scenario) Perhaps a few hours to code and perhaps max 1 week of unit testing after it has been implemented.

@titpetric

This comment has been minimized.

Copy link

commented Mar 16, 2016

Fyi, timelimit wrap around docker run seems to work as expected. Why is it not good enough? Perhaps you would be fine with more detailed documentation as to what happens to the docker container if not detached and gets a kill 15/9? If 'screen' or 'tmux' is fine for putting a foreground process in the background, why is timelimit not enough to kill a process and it's children after some time?

@espadrine

This comment has been minimized.

Copy link

commented Jul 11, 2016

I am probably misunderstanding the situation, but isn't docker run --ulimit cpu=10 enough to stop the process after 10 seconds? (It does not work locally, but I would expect it to work.)

@jfhbrook jfhbrook referenced this issue Sep 12, 2016

Closed

build timeouts #134

@jayaprabhakar

This comment has been minimized.

Copy link

commented Sep 13, 2016

@ondrejhlavacek The issue with timeout is that, it doesn't do the cleanup correctly. Specifically when run with the --rm option that is supposed to cleanup after the container completes, using timeout will block the cleanup from happening.

Try running docker ps -a after the timeout command to see that the cleanup didn't complete.

@ondrejhlavacek

This comment has been minimized.

Copy link

commented Sep 13, 2016

@jayaprabhakar And that's another very good point. I forgot to mention, that we do the cleanup manually. Filling the void in the blogpost!

@jayaprabhakar

This comment has been minimized.

Copy link

commented Sep 13, 2016

I am having an weird issue using --ulimit cpu=10.

This works fine.
docker run --rm -i -v /home/jayaprabhakar/test:/classes --ulimit cpu=5 java:8 bash -c 'date; java -cp /classes/ InfiniteLoop; date'

But this doesn't.
docker run --rm -i -v /home/jayaprabhakar/test:/classes --ulimit cpu=5 java:8 java -cp /classes/ InfiniteLoop

Any idea how to fix this issue?

@HugoDelval

This comment has been minimized.

Copy link

commented Mar 31, 2017

Hi,
Is this issue still in docker ?
It seems to me like a nice feature to have in docker run. If you want to launch untrusted code you have to delete your container after launching it. Of course it is possible to script it as @jayaprabhakar said but it is not really elegant.

@thaJeztah

This comment has been minimized.

Copy link
Member

commented Mar 31, 2017

Docker 1.13 and up (including 17.03) now moved the --rm feature to the daemon, this means that you can set --rm on any container, and it will automatically removed when the container exits

@HugoDelval

This comment has been minimized.

Copy link

commented Mar 31, 2017

Yes but the container has to exit first before being removed. The point of this issue is to set a timeout on a container after which the container will automatically exit (and then removed if the --rm option is enabled).

@jakajancar

This comment has been minimized.

Copy link

commented Jul 6, 2017

3 years to realize why it's good to have --rm on the daemon (= to not have orphaned stopped containers).

Perhaps in another 3 (~year 2020) it will be clear why it's good to have --timeout on the daemon (=to not have orphaned running containers).

:)

@anhilde

This comment has been minimized.

Copy link

commented Aug 3, 2017

Hi, I want to debug the oci-systemd-hook since I can't currently start my containers. I fail because I can't set the --timeout as requested here. Is there any idea how I should proceed without?

@thaJeztah

This comment has been minimized.

Copy link
Member

commented Aug 3, 2017

@anhilde oci-systemd-hook is not used by docker, only in the Red Hat fork. The --timeout requested here would just docker kill the container, so you can do that manually

@anhilde

This comment has been minimized.

Copy link

commented Aug 7, 2017

@thaJeztah docker wants the container to be up and running within a certain time period. Since I need to attach the debugger and do my debugging, docker does not give me the time to do so. How can I change the appropriate timeout, so docker waits longer for the container to be up and running.

@thaJeztah

This comment has been minimized.

Copy link
Member

commented Aug 7, 2017

@anhilde your question is not related to the issue being discussed here, and doesn't look to be a bug-report. Keep in mind that the GitHub issue tracker is not intended as a general support forum,
but for reporting bugs and feature requests. For other type of questions, consider using one of;

@lavvy

This comment has been minimized.

Copy link

commented Sep 29, 2017

Any resolution yet

@DesktopMan

This comment has been minimized.

Copy link

commented Nov 6, 2017

I'm trying to solve this without having a timeout option in docker itself (which would be very useful):

timeout 5 docker run -i -t --rm continuumio/anaconda3 /bin/bash

I would expect this container to be stopped and removed, but it is not being stopped nor being removed. Is there any signal I can send to trigger this?

@ondrejhlavacek

This comment has been minimized.

Copy link

commented Nov 6, 2017

@DesktopMan You can run the cleanup manually afterwards.

timeout 5 docker run -it --name mycontainer --rm continuumio/anaconda3 /bin/bash
docker rm mycontainer
@DesktopMan

This comment has been minimized.

Copy link

commented Nov 6, 2017

The container doesn't even stop though, is there documentation on how docker run reacts to signals anywhere?

@ondrejhlavacek

This comment has been minimized.

Copy link

commented Nov 6, 2017

@DesktopMan you have to specify the signal which is sent from the timeout command. Fixing my previous example. This works for me:

$ timeout --signal=SIGKILL 5 docker run --rm -it alpine /bin/sh
/ # Killed
$
@DesktopMan

This comment has been minimized.

Copy link

commented Nov 6, 2017

Interesting. That still doesn't stop the container for me.

@ondrejhlavacek

This comment has been minimized.

Copy link

commented Nov 6, 2017

@DesktopMan Interesting. You're right! The container is still running. So a docker kill or docker rm -f is required afterwards.

@saden1

This comment has been minimized.

Copy link

commented Dec 3, 2017

This is really a useful feature for testing purpose and any situation where ephemeral containers could be of use (i.e providing a customer a short lived sandbox environment to play with).

@soullivaneuh

This comment has been minimized.

Copy link

commented Dec 22, 2017

You are regularly talking about timeout 5 docker run... on command line. What about the Docker API? I'm currently trying to manage timeout with docker running with docker-php using the API, and this is not a solution.

The implementation of a timeout option on the docker API should be reconsidered.

@nevmerzhitsky

This comment has been minimized.

Copy link

commented Apr 24, 2018

If the main process of a container does an infinite loop or sleep then timeout will don't stop the container in due time. Thus the solution should use docker rm/kill to stop the container. I wrote this Bash script to achieve the goal.

The main benefit over previous solutions is that if the script process will be killed then it send docker rm before dying. This reduce a risk of orphan non---rm containers on an accident in a host OS.

@pipehappy1

This comment has been minimized.

Copy link

commented May 30, 2018

It would be really handy if docker run has the timeout option. Use timeout is like in order to shutdown a computer, you need to pull the power plug off the wall instead of push some button on the machine itself.

@fulltek

This comment has been minimized.

Copy link

commented Jun 7, 2018

+1 @pipehappy1
It would do help

@esabol

This comment has been minimized.

Copy link

commented Jun 20, 2018

Can we get this issue reopened at least? I would find this feature extremely useful.

@hifall

This comment has been minimized.

Copy link

commented Jun 27, 2018

Anyone else tried the --stop-timeout option yet? Did it work for you?

I wasn't lucky enough to make it work last time I tried.

@chrisgavin

This comment has been minimized.

Copy link
Contributor

commented Aug 31, 2018

@hifall That doesn't look relevant to me. It doesn't limit the time a container can run for, it just limits how long the container has to stop gracefully when it gets stopped.

@jtpavlock

This comment has been minimized.

Copy link

commented Oct 4, 2018

This may not work for everyone, but it was good enough for my use:

docker stop -t 15 $(docker run -d --stop-signal 2 --rm $(docker build -q .))

I utilized the fact that docker stop will send a SIGTERM by default, and then after a specified timeout, it will send a SIGKILL to the container. Normally, this would still stop the container once the command is issued (most things will stop to SIGTERM), but by setting --stop-signal 2 (or really any signal that isn't SIGTERM or SIGKILL) you can tell the container not to respond to a SIGTERM signal.

This means docker stop will send the SIGTERM, have no effect, and then after a timeout will send a SIGKILL which will kill the running container.

If you need to run the container interactively, the two commands (run and stop) will have to be split and run in different processes e.g. open a new shell.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.