Using docker.client.DockerClient.containers.run with auto_remove=True causes a crash

[ceridwen]

What happens is straightforward: setting auto_remove=True in the arguments to run() causes the container to be removed before the Python client attempts to read the logs. This then raises an error like:

docker.errors.NotFound: 404 Client Error: Not Found ("No such container: 0e9a9dd8072a1b80ead74bc47466493ffea92eca7acf1c5f3b29ce6faf861161")

This is not how executing docker run -rm works, where the output still goes to stdout, and it means that calling run() with auto_remove=True isn't useful.

[thiagofigueiro]

+1
Minimal steps to reproduce using example from manual:

In [1]: import docker
In [2]: client = docker.from_env()
In [3]: client.containers.run('alpine:2.7', 'echo hello world')
Out[3]: b'hello world\n'
In [4]: client.containers.run('alpine:2.7', 'echo hello world', auto_remove=True)
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
(...)
~/.pyenv/versions/3.6.3/lib/python3.6/site-packages/docker/errors.py in create_api_error_from_http_exception(e)
     29         else:
     30             cls = NotFound
---> 31     raise cls(e, response=response, explanation=explanation)
     32
     33

NotFound: 404 Client Error: Not Found ("No such container: 6151f0219d2dbb90df7288f30f1221e817fd9d3c63afdbadf2ad2a87ba6e7e45")

[StephenPCG]

This problem is not totally resolved by #1836 . I can still reproduce this issue with current master code.

In [1]: import docker
In [2]: client = docker.from_env()
In [3]: client.containers.run('alpine:latest', 'echo hello world', auto_remove=True)
...
~/.venvs/docker/lib/python3.6/site-packages/docker/errors.py in create_api_error_from_http_exception(e)
     29         else:
     30             cls = NotFound
---> 31     raise cls(e, response=response, explanation=explanation)
     32
     33

NotFound: 404 Client Error: Not Found ("No such container: 2e5677d940399606886ad6ce9303d458c27b6c251c33d624f0660640b9489547")

I think there are still two problems with the code when auto_remove is True:

# docker/models/containers.py
725    container.start()
726
727    if detach:
728     return container
729
730 logging_driver = container.attrs['HostConfig']['LogConfig']['Type']
731
732 out = None
733 if logging_driver == 'json-file' or logging_driver == 'journald':
        # the container may have already been removed if it exited too fast,
        # just like my above example, cause logs() to fail with NotFound.
734     out = container.logs(
735         stdout=stdout, stderr=stderr, stream=True, follow=True
736     )
737
738 exit_status = container.wait()
739 if exit_status != 0:
        # after wait(), container was already removed, thus this logs()
        # call will certainly fail with NotFound.
740     out = container.logs(stdout=False, stderr=True)

[feliperuhland]

Hi, @StephenPCG. I usually use remove=True and docker-py remove the container for me, after getting the log. The auto_remove flag enable auto-removal of the container on daemon side when the container’s process exits and I think it will be harder to get the log.

[shin-]

Thanks guys. As @feliperuhland points out, if you're interested in the container's output, you probably want to use remove over auto_remove.

That said, I submitted #1840 to make sure we don't attempt to get the logs after the wait() call has completed. On the other hand, if the container is removed before we're able to get the logs, I think raising an exception is the expected outcome in this case.

[StephenPCG]

@feliperuhland @shin- Thank you for pointing that out! I didn't realize there is a remove param and the difference between auto_remove. That solves my problem.

[alextriaca]

Been running into this issue too. I've been using the .wait() to get the exit code and then raise an exception if its non zero. Not perfect but at least I know there was an error.

container = client.containers.run(
    image="image",
    command="commands...",
    remove=True,
    detach=True,
)
result = container.wait()
if result["StatusCode"] != 0:
    raise Exception("My exception")