Add docker_exec module

[bschelberg]

ISSUE TYPE

• New Module Pull Request

COMPONENT NAME

docker_exec

ANSIBLE VERSION

ansible 2.3.0 (devel f04aee9754) last updated 2017/01/11 13:53:28 (GMT +1100)
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

SUMMARY

Allows running "docker exec" on remote Docker hosts. The new Docker modules such as docker_container support management of remote Docker hosts, and being able to do an exec remotely is useful to me. It appears to be useful to others also, see the comment on the question here: http://stackoverflow.com/questions/32878795/run-command-inside-of-docker-container-using-ansible

The module can be used in a playbook like so:

  - name: Register gitlab runner
    docker_exec: 
      command: <some command>
      docker_host: <docker host>
      name: <container name>
    register: exec_output

  - name: Registration output
    debug: msg="{{ exec_output.result }}"

Using a register and debug was the best way that I could find to see the result of the exec. Also note that it's not possible AFAIK (at least using the docker.py library) to get the return status of the command. As far as I can see, it's thrown away in docker/api/exec_api.py when it calls _get_result_tty(). As a result, we can only assume that the result is "changed=True", and leave it up to the user to determine whether or not the exec worked. Parsing the result and determining success or failure will depend on what the command was, so can't be implemented in the module.

This is my first attempt at an Ansible module, and my first use of Python at all, so any suggestions for improvements are quite welcome.

[gundalow]

needs_revision

[gundalow]

Please remove this line

[gundalow]

version_added: "2.3"

[gundalow]

Would be good if an example showed how to get the output returned by running the command.

• Does stdout and stderr get combined, or are they separate streams?
• Document RETURN

[chouseknecht]

I'm not trying to be snarky or rude by asking this, so please don't take it that way. I appreciate that writing a module is a time consuming, and often tedious process. The payoff of course is seeing your creation merged into the code base. I've been there.

But all that being said, I feel I need to be that guy, and ask... Maybe I'm missing the obvious, but can you explain how using your module is better than simply using the docker connection plugin?

[bschelberg]

@chouseknecht No offense taken, thanks for being that guy and pointing out something I may have missed. I couldn't find how to specify the Docker remote host using the connection driver. Yes, the connection driver would allow to run docker exec on a container on the host where the ansible script is running, but it doesn't seem to support connecting to a docker container on a remote host (actually, I could do this even with the command module, and running "docker exec ..."). It seems I'm not the only one who doesn't know how to do this. See the comments at the bottom of this page: http://blog.oddbit.com/2015/10/13/ansible-20-the-docker-connection-driver/

[Yannig]

You have severals options: setting DOCKER_HOST=tcp://xxxx:xxx or using docker_extra_args..

[sivel]

We do additionally have documentation about non ssh connection methods at along with examples of how to use the docker connection plugin: http://docs.ansible.com/ansible/intro_inventory.html#non-ssh-connection-types

[bschelberg]

Please forgive me, I don't know if I'm still doing something wrong, but it looks like the docker connection only works if python is installed in the docker container??

This is my playbook:

- hosts: localhost

  tasks:
    - name: add container to inventory
      add_host:
        name: gitlab-runner
        ansible_connection: docker
        ansible_docker_extra_args: "-H=tcp://dev-docker-ci1.local:2375"
      changed_when: false

    - name: List gitlab runners
      delegate_to: gitlab-runner
      command: gitlab-runner list

This is the output I get from running: ansible-playbook -vvv gitlab-runner-list.playbook:
...skipping the first part...

task path: /work/ansible-playbooks/gitlab-runner-list.playbook:12
 [WARNING]: unable to retrieve default user from docker container:  Cannot connect to the Docker daemon. Is the docker daemon running on this host?

Using module file /work/ansible/lib/ansible/modules/commands/command.py
<gitlab-runner> ESTABLISH DOCKER CONNECTION FOR USER: ?
<gitlab-runner> EXEC ['/usr/bin/docker', u'-H=tcp://dev-docker-ci1.dev.arsrs.local:2375', 'exec', '-i', u'gitlab-runner', u'/bin/sh', '-c', u"/bin/sh -c '/usr/bin/python && sleep 0'"]
fatal: [localhost -> gitlab-runner]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_name": "command"
    }, 
    "module_stderr": "/bin/sh: 1: /usr/bin/python: not found\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 127
}

/usr/bin/python doesn't exist in the container that I'm using (gitlab/gitlab-runner:latest from Docker Hub).

I can run a command on a Docker container that has python installed using the above method, but that requirement would be somewhat annoying when using containers from Docker Hub.

[sivel]

All ansible modules (*nix), other than raw and script, require python to be installed on the remote managed machine.

Try switching your command to raw instead.

[bschelberg]

@sivel Thanks, that gets my playbook working. I admit now, there is no point to my module.

[bjakubiak]

There is actually point to use it. With it one can use ssh to remote docker host and docker exec locally. Useful when you can't connect to remote docker with docker client because of firewall restrictions or whatever.

[dm-coding]

+1 for this seemingly obvious idea. Most of my infrastructure is containerized, most of my hosts remote, and most of my playbooks local. When I'm executing a bunch of tasks against running docker containers I don't want to have to care about what host to run the command on. I want to be able to give ansible an inventory of remote hosts, all of which have docker containers running on them, and let it go to work on each one. The rationale behind the docker connection plugin, that I would want to use ansible to only manage local containers, seems a bit strange to me.

[bschelberg]

@bjakubiak this is not actually what the docker_exec module does. It uses the Docker API to exec a command in a running container. If you have firewall restrictions against docker exec connecting to a remote host, you will probably be blocked if you tried to use docker_exec as well. In your case, you should be able to use the command module to run docker exec like so: http://stackoverflow.com/a/39380620/1303158

[bschelberg]

@djcf I'm not sure that my module would have helped you either - it handles a single docker_host only. The docker connection plugin is certainly designed to manage containers on remote hosts, although I haven't figured out how to add a docker container as a host in an inventory file. This sounds more like what you need.

[bschelberg]

@djcf wait, I got it. By adding a line like this to my inventory:

tmch ansible_connection=docker ansible_host=<container-name> ansible_docker_extra_args="-H=tcp://docker.local:2375"

I could refer to my docker container as "tmch" anywhere that I could usually refer to a normal host. Replace "docker.local" with whatever your docker host is. If you want to use the your container name as the name of your host in ansible, you can omit the ansible_host parameter.

You can make a group of these, just like any typical host, and I think that meets your requirements.

[bjakubiak]

@bschelberg It uses Docker API to exec a command in a running container, agree. But unlike the connection plugin, one can connect to docker host by ssh and use the API locally. This is one of ansible's advantages - to utilize existing ssh accounts and connections.

I find it a common network configuration to have ssh openings by default, but not necessarily docker openings. Using docker API locally works always naturally. Without it docker command wouldn't work at all.

And yes, I came here from that very stackoverflow question. I know using command module I can do everything, but it does just simple error handling, based on command's exit code and output. This is why we tend to use modules instead, isn't it?

[jpic]

Since this was closed, how are we supposed to do on hosts without the docker command and just a docker deamon ?

[leseb]

My use case is a bit different, how can I wrap the execution of a custom module inside a container? I can't really do a docker_exec, I need something that copies the module inside the container instead of the host and then execute it from inside the container.
My problem is that my container has the libraries my modules are consuming to perform the right action.

Maybe it's too specific but I'm sure they are people out there with a similar use case.

Also using ansible_docker_extra_args: "-H=tcp://hostB:2375" implies that the docker daemon do not listen locally which means:

• changing the configuration of the docker daemon to listen on an interface and not a local socket
• introducing a security breach or use certificates, the obvious downside is how complex this implementation becomes

Any idea? Thanks!