docker_service Unable to load docker-compose.

[grizzlyanderson]

The symptoms are the same as 3906, which is closed as resolved.

ISSUE TYPE

• Bug Report

COMPONENT NAME

docker_service

ANSIBLE VERSION

ansible --version
ansible 2.2.0.0
  config file =
  configured module search path = Default w/o overrides

$ docker-compose --version
docker-compose version 1.8.0, build f3628c7

CONFIGURATION

OS / ENVIRONMENT

OS-X 10.11.16 (executing playbook here)

host, running on virtualbox:
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

SUMMARY

"Unable to load docker-compose. Try pip install docker-compose. Error: cannot import name build_action_from_opts"
The results seems to be the same as 3906, which I believe is included in the 2.2.0 release.

STEPS TO REPRODUCE

added docker_service section to playbook, ran playbook on OS-X, host is Ubuntu 16.04

  - name: copy compose file to remote server
    copy:
      force: yes
      src: "{{ local_path }}"
      dest: "{{ remote_path }}{{ remote_file }}"
  - name: docker-compose via ansible docker_service
    docker_service:
      files:
        - "{{ remote_file }}"
      project_src: "{{ remote_path }}"
      project_name: "bbs-services"
      pull: true
      state: present
      restarted: true

EXPECTED RESULTS

PLAY [local-services] **********************************************************

TASK [setup] *******************************************************************
ok: [192.168.67.25]

TASK [copy compose file to remote server] **************************************
ok: [192.168.67.25]

TASK [docker-compose via ansible docker_service] *********************
ok: [192.168.67.25]

PLAY RECAP *********************************************************************
192.168.67.25 : ok=3 changed=0 unreachable=0 failed=0

ACTUAL RESULTS

partial result (failed task only)

TASK [docker-compose via ansible docker_service] *********************
task path: /Users/eric.anderson/projects/stash.innitrode.com/SFB/bbs-services/ansible/servers.yaml:52
Using module file /Users/eric.anderson/projects/virtualenvs/ansible/lib/python2.7/site-packages/ansible/modules/core/cloud/docker/docker_service.py
<192.168.67.25> ESTABLISH SSH CONNECTION FOR USER: billing
<192.168.67.25> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=billing -o ConnectTimeout=10 -o ControlPath=/Users/eric.anderson/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.67.25 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1478553260.5-110070586491346 `" && echo ansible-tmp-1478553260.5-110070586491346="` echo $HOME/.ansible/tmp/ansible-tmp-1478553260.5-110070586491346 `" ) && sleep 0'"'"''
<192.168.67.25> PUT /var/folders/nb/6r1j2wjn5qxb_bbqj31nmx7rncv064/T/tmpXfl3DW TO /home/billing/.ansible/tmp/ansible-tmp-1478553260.5-110070586491346/docker_service.py
<192.168.67.25> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=billing -o ConnectTimeout=10 -o ControlPath=/Users/eric.anderson/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.67.25]'
<192.168.67.25> ESTABLISH SSH CONNECTION FOR USER: billing
<192.168.67.25> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=billing -o ConnectTimeout=10 -o ControlPath=/Users/eric.anderson/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.67.25 '/bin/sh -c '"'"'chmod u+x /home/billing/.ansible/tmp/ansible-tmp-1478553260.5-110070586491346/ /home/billing/.ansible/tmp/ansible-tmp-1478553260.5-110070586491346/docker_service.py && sleep 0'"'"''
<192.168.67.25> ESTABLISH SSH CONNECTION FOR USER: billing
<192.168.67.25> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=billing -o ConnectTimeout=10 -o ControlPath=/Users/eric.anderson/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.67.25 '/bin/sh -c '"'"'/usr/bin/python /home/billing/.ansible/tmp/ansible-tmp-1478553260.5-110070586491346/docker_service.py && sleep 0'"'"''
fatal: [192.168.67.25]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "api_version": null,
            "build": false,
            "cacert_path": null,
            "cert_path": null,
            "debug": false,
            "definition": null,
            "dependencies": true,
            "docker_host": null,
            "files": [
                "bbs-services-local.yml"
            ],
            "filter_logger": false,
            "hostname_check": false,
            "key_path": null,
            "nocache": false,
            "project_name": "bbs-services",
            "project_src": "/home/billing/",
            "pull": true,
            "recreate": "smart",
            "remove_images": null,
            "remove_orphans": false,
            "remove_volumes": false,
            "restarted": true,
            "scale": null,
            "services": null,
            "ssl_version": null,
            "state": "present",
            "stopped": false,
            "timeout": 10,
            "tls": null,
            "tls_hostname": null,
            "tls_verify": null
        },
        "module_name": "docker_service"
    },
    "msg": "Unable to load docker-compose. Try `pip install docker-compose`. Error: cannot import name build_action_from_opts"
}
	to retry, use: --limit @/Users/eric.anderson/projects/stash.innitrode.com/SFB/bbs-services/ansible/servers.retry

PLAY RECAP *********************************************************************
192.168.67.25              : ok=2    changed=0    unreachable=0    failed=1

[ansibot]

@ansible ping, this issue is waiting for your response.
click here for bot help

[jappievw]

Any luck on this in the meantime? Workaround probably?

[jappievw]

Found a workaround in the meantime on my mac. Ansible uses a different Python version than macOS does. Running /usr/local/Cellar/ansible/2.2.0.0_1/libexec/bin/pip install docker-compose worked for me.

[grizzlyanderson]

tried the docker install on the OS X machine, no change. I was using the word "host" wrong in my description (changed now)

[jappievw]

Ah, clear. Did you ensure docker is running and available in the box? What versions of docker and docker-compose do you use there?

[grizzlyanderson]

yes - if I ssh to the virtual box and run docker-compose on the yml file that ansible copied over to that box, the containers spin up fine.

billing@ansibleTarget:~$ docker version
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64
billing@ansibleTarget:~$ docker-compose version
docker-compose version 1.8.0, build f3628c7
docker-py version: 1.9.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
billing@ansibleTarget:~$ docker-compose -f bbs-services-local.yml up -d
Starting hudson_bbs-service-rt_1
Starting hudson_bbs-service-l3_1

[coodix]

Got same error:

TASK [Run/restart docker compose] **********************************************
fatal: [x.x.x.x]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to load docker-compose. Try `pip install docker-compose`. Error: No module named compose"}
        to retry, use: --limit @/.../ansible_deploy.retry

Docker installed on OS X by Docker Tollbox:

✗ docker-compose --version
docker-compose version 1.9.0-rc4, build 181a4e9

Ansbile:

ansible --version
ansible 2.2.0.0
  config file = .../ansible.cfg
  configured module search path = Default w/o overrides

deploy.yml:

---
- hosts: all
  user: root

  tasks:
    - name: Clone git repository
      git: >
        dest=<remote_project_path>
        repo=<repo.git>
    - name: Run/restart docker compose
      docker_service: >
        project_src=<remote_project_path>
        restarted=yes

[ansibot]

@ansible, ping. This issue is still waiting on your response.
click here for bot help

[coodix]

Problem was in that there wasn't pip and mandatory libs installed on target server. Resolved by adding appropriate instructions into playbook.

[psionikangel]

@coodix What were the libs that you had to install on the target server in order for it to work? They're not mentionned in the documentation at https://docs.ansible.com/ansible/docker_service_module.html

[coodix]

@psionikangel , actually requirements are described on that page below the "Requirements (on host that executes module)" header.

My working playbook is:

---
- hosts: all
  user: root

  tasks:
    - name: Install pip, python-dev package with apt
      apt:
        pkg: "{{ item }}"
        state: latest
        update_cache: yes
        cache_valid_time: 600
      with_items:
        - python-dev
        - python-pip
    
    # Upgrade pip with pip to fix angstwad/docker.ubuntu/pull/35 and docker-py/issues/525
    # Install latest version when no specific release is set.
    - name: Upgrade latest pip, setuptools, docker-py and docker-compose with pip
      pip:
        name: "{{ item.name }}"
        state: latest
      with_items:
        - { name: pip, version: "latest", install: true }
        - { name: setuptools, version: "latest", install: true }
        - { name: docker-py, version: "latest", install: true }
        - { name: docker-compose, version: "latest", install: true }
      when: (item.version=="latest" and item.install)
      ignore_errors: yes

    # Install specific version when set in the variables
    - name: Install specific pip, setuptools, docker-py and docker-compose with pip
      pip:
        name: "{{ item.name }}"
        state: present
        version: "{{ item.version }}"
      with_items:
        - { name: pip, version: "latest", install: "true" }
        - { name: setuptools, version: "latest", install: "true" }
        - { name: docker-py, version: "latest", install: "true" }
        - { name: docker-compose, version: "latest", install: "true" }
      when: (item.version!="latest" and item.install)

    - name: Run/restart docker compose
      docker_service: >
        project_src=/root/www
        restarted=yes
        debug=yes

The idea is that python, pip and docker_compose package should be installed on target machine.

[psionikangel]

This was strange. I had all the requirements already installed on the target server. The only difference is I had installed them through apt instead of pip.

Installing them with pip fixed the issue.

[grizzlyanderson]

yep - this is definitely a problem with docker components installed using apt instead of pip. Installing via pip in ansible or via pip on the host works.

[ansibot]

This repository has been locked. All new issues and pull requests should be filed in https://github.com/ansible/ansible

Please read through the repomerge page in the dev guide. The guide contains links to tools which automatically move your issue or pull request to the ansible/ansible repo.

[kakawait]

Just for information, you can also have another Unable to load docker-compose (but for different reason Error: cannot import name splitdrive) if you're using latest (==1.16.0) pip docker-compose package.

See upstream issue on docker/compose#5156

[charignon]

I can also repro this with the latest docker-compose and pip, the same playbook was running fine three days ago. docker-compose 1.15.0 works for me with ansible 2.4.0 while docker-compose 1.16.1 fails with the same version of ansible.

[sdcarter]

Same error here: fatal: [******]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to load docker-compose. Try pip install docker-compose. Error: cannot import name splitdrive"}

[shithead]

+1

ansible --version
ansible 2.3.2.0
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides
python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

docker-compose --version:
docker-compose version 1.16.1, build 6d1ac219

[bpiselli]

+1

• ansible 2.3.1.0
• Python 2.7.12

on remote host

• Docker version 17.05.0-ce, build 89658be
• docker-compose version 1.16.1, build 6d1ac219

[yajo]

Those of you saying +1: Your comments are a hassle and completely unnecessary, please stop doing that anywhere.

Now, on the issue at hand, it seems like the official resolution from the docker-compose team will be to say that there's no API guarantee for those trying to use compose as a library. Maybe the ansible modules should issue system commands to communicate with docker-compose instead of trying to use it as a library, to avoid these kind of problems in the future.

[ansibot]

This issue was migrated to ansible/ansible#29510

[j00bar]

For the folks getting the splitdrive error, that's really ansible/ansible#29498, not this bug.

For those users, Ansible 2.3.x and Docker Compose 1.16.x don't get along. Either move up to 2.4rc's or limit yourself to Docker Compose <1.16.