ansible pull clones double: 127.0.0.1 and localhost

[robinro]

Issue Type: Bug

Ansible Version: 2.0rc3

Ansible Configuration: default

Environment: openSuSE 42.1

Summary:

ansible-pull runs clone twice.

Steps To Reproduce:

Run ansible-pull on a system with ipv4 working and localhost resolving, i.e.
/usr/bin/ansible-pull -d /var/lib/ansible/local -U your_git_repo

Expected Results:

The ansible git module clone the repo once and then runs inside it.

Actual Results:

The clone is run twice, and one of them fails.

Starting Ansible Pull at 2015-12-28 16:00:42
/usr/bin/ansible-pull -o -d /var/lib/ansible/local -U git@bla

localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "msg": "Failed to download remote objects and refs:  error: cannot lock ref 'refs/remotes/origin/master': Unable to create '/var/lib/ansible/local/.git/refs/remotes/origin/master.lock': File exists.\n\nIf no other git process is currently running, this probably means a\ngit process crashed in this repository earlier. Make sure no other git\nprocess is running and remove the file manually to continue.\nFrom bla\n ! XXX..XXX  master     -> origin/master  (unable to update local ref)\n"
}
127.0.0.1 | SUCCESS => {
    "after": "XXX", 
    "before": "XXX", 
    "changed": true
}

[robinro]

There are 2 possible fixed I can think of. Probably both of them should be implemented:

• remove mention of 127.0.0.1 in https://github.com/ansible/ansible/blob/v2.0.0-0.8.rc3/lib/ansible/cli/pull.py#L133, localhost should always resolve to something and then this is just unnecessary. Also it is inconsistent as it assumes ipv4
• -c local does a local connection no matter what the hostname is, so one could throw away all but the machines hostname; this might cause regressions, but on the other hand a ansible -m destoy)machine -c local othermachine might be dangerous

[bcoca]

or we should limit the number of hosts executed on to 1

[bcoca]

most people don't have both 127.0.0.1 and localhost defined in inventory, probably the reason this has not come up before.

[robinro]

we have our inventory inside the repo (I thought this is standard), therefore during the clone phase the inventory is empty and gets filled with 'localhost,127.0.0.1,::1'

[robinro]

I'll open a pull request soon

[bcoca]

a simple workaround then is calling ansible-pull with -i 'localhost,'

[robinro]

Limiting the number of hosts to 1 is dangerour if an existing playbook uses both hostname and fqdn.

The workaround is possible, but not nice. Ansible 1.9 worked without it.

Can you think of a scenario where ansible-pull --limit might be used. To me it doesn't make sense since

[bcoca]

well, your scenario can be solved by adding --limit to fqdn or localhost or 127.0.0.1

[robinro]

sure, there is a workaround

nevertheless: why not continue to support the 1.9 behavior? It's nice to be able to do ansible-pull without arguments except -U. Defaults are nice and my change does not break anything (as far as I see).

[robinro]

Without this fixed one can not write a call to ansible-pull that works with both 1.9.4 and 2.0, because 1.9.4 does not have a -l argument.

[robinro]

This is still present with 2.0rc4.

[opoplawski]

And with 2.0.0.2. Pretty much breaks my setup that lists multiple groups in local.yml.

[itatabitovski]

The workaround with adding --limit is better than limiting the inventory to localhost only, especially if you keep your inventory in the repo and need to use some data from it.

[robinro]

if you have an inventory in your repo it's currently ignored anyhow, see #13688. Why is a custom --limit better than the default that existed in 1.9.4?

Can you specify your --limit workaround? I couldn't find one that made my setup work, because the clone step needs localhost and the actual run needs the hostname, which is used in the local.yml in my repository.

[itatabitovski]

I have this wrapper script that works correctly with ansible 2.0.0.2, with ansible 1.9.4 --limit is not necessary.

#!/bin/sh

# set some environment variables that ansible-pull needs
export HOME=/root
export PATH={{ ansible_pull_venv_dir }}/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:$PATH

export ANSIBLE_SYSLOG_IDENT="ansible-pull"

{% for k,v in ansible_pull_env_vars.iteritems() -%}
export {{ k }}={{ v }}
{% endfor -%}

ansible-pull \
    -s {{ ansible_pull_sleep }} \
    --accept-host-key \
    -i {{ ansible_pull_hosts }} \
    -e ansible_python_interpreter={{ ansible_pull_venv_dir }}/bin/python \
    {% if ansible_pull_vault_pass is defined -%}
    --vault-password-file {{ ansible_pull_vault_file }} \
    {% endif -%}
    {% if ansible_pull_private_key is defined -%}
    --key-file={{ ansible_pull_private_key_file }} \
    {% endif -%}
    -d {{ ansible_pull_checkout_dir }} \
    -U {{ ansible_pull_repo_url }} \
    --limit {{ inventory_hostname }} \
    {{ ansible_pull_playbook }}
``

[robinro]

@itatabitovski so you have 2 inventories, one that ansible-pull gets {{ ansible_pull_hosts }} and then one inside the repo you actually pull? In my opinion that's not really elegant, as you have to keep those inventories in sync. Or, if you have a dynamical inventory, your ansible-pull setup becomes more complicated because you need authentication info for that dynamical inventory.

I still don't see why you shouldn't use a sensible default when neither -l nor -i are specified. For the clone step there is a very obvious answer who should clone: localhost, and for the actual playbook you should respect ansible.cfg and inventory inside the repository, which currently is not the case.

This would not only restore 1.9.4 behaviour, but as soon as -l and/or -i are set, they are used, so everyones workaround will continue to work.

[itatabitovski]

@robinro sorry I guess it was not clear from my comment.

We have several inventories in our repository depending on the environment (prod, demo, ...), so our project folder structure looks like:

ansible-workspace
├── ci
│   ├── hosts
│   └── group_vars
├── demo
│   ├── hosts
│   ├── group_vars
│   └── host_vars
├── production
│   ├── hosts
│   ├── group_vars
│   └── host_vars

{{ ansible_pull_hosts }} can be ci/hosts or demo/hosts or production/hosts, there is no need to sync to a inventory locally on the pull host.

Regarding the ansible.cfg at the moment I am using ANSIBLE_* env variables to configure ansible. The wrapper script will export all configured variables.

[robinro]

@itatabitovski but this is all in the repository to be checked out by ansible-pull, right? So before the repo is checked-out, ansible v2 still needs an inventory. If you don't give it one you end up with some default (localhost, 127.0.0.1 and ::1), but then you can't us --limit with any fqdn.

I guess your setup works because you maybe start with some initial checkout of your repository and use the inventory in that repository. According to the documentation (and 1.9.4) you should be able to start with a clean system with nothing but ansible. And then your {{ ansible_pull_hosts }} does not exist before the first pull.

Also if the inventory changes your clone runs with a different version than the actualy ansible-playbook run, which might lead to weird bugs.

[itatabitovski]

@robinro yes the inventory exists only in the repository. I did not start with initial checkout, git clone worked fine with and without --limit. The ansible playbook, when running it without limit, ran locally for every host in the inventory. When adding the limit it ran just for the actual pull host.

[gvenka008c]

@here @robinro @bcoca I am getting the error as shown below while running ansible-pull

MacBook-Pro:tmp gvenka008c$ ansible-pull -U git@github.com:XPlat/caps-sps.git -d /var/tmp/caps-sps --inventory=/var/tmp/caps-sps/jenkins_mesos/ansible/inventory --key-file=/Users/gvenka008c/.ssh/gvenkatesan -m git /var/tmp/caps-sps/jenkins_mesos/ansible/jenkins_mesos_slave.yml --limit hostname.sys.net

Starting Ansible Pull at 2016-03-21 14:25:45
/usr/local/bin/ansible-pull -U git@github.com:XPlat/caps-sps.git -d /var/tmp/caps-sps --inventory=/var/tmp/caps-sps/jenkins_mesos/ansible/inventory --key-file=/Users/gvenka008c/.ssh/gvenkatesan -m git /var/tmp/caps-sps/jenkins_mesos/ansible/jenkins_mesos_slave.yml --limit hostname.sys.net

localhost | SUCCESS => {
"after": "36e7682d6fc33d0db95ad163ec819c290e39a55e",
"before": "36e7682d6fc33d0db95ad163ec819c290e39a55e",
"changed": false
}
127.0.0.1 | SUCCESS => {
"after": "36e7682d6fc33d0db95ad163ec819c290e39a55e",
"before": "36e7682d6fc33d0db95ad163ec819c290e39a55e",
"changed": false
}

PLAY ***************************************************************************

TASK [setup] *******************************************************************
fatal: []: FAILED! => {"changed": false, "failed": true, "module_stderr": "sudo: a password is required\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
IPv6 address : ok=0 changed=0 unreachable=0 failed=1

Any thoughts why i am getting the error as
"module_stderr": "sudo: a password is required\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false

Let me know if any details required. Thanks.

ansible --version
ansible 2.0.0.2

[robinro]

Dear @gvenka008c

your problem is unrelated to this issue, so it shouldn't be discussed here but in a separate thread.
Nevertheless some comments:

• the problem in this issue (double 127.0.0.1 and localhost) is fixed in ansible 2.0.1.0, please update your ansible version
• while your output shows the 127*, localhost doubling, this is unrelated to your problem
• the error message you get is fairly descriptive: sudo: a password is required, you seem to use sudo in a setup that requires a password, which obviously does not work with a non-interactive pull. Have a look at your ansible.cfg (in the repo or in /etc/ansible/ansible.cfg) and also try to run the playbook by hand with ansible-playbook to make sure no manual input is required.