Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2.0] wait_for fails with timeout #14237

Closed
tbondarchuk opened this issue Feb 1, 2016 · 10 comments
Closed

[2.0] wait_for fails with timeout #14237

tbondarchuk opened this issue Feb 1, 2016 · 10 comments

Comments

@tbondarchuk
Copy link

Playbook (wait_for task example straight from documentation):

    - name: Reboot
      command:
        "shutdown -r now 'Rebooted by Ansible'"
      async: 0
      poll: 0
      ignore_errors: true

    - name: Wait for server's sshd to start
      local_action:
        wait_for port="22" host="{{ ansible_host }}" search_regex="OpenSSH" delay=60

result:

TASK [Reboot] ******************************************************************
changed: [test1]

TASK [Wait for server's sshd to start] *****************************************
fatal: [test1 -> localhost]: FAILED! => {"changed": false, "elapsed": 300, "failed": true, "msg": "Timeout when waiting for search string OpenSSH in localhost:22"}

looks like 'wait_for' is checking localhost instead of remote host.
Removed 'local_action' from task, fails immediately with:

TASK [Wait for server's sshd to start] *****************************************
fatal: [test1]: UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh", "unreachable": true}
@bcoca
Copy link
Member

bcoca commented Feb 1, 2016

This is working as expected:
[test1 -> localhost] <= this means the wait_for has been delegated to 'localhost' on behalf of 'test1', using host={{ansible_host}} should point the check 'from localhost to test1'.

That it is hitting the default timeout (300) indicates that the test1 host was not up in that time, increase the timeout if it is just taking the host longer to boot and get ssh up and running.

Closing this as it is not a bug, please post to IRC or the mailing lists first as you can probably get help much faster:

@bcoca bcoca closed this as completed Feb 1, 2016
@tbondarchuk
Copy link
Author

Guess my description was really poor..

The issue is, host is up and accessible long before timeout ends, every time I've run this test.

Here is timeout error on version 1.9.4, when host is indeed not accessible:

TASK: [Wait for server's sshd to start] ***************************************
failed: [test1 -> 127.0.0.1] => {"elapsed": 301, "failed": true}
msg: Timeout when waiting for search string OpenSSH in test1:22

And here is error message from example above:
"msg": "Timeout when waiting for search string OpenSSH in localhost:22"

@bcoca
Copy link
Member

bcoca commented Feb 1, 2016

can you use debug on ansible_host? you might be wanting to use inventory_hostname

@tbondarchuk
Copy link
Author

Run more tests with following playbook:

    - name: show host
      debug:
        msg: "{{ ansible_host }}"

    - name: Reboot
      command:
        "shutdown -r now 'Rebooted by Ansible'"
      async: 0
      poll: 0
      ignore_errors: true

    - name: Wait for server's sshd to start
      local_action:
        wait_for port="22" host="{{ ansible_host }}" search_regex="OpenSSH" delay=60

First result was the same:

TASK [show host] ***************************************************************
ok: [test1] => {
    "msg": "192.168.1.13"
}

TASK [Reboot] ******************************************************************
changed: [test1]

TASK [Wait for server's sshd to start] *****************************************
fatal: [test1 -> localhost]: FAILED! => {"changed": false, "elapsed": 120, "failed": true, "msg": "Timeout when waiting for search string OpenSSH in localhost:22"}

But then I've looked at inventory, and yes it was set to use deprecated variable :(
test1 ansible_ssh_host=192.168.1.13

changed it to new variable test1 ansible_host=192.168.1.13 and finally all worked:

TASK [show host] ***************************************************************
ok: [test1] => {
    "msg": "192.168.1.13"
}

TASK [Reboot] ******************************************************************
changed: [test1]

TASK [Wait for server's sshd to start] *****************************************
ok: [test1 -> localhost]

Looks like variable ansible_host is substituted for deprecated ansible_ssh_host everywhere but wait_for task.

Then I've run slightly more complicated playbook:

    - name: show host
      debug:
        msg: "{{ ansible_host | default(inventory_hostname,boolean=True) }}"

    - name: Reboot
      command:
        "shutdown -r now 'Rebooted by Ansible'"
      async: 0
      poll: 0
      ignore_errors: true

    - name: Wait for server's sshd to start
      local_action:
        wait_for port="22" host="{{ ansible_host | default(inventory_hostname,boolean=True) }}" search_regex="OpenSSH" delay=60

with inventory without ansible_host at all:
192.168.1.13

Got error again:

TASK [show host] ***************************************************************
ok: [192.168.1.13] => {
    "msg": "192.168.1.13"
}

TASK [Reboot] ******************************************************************
changed: [192.168.1.13]

TASK [Wait for server's sshd to start] *****************************************
fatal: [192.168.1.13 -> localhost]: FAILED! => {"changed": false, "elapsed": 120, "failed": true, "msg": "Timeout when waiting for search string OpenSSH in localhost:22"}

So wait_for seems to work only with strictly declared ansible_host but not with any other variable.

@edrzmr
Copy link

edrzmr commented Aug 19, 2016

this really works for me.

ansible: 2.1.1.0

@vinaykonanki
Copy link

If you use same variable name (ansible_ssh_host or ansible_host) in both inventory and module then it is working fine or else wait_for will take it as localhost.

I tested with both ansible 2.1.1.0 and 2.1.0.0.

@shaiksulaiman
Copy link

I set the exact host IP / hostname to wait_for task to avoid confusion.

My snippet below. Hope it helps someone

  • name: Set facts
    set_fact:
    node_ip: "{{ ansible_host | default(inventory_hostname) }}"

  • name: Print node IP
    debug:
    msg: "node ip is {{ node_ip }}"

  • name: Verify the Node reachability
    wait_for:
    port: 893
    #host: "{{ ansible_host | default(inventory_hostname) }}"
    host: "{{ node_ip }}"
    delay: 90
    timeout: 300
    delegate_to: localhost

@jlebon
Copy link

jlebon commented Feb 3, 2017

The tricky part here is that inside a wait_for task, the ansible_host variable becomes localhost, which is most definitely not what you want (you can verify this by running with -vvv). What we ended up doing was saving the value of ansible_host beforehand and using that:

- set_fact:
    real_ansible_host: "{{ ansible_host }}"

- name: restart hosts
  shell: sleep 3 && shutdown -r now
  async: 1
  poll: 0
  ignore_errors: true

- name: wait for hosts to go down
  local_action:
    wait_for host={{ real_ansible_host }}
    port=22 state=absent delay=1 timeout=120
  become: false

- name: wait for hosts to come back up
  local_action:
    wait_for host={{ real_ansible_host }}
    port=22 state=started delay=30 timeout=120
  become: false

@chrowe
Copy link

chrowe commented Feb 12, 2017

Looks like #21269 was filed for the above issue.

@revmischa
Copy link

I'm getting this issue too on EC2

@ansible ansible locked and limited conversation to collaborators Apr 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants