Handlers delegated to a list of items fails when "with_items" is provided with a fact

[akatch]

Issue Type:

• Bug Report

Ansible Version:

ansible 2.0.1.0
  config file = /home/bwlz/.ansible.cfg
  configured module search path = Default w/o overrides

Ansible Configuration:

N/A

Environment:

N/A

Summary:

A common workflow is to delegate a task to a list of hosts, like so:

- name: do stuff on a bunch of hosts that aren't in the play
  debug:
    msg: 'We are database hosts!'
  delegate_to: '{{ item }}'
  with_items: '{{ groups.database }}'

In 1.x, this works in both regular tasks and in handlers. In 2.x, tasks work fine but handlers fail with (in this example) "ERROR! groups is undefined". This seems to be the case if any "fact" is used in with_items. It does not happen when a variable that is not a fact is used. It also does not fail if the fact is set in delegate_to or if the fact is set in with_items without being delegated - failure occurs only when the fact is supplied to a with_items in the same action as a delegate_to. It does not matter whether {{ item }} is referenced by delegate_to or some other part of the action.

Steps To Reproduce:

1. Run a playbook with a handler that includes a delegate_to and a with_items, and supply with_items with any fact.
2. The play will fail when it tries to source the handler (this happens right away in a regular playbook, but if the handler is in a role, then it won't happen until the role is referenced).

This playbook will demonstrate the issue - please see the comments for details.

---
- hosts: localhost
  connection: local

  vars:
    # If host_list is defined here, the play succeeds
    host_list:
      - localhost

  tasks:
  # If host_list is instead set here, the play fails. If it is set in *both* places, the play succeeds.
  #- set_fact:
  #   host_list: [ "localhost" ]

  - debug:
      msg: '{{ ansible_fqdn }}'
    changed_when: true
    notify: thing

  handlers:
  - name: thing
    debug:
      msg: ping
    delegate_to: '{{ item }}'
    # This with_items line will run successfully *unless* we set host_list using set_fact.
    with_items: '{{ host_list }}'
    # Uncommenting either of these lines (and commenting the one above) will fail immediately
    #with_items: '{{ ansible_fqdn }}'
    #with_items: '{{ groups.all }}'

Expected Results:

I would expect to see the handler run on each of the delegated hosts.

Actual Results:

The play fails when the handler is sourced (immediately if it's a regular playbook, or when the role is called if the handler is in a role) with the error: "ERROR! [some_fact] is undefined" (with whatever fact was used in with_items):

PLAY ***************************************************************************
ERROR! 'groups' is undefined

[akatch]

Made some edits for clarity.

[ktosiek]

This happens when trying to run flush_handlers after pre_tasks. I've added a traceback.print_exc() before the sys.exit to get this traceback:

Traceback (most recent call last):
  File "/home/tkontusz/ansible/bin/ansible-playbook", line 86, in <module>
    sys.exit(cli.run())
  File "/home/tkontusz/ansible/lib/ansible/cli/playbook.py", line 150, in run
    results = pbex.run()
  File "/home/tkontusz/ansible/lib/ansible/executor/playbook_executor.py", line 141, in run
    result = self._tqm.run(play=play)
  File "/home/tkontusz/ansible/lib/ansible/executor/task_queue_manager.py", line 237, in run
    play_return = strategy.run(iterator, play_context)
  File "/home/tkontusz/ansible/lib/ansible/plugins/strategy/linear.py", line 215, in run
    self._execute_meta(task, play_context, iterator)
  File "/home/tkontusz/ansible/lib/ansible/plugins/strategy/__init__.py", line 653, in _execute_meta
    self.run_handlers(iterator, play_context)
  File "/home/tkontusz/ansible/lib/ansible/plugins/strategy/__init__.py", line 517, in run_handlers
    handler_vars = self._variable_manager.get_vars(loader=self._loader, play=iterator._play, task=handler)
  File "/home/tkontusz/ansible/lib/ansible/vars/__init__.py", line 343, in get_vars
    all_vars['ansible_delegated_vars'] = self._get_delegated_vars(loader, play, task, all_vars)
  File "/home/tkontusz/ansible/lib/ansible/vars/__init__.py", line 412, in _get_delegated_vars
    loop_terms = listify_lookup_plugin_terms(terms=task.loop_args, templar=templar, loader=loader, fail_on_undefined=True, convert_bare=True)
  File "/home/tkontusz/ansible/lib/ansible/utils/listify.py", line 35, in listify_lookup_plugin_terms
    terms = templar.template(terms.strip(), convert_bare=convert_bare, fail_on_undefined=fail_on_undefined)
  File "/home/tkontusz/ansible/lib/ansible/template/__init__.py", line 331, in template
    result = self._do_template(variable, preserve_trailing_newlines=preserve_trailing_newlines, escape_backslashes=escape_backslashes, fail_on_undefined=fail_on_undefined, overrides=overrides)
  File "/home/tkontusz/ansible/lib/ansible/template/__init__.py", line 525, in _do_template
    raise AnsibleUndefinedVariable(e)
AnsibleUndefinedVariable: 'groups' is undefined

[ktosiek]

This seems to fix it:

diff --git a/lib/ansible/vars/__init__.py b/lib/ansible/vars/__init__.py
index 26d03f0..fec857c 100644
--- a/lib/ansible/vars/__init__.py
+++ b/lib/ansible/vars/__init__.py
@@ -366,10 +366,11 @@ class VariableManager:
         if host:
             variables['group_names'] = sorted([group.name for group in host.get_groups() if group.name != 'all'])

-            if self._inventory is not None:
-                variables['groups']  = dict()
-                for (group_name, group) in iteritems(self._inventory.groups):
-                    variables['groups'][group_name] = [h.name for h in group.get_hosts()]
+        if self._inventory is not None:
+            variables['groups']  = dict()
+            for (group_name, group) in iteritems(self._inventory.groups):
+                variables['groups'][group_name] = [h.name for h in group.get_hosts()]
+
         if play:
             variables['role_names'] = [r._role_name for r in play.roles]

I'm not sure why groups were only added if the host argument was provided - is that just a mistake, or was it intentional? Should I make a PR?

[ludovic-gasc]

I confirm the bug, I've the same issue in my playbook with Ansible 2.1.0.

@ktosiek I vote to make a pull request for that.

[ifelsefi]

I am having the same issue

ansible 2.1.0.0
  config file = /home/douglas/repos/ansible/ansible.cfg
  configured module search path = Default w/o overrides

/home/douglas/repos/ansible/playbooks
 .
 |-group_vars
 |-host_vars
 |-roles
 |---accounts
 |-----files
 |-----tasks
 |-----templates
 |-----vars
 |---disk_mgmt
 |-----tasks
 |---iptables
 |-----files
 |-----handlers
 |-----tasks
 |-----templates

accounts.yml

- hosts: "{{ groups.vagrant | random }}"
  roles: 
    - accounts
  gather_facts: True

roles/accounts/tasks/main.yml

- include: ldap.yml 
  when: inventory_hostname in groups.ldap

- include: nfs.yml
  when: inventory_hostname in groups.vagrant

- include: email.yml
  when: sendmail is defined

ansible-playbook -i ~/vagrant/ansible/hosts ~/repos/ansible/playbooks/accounts.yml -e user=foo2 -e group=foo --sudo --step

This does not help either in accounts.yml:

accounts.yml:
- hosts: "{{ selected_host }}"
  roles: 
    - accounts
  gather_facts: True

group_vars/all:
selected_hosts: '{{ groups.vagrant | random }}'

machine1 ansible_ssh_host=192.168.122.21 ansible_ssh_private_key_file=/home/douglas/vagrant/.vagrant/machines/machine1/libvirt/private_key
machine2 ansible_ssh_host=192.168.122.22 ansible_ssh_private_key_file=/home/douglas/vagrant/.vagrant/machines/machine2/libvirt/private_key
machine3 ansible_ssh_host=192.168.122.23 ansible_ssh_private_key_file=/home/douglas/vagrant/.vagrant/machines/machine3/libvirt/private_key
machine4 ansible_ssh_host=192.168.122.24 ansible_ssh_private_key_file=/home/douglas/vagrant/.vagrant/machines/machine4/libvirt/private_key 
machine5 ansible_ssh_host=192.168.122.25 ansible_ssh_private_key_file=/home/douglas/vagrant/.vagrant/machines/machine5/libvirt/private_key
machine6 ansible_ssh_host=192.168.122.26 ansible_ssh_private_key_file=/home/douglas/vagrant/.vagrant/machines/machine6/libvirt/private_key

If I run a debug by adding tasks into accounts.yml it does print the group:

- hosts: localhost
  connection: local
  gather_facts: false
   tasks:
    - set_fact: random_host="{{ groups.vagrant | random }}"
    - debug: var=groups.vagrant|random

PLAY [localhost] ***************************************************************
Perform task: TASK: debug (N)o/(y)es/(c)ontinue: y

Perform task: TASK: debug (N)o/(y)es/(c)ontinue: *******************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "groups.vagrant|random": "machine5"
}

[turb]

Note that the workaround proposed in #12170 does not seem to work, blocking us from migrating to Ansible 2.x.

[turb]

After some tests, it seems it only happens for me when a given host is skipped. set_fact is skipped, but with_items is still evaluated for the next task.

https://gist.github.com/turb/278dc6378e9045da1fe5ad278c7bed01

[bcoca]

So i can reproduce this on 2.0.1 but it seems to have been working since 2.0.2 (I'm not going to track down exact commit that fixed it).

I also checked 2.1.3 and 2.2, so I'm going to close this as 'fixed'.

[bcoca]

@turb your issue is unrelated, when does not avoid the evaluation of with as it runs PER item in the loop, as documented http://docs.ansible.com/ansible/playbooks_conditionals.html#loops-and-conditionals

[turb]

@bcoca it did work in 1.9

By all means the when: false is just here to simulate the case when one host is skipped. I would have supposed the whole task to be skipped for this host without be evaluated in such case, as in 1.9.

[bcoca]

@turb it did AND did not 'work' in 1.9, 1.9 was just ignoring the undefined error.

In 2.0 we 'fixed that' and showed the error, but since it broke many people's playbook we changed it from error to a deprecation message. As is our normal procedure with deprecations, we removed it and it is an error again in 2.2.

[turb]

@bcoca ok, will proceed by the doc, thanks.