Ansible 2.7 breaks include_tasks in handlers

[gliech]

SUMMARY

Using the include_tasks module inside a handler results in an unexpected exception in Ansible 2.7 . The same code runs as expected in Ansible 2.6.5 .

ISSUE TYPE

• Bug Report

COMPONENT NAME

include_tasks

ANSIBLE VERSION

ansible 2.7.0
  config file = None
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/vagrant/.local/lib/python2.7/site-packages/ansible
  executable location = /home/vagrant/.local/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]

CONFIGURATION

OS / ENVIRONMENT

The problem was first identified in a role that updates a VM Hypervisor on Scientific Linux 7 hosts, where include_tasks was used inside a handler to loop over a list of VMs as part of the cleanup after an update.

The problem was confirmed using the minimal example that is provided below on Scientific Linux 7 and on Ubuntu 18.04 running in a bento/ubuntu-18.04 vagrant box.

STEPS TO REPRODUCE

Just run ansible-playbook playbook.yml under Ansible 2.7 in a directory where the following two files are present. Alternatively you can run this Vagrantfile to reproduce the issue.

playbook.yml

---
- hosts: localhost
  tasks:
    - name: include a task from the tasks section
      include_tasks: include.yml

    - name: notify a handler
      debug:
        msg: bar
      changed_when: yes
      notify: include a task from the handlers section

  handlers:
    - name: include a task from the handlers section
      include_tasks: include.yml

include.yml

---
- name: an included task
  debug:
    msg: foo

EXPECTED RESULTS

The play should conclude without errors. The following ist the verbatim output of the minimal example when run with Ansible 2.6.5

 [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'


PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [include a task from the tasks section] ***********************************
included: /vagrant/include.yml for localhost

TASK [an included task] ********************************************************
ok: [localhost] => {
    "msg": "foo"
}

TASK [notify a handler] ********************************************************
changed: [localhost] => {
    "msg": "bar"
}

RUNNING HANDLER [include a task from the handlers section] *********************
included: /vagrant/include.yml for localhost

RUNNING HANDLER [an included task] *********************************************
ok: [localhost] => {
    "msg": "foo"
}

PLAY RECAP *********************************************************************
localhost                  : ok=6    changed=1    unreachable=0    failed=0

ACTUAL RESULTS

The play run fails for a host after a handler is executed that contains an include_tasks module.

ansible-playbook 2.7.0
  config file = None
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/vagrant/.local/lib/python2.7/site-packages/ansible
  executable location = /home/vagrant/.local/bin/ansible-playbook
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]
No config file found; using defaults
setting up inventory plugins
 [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'

Loading callback plugin default of type stdout, v2.0 from /home/vagrant/.local/lib/python2.7/site-packages/ansible/plugins/callback/default.pyc

PLAYBOOK: playbook.yml *********************************************************
1 plays in playbook.yml

PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************
task path: /vagrant/playbook.yml:2
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: vagrant
<127.0.0.1> EXEC /bin/sh -c 'echo ~vagrant && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/vagrant/.ansible/tmp/ansible-tmp-1539864630.05-54884794442752 `" && echo ansible-tmp-1539864630.05-54884794442752="` echo /home/vagrant/.ansible/tmp/ansible-tmp-1539864630.05-54884794442752 `" ) && sleep 0'
Using module file /home/vagrant/.local/lib/python2.7/site-packages/ansible/modules/system/setup.py
<127.0.0.1> PUT /home/vagrant/.ansible/tmp/ansible-local-31683hqxKea/tmpf_hFqb TO /home/vagrant/.ansible/tmp/ansible-tmp-1539864630.05-54884794442752/AnsiballZ_setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/vagrant/.ansible/tmp/ansible-tmp-1539864630.05-54884794442752/ /home/vagrant/.ansible/tmp/ansible-tmp-1539864630.05-54884794442752/AnsiballZ_setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1539864630.05-54884794442752/AnsiballZ_setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/vagrant/.ansible/tmp/ansible-tmp-1539864630.05-54884794442752/ > /dev/null 2>&1 && sleep 0'
ok: [localhost]
META: ran handlers

TASK [include a task from the tasks section] ***********************************
task path: /vagrant/playbook.yml:4
included: /vagrant/include.yml for localhost

TASK [an included task] ********************************************************
task path: /vagrant/include.yml:2
ok: [localhost] => {
    "msg": "foo"
}

TASK [notify a handler] ********************************************************
task path: /vagrant/playbook.yml:7
NOTIFIED HANDLER include a task from the handlers section for localhost
changed: [localhost] => {
    "msg": "bar"
}

RUNNING HANDLER [include a task from the handlers section] *********************
task path: /vagrant/playbook.yml:14
included: /vagrant/include.yml for localhost

RUNNING HANDLER [an included task] *********************************************
task path: /vagrant/include.yml:2
ok: [localhost] => {
    "msg": "foo"
}
ERROR! Unexpected Exception, this is probably a bug: '08002781-145e-e30a-ccc8-000000000031'
the full traceback was:

Traceback (most recent call last):
  File "/home/vagrant/.local/bin/ansible-playbook", line 118, in <module>
    exit_code = cli.run()
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/cli/playbook.py", line 122, in run
    results = pbex.run()
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/executor/playbook_executor.py", line 156, in run
    result = self._tqm.run(play=play)
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 291, in run
    play_return = strategy.run(iterator, play_context)
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/plugins/strategy/linear.py", line 267, in run
    results.extend(self._execute_meta(task, play_context, iterator, host))
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/plugins/strategy/__init__.py", line 1030, in _execute_meta
    self.run_handlers(iterator, play_context)
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/plugins/strategy/__init__.py", line 884, in run_handlers
    result = self._do_handler_run(handler, handler_name, iterator=iterator, play_context=play_context)
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/plugins/strategy/__init__.py", line 955, in _do_handler_run
    notified_hosts=included_file._hosts[:],
  File "/home/vagrant/.local/lib/python2.7/site-packages/ansible/plugins/strategy/__init__.py", line 968, in _do_handler_run
    h for h in self._notified_handlers[handler._uuid]
KeyError: '08002781-145e-e30a-ccc8-000000000031'

[ansibot]

Hi @gliech, thank you for submitting this issue!

click here for bot help

[ansibot]

Files identified in the description:

• lib/ansible/modules/utilities/logic/include_tasks.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[sivel]

This bug appears to have been introduced by 5f953d1

[sivel]

cc @pablorf-dev

[sivel]

Specifically the issue appears to be with the following lines:

ansible/lib/ansible/plugins/strategy/__init__.py

Lines 962 to 964 in 5f953d1

	self._notified_handlers[handler._uuid] = [
	h for h in self._notified_handlers[handler._uuid]
	if h not in notified_hosts]

Changing that back to the following resolves the problem, but likely breaks what the commit fixed:

self._notified_handlers[handler._uuid] = []

[Asgoret]

@sivel doesn't work in 2.7.7

Ansible version:

ansible 2.7.7
  config file = None
  configured module search path = [u'/Users/{user}/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Library/Python/2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.10 (default, Aug 17 2018, 19:45:58) [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.0.42)]

Code:

  - name: Deploy configmap
    k8s:
      merge_type: merge
      verify_ssl: "{{ verify_ssl }}"
      api_key: "{{ deploy_token }}"
      host: "{{ master_api }}"
      state: "{{ state }}"
      definition: "{{ lookup ('template', 'configmap.yaml.j2')}}" 
    notify:
      - main.yml

Handler:

  - name: Include notify
    include:
      - name: rollout.yml

Error:

ERROR! unexpected parameter type in action: <class 'ansible.parsing.yaml.objects.AnsibleSequence'>

The error appears to have been in '/Users/{user}/Documents/Bitucket/{repo}/roles/deploy/handlers/main.yml': line 2, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
  - name: Include notify
    ^ here