Speedup inventory breaks include_role over with_items

[willthames]

ISSUE TYPE

• Bug Report

COMPONENT NAME

include_role

ANSIBLE VERSION

ansible 2.5.0 (devel cb5f2c7ac3) last updated 2017/09/26 11:45:15 (GMT +1000)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/will/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/will/src/ansible/lib/ansible
  executable location = /home/will/src/ansible/bin/ansible
  python version = 2.7.13 (default, Sep  5 2017, 08:53:59) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]

CONFIGURATION

N/A

OS / ENVIRONMENT

N/A

SUMMARY

include_role combined with with_items no longer works after the inventory speedup #30587

STEPS TO REPRODUCE

playbook:

- hosts: host1
  connection: local
  gather_facts: false

  tasks:
    - name: apply role whatever over list of items
      include_role:
        name: whatever
      with_items:
        - a

inventory file:

[group4]
host1

[group2]
host2

[group3]
host1
host2

[group3]
host3

[group2]
host3

To see the working version (it's not needed for the failure)

mkdir -p roles/whatever/tasks && echo "- debug: msg=hello" > roles/whatever/tasks/main.yml

EXPECTED RESULTS

With Ansible 2.4.0, I get:

$ ansible-playbook playbook.yml -i inventory

PLAY [host1] *************************************************************************************************************************************************

TASK [apply role whatever over list of items] ****************************************************************************************************************

TASK [whatever : debug] **************************************************************************************************************************************
ok: [host1] => {
    "msg": "hello"
}

PLAY RECAP ***************************************************************************************************************************************************
host1                      : ok=1    changed=0    unreachable=0    failed=0   

ACTUAL RESULTS

$ ansible-playbook playbook.yml -i inventory

PLAY [host1] *************************************************************************************************************************************************

TASK [apply role whatever over list of items] ****************************************************************************************************************
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.7/threading.py", line 757, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/will/src/ansible/lib/ansible/plugins/strategy/__init__.py", line 84, in results_thread_main
    result = strategy._final_q.get()
  File "/usr/lib64/python2.7/multiprocessing/queues.py", line 117, in get
    res = self._recv()
  File "/home/will/src/ansible/lib/ansible/inventory/host.py", line 37, in __setstate__
    return self.deserialize(data)
  File "/home/will/src/ansible/lib/ansible/inventory/host.py", line 82, in deserialize
    g.deserialize(group_data)
  File "/home/will/src/ansible/lib/ansible/inventory/group.py", line 77, in deserialize
    self._hosts = set(self.hosts)
  File "/home/will/src/ansible/lib/ansible/inventory/host.py", line 48, in __hash__
    return hash(self.name)
AttributeError: 'Host' object has no attribute 'name'

At which point the playbook just hangs.

[bcoca]

... wrong deserialization, should be host names ..

[mattclay]

@willthames Thanks for the repro, it helped me isolate the issue faster than running it through CI.

The timeout issue we're seeing in CI is intermittent, and only on Python 3, but this repro triggers it and yields the same error. It occurs while deserializing a Group which has multiple Host references.

During deserialization of Group, it tries to add all the Host objects to a set. However, not all of the Host objects have been deserialized yet, so they are missing attributes, causing the AttributeError.

[mattclay]

@willthames Can you test the fix in #31033 to see if it resolves your issue?

[bcoca]

@willthames can you test that #31103 fixes it for you? nvmd .. 2.6 doesn't support fix, try the one below

[olfway]

#31111 fixes #31093 for me on stable-2.4 and devel