Undefined variables with three nodes

[ivomarino]

Hi all,
when I launch "ansible-playbook -s -i production/inventory ceph.yml" with three nodes in the inventory I get this error, with two nodes all seems fine, it's quite strange. Any ideas? Thanks.

TASK: [ceph-common | generate Ceph configuration file] ************************
fatal: [node0] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute u'ansible_eth1'", 'failed': True}
fatal: [node1] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute u'ansible_eth1'", 'failed': True}
fatal: [node2] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute u'ansible_bond0'", 'failed': True}

FATAL: all hosts have already failed -- aborting

[leseb]

can you share your variables?

[ivomarino]

node0 and node1:

`## Monitor options

monitor_interface: bond0`

node2:

`## Monitor options

monitor_interface: eth1`

When I do "ansible-playbook -s -i production/inventory ceph.yml --limit node0,node1" all is fine, when I don't specify --limit or add node2 also to the limit option the problem arises, quite strange.

[ivomarino]

In case I check the vars I can see:

ansible all -m setup --tree /tmp/facts -i production/inventory > /tmp/foo

grep bond0 /tmp/foo "ansible_bond0": { "device": "bond0", "bond0", "ansible_bond0": { "device": "bond0", "ansible_bond0_0": { "bond0_0", "bond0",

and

grep eth1 /tmp/foo "ansible_eth1": { "device": "eth1", "eth1",

so this vars should be there.

P.S. In staging I also have three nodes without problems, just in prod (with also three nodes) I have this issue, same code basically, quite strange.

[ivomarino]

The problem is related to the fact the some nodes have bond0 other nodes only eth1, if I test with
mon addr = {{ hostvars[host]['ansible_lo']['ipv4']['address'] }} in ceph.conf.j2 it works as far as loopback is available everywhere, probably something related to ansible/ansible#4848 -- this may also be related: ansible/ansible#9260

[ivomarino]

Solved. On line 83 in ceph.conf.j2 I had to add the following code:

    {% if hostvars[host]['ansible_' + monitor_interface] is defined %}
    mon addr = {{ hostvars[host]['ansible_' + monitor_interface]['ipv4']['address'] }}
    {% endif %}

Now all is fine. The problem is that different machines can have different monitoring interfaces and this must be checked before applying the template.

[leseb]

I'm not sure why this solves your problem.
Happy to hear that you solved this, can you PR your fix?

[ivomarino]

in fact it didn't really solve the problem, in the end I've renamed eth1 to bond0 on node2 in order to be in line with the other nodes, now it works, previous fix not required.

[leseb]

Ok but we should be able to handle such use case where machines are not identical.

[ivomarino]

For sure, at least for now we know this fact, better than nothing.

[snerdish]

We've encountered this as well. We tried to overcome it by using

monitor_interface: "{{ ansible_default_ipv4.interface }}"

but encountered the apparent fact scope problems mentioned in the ansible issues above.

Using host_vars gave us the same errors, so it seems that unless we can rename all our interfaces (unlikely) we're stuck with this.

[leseb]

Hey @snerdish, @ivomarino
Another solution to solve this is to edit your inventory files like this:

[mons]
mon1 monitor_interface=bond0
mon2 monitor_interface=eth0
mon3 monitor_interface=eth1