Failing to Populate the namenode groups list

[cavemandaveman]

First, I just want to say this is a great collection of playbooks/scripts. Thank you for this.

I am attempting to use updated versions of the ansible and azure python packages. After adding/tweaking the pip dependencies, I was able to get build_cloud to execute successfully (Azure).

I'm currently stuck at the prepare_nodes playbook. It won't get past TASK [Populate the namenode groups list]:

fatal: [host-mgmt.example.com]: FAILED! => {
    "failed": true, 
    "msg": "'dict object' has no attribute 'blueprint_dynamic'"
}

The problem is in playbooks/set_variables.yml line 38. It seems as though the vars in playbooks/group_vars/ambari-server are not being added to the host, and therefore blueprint_dynamic is not a var on the host. I can't past this. Do you have any idea why those vars are not being added?

[alexandruanghel]

Hi!
This is a bit strange, I haven't seen this error during my tests.
I know there have been some changes in Ansible 2.4 so not sure if this would impact (I haven't got the chance to test with 2.4).

The blueprint_dynamic variable is defined in playbooks/group_vars/ambari-server as you noticed and because this is a group variable it will automatically be applied by Ansible to all hosts that are in this ambari-server group.
So maybe your ambari-server group is empty.

This group is created by https://github.com/hortonworks/ansible-hortonworks/blob/master/playbooks/clouds/add_nodes_azure.yml#L18 based on which inventory role you set the ambari_server: true variable in https://github.com/hortonworks/ansible-hortonworks/blob/master/inventory/azure/group_vars/all#L34

But the above add_nodes_azure.yml also depends on how the Azure inventory script returns the grouping of nodes (see an example in line 30 of add_nodes_azure.yml).

Since you've upgrade some library versions, not sure if there was any change that would impact this, there are so many things that could break here.
Can you tell me, which Ansible version and which azure lib version are you currently testing with?
Was there a specific reason why you tried to upgrade?

Also try to add some debug variables before the failed task to see maybe we can spot where is the failure:

- debug: var=groups
- debug: var=hostvars[groups['ambari-server']|sort|list|first]

[cavemandaveman]

One of the reasons I was trying to upgrade is because I was making the Azure security groups creation more robust (multiple groups, multiple rules per group). I ran into a really trivial problem that ansible 2.4 fixed, which was that setting the priority in a rule for a security group would result in it being passed as a string, not integer.

I'm almost certain my issue is the fact that the hosts in the ambari-server group are not picking the needed group_vars. After debugging, I can see that the host has been added to the correct group, but missing all vars from playbooks/group_vars/ambari-server.

I'm trying to figure if this is part of my problem: according to the ansible changelog for 2.4: Loading group_vars/host_vars is now a vars plugin and can be overridden. Also, this issue may be related but not sure how yet.

If you care to test, here are the libraries I've found to work so far, as my requirements.txt:

ansible==2.4.1.0
azure==2.0.0
azure-mgmt-compute==3.0.1
azure-mgmt-containerservice==2.0.0
azure-mgmt-network==1.5.0
azure-mgmt-resource==1.2.2
azure-mgmt-storage==1.4.0
msrest
msrestazure
packaging
pip
pycparser
setuptools

which I run with pip install -r requirements.txt --upgrade

If I can get past this, I hope to open a PR for you so everyone can use the latest versions.

[JackOneil]

Hi,
i'm having exactly the same error while using static configuration
i have modified the file inventory/static to refer the servers.
Are there group_vars as for e.g. in inventory/aws/group_vars/all for static configuration ?
Thanks

[fossouo]

Hello,
That is a gather_facts issue :

Try adding this in the yaml of the playbook :

• name: Gather facts from all nodes
  hosts: all
  any_errors_fatal: true
  become: yes

[alexandruanghel]

Hi there, I've finally managed to update these playbooks to be in line with the latest Ansible 2.4 and 2.5, including the support for the latest Azure python bindings: #25

Hopefully, this should fix the issues reported here.