Openstack inventory plugin errors due to group special characters

[AlanCoding]

SUMMARY

If using the openstack inventory plugin with the setting to transform invalid characters set to "always", then it errors for me.

This probably only happens when you have tags or groups or something with periods or dashes in them in your openstack infra.

ISSUE TYPE

• Bug Report

COMPONENT NAME

openstack inventory plugin

ANSIBLE VERSION

ansible --version
ansible 2.8.0.dev0
  config file = None
  configured module search path = [u'/Users/alancoding/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/alancoding/Documents/repos/ansible/lib/ansible
  executable location = /Users/alancoding/.virtualenvs/ansible/bin/ansible
  python version = 2.7.11 (default, Oct 17 2016, 14:59:40) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]

CONFIGURATION

ANSIBLE_JINJA2_NATIVE=true ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=always ansible-config dump --only-changed
DEFAULT_JINJA2_NATIVE(env: ANSIBLE_JINJA2_NATIVE) = True
TRANSFORM_INVALID_GROUP_CHARS(env: ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS) = always

OS / ENVIRONMENT

mac, openstack instance running

STEPS TO REPRODUCE

ANSIBLE_JINJA2_NATIVE=true ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=always ansible-inventory -i private/openstack/openstack.yml --list --export -vvv

EXPECTED RESULTS

Gives the inventory structure

ACTUAL RESULTS

Parsing error

nstack.yml --list --export -vvv
ansible-inventory 2.8.0.dev0
  config file = None
  configured module search path = [u'/Users/alancoding/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/alancoding/Documents/repos/ansible/lib/ansible
  executable location = /Users/alancoding/.virtualenvs/ansible/bin/ansible-inventory
  python version = 2.7.11 (default, Oct 17 2016, 14:59:40) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]
No config file found; using defaults
host_list declined parsing /Users/alancoding/Documents/repos/ansible-inventory-file-examples/private/openstack/openstack.yml as it did not pass it's verify_file() method
script declined parsing /Users/alancoding/Documents/repos/ansible-inventory-file-examples/private/openstack/openstack.yml as it did not pass it's verify_file() method
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
Keystone catalog entry not found (service_type=network,service_name=Noneinterface=public,region_name=)
[DEPRECATION WARNING]: InventoryModule should utilize self._cache as a dict instead of self.cache. To set the self._cache dictionary, use self._cache[key] = value instead of 
self.cache.set(key, value). To force update the underlying cache plugin with the contents of self._cache before parse() is complete, call self.set_cache_plugin and it will use the 
self._cache dictionary to update the cache plugin. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in 
ansible.cfg.
 [WARNING]: Invalid characters were found in group names and automatically replaced, use -vvvv to see details

toml declined parsing /Users/alancoding/Documents/repos/ansible-inventory-file-examples/private/openstack/openstack.yml as it did not pass it's verify_file() method
 [WARNING]:  * Failed to parse /Users/alancoding/Documents/repos/ansible-inventory-file-examples/private/openstack/openstack.yml with auto plugin: instance-
acfd0343-2fe6-4597-a209-121c18cc9c49 is not a known group

  File "/Users/alancoding/Documents/repos/ansible/lib/ansible/inventory/manager.py", line 272, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/Users/alancoding/Documents/repos/ansible/lib/ansible/plugins/inventory/auto.py", line 58, in parse
    plugin.parse(inventory, loader, path, cache=cache)
  File "/Users/alancoding/Documents/repos/ansible/lib/ansible/plugins/inventory/openstack.py", line 205, in parse
    self._populate_from_source(source_data)
  File "/Users/alancoding/Documents/repos/ansible/lib/ansible/plugins/inventory/openstack.py", line 237, in _populate_from_source
    self._set_variables(hostvars, groups)
  File "/Users/alancoding/Documents/repos/ansible/lib/ansible/plugins/inventory/openstack.py", line 263, in _set_variables
    self.inventory.add_child(group_name, host)
  File "/Users/alancoding/Documents/repos/ansible/lib/ansible/inventory/data.py", line 270, in add_child
    raise AnsibleError("%s is not a known group" % group)

@bcoca you're probably going to save the result of add_group here and then pass what you get to add_child

ansible/lib/ansible/plugins/inventory/openstack.py

Lines 260 to 263 in a8eb25a

	for group_name, group_hosts in groups.items():
	self.inventory.add_group(group_name)
	for host in group_hosts:
	self.inventory.add_child(group_name, host)

I would say that you could have add_child transform the group name, but it was probably your plan to have errors like this happen so that you find all the places to fix.

[ansibot]

Files identified in the description:

• lib/ansible/plugins/inventory/openstack.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @Shrews @cloudnull @dagnello @emonty @evrardjp @juliakreger @mnaser @odyssey4me @rcarrillocruz
click here for bot help

[kdelee]

+1 I ran into this as well

[AlanCoding]

this is also a problem for the tower inventory plugin

[ansibot]

cc @gtema
click here for bot help

[jctanner]

resolved_by_pr #53778