Better defined namespace handling

[opoplawski]

SUMMARY

The use of collections and adding collection names to the module search path creates the possibility of module name collisions between collections and core and between various collections. It would be good to define the expected behavior as much as possible. It may also be required to have a namespace for the core modules (if there isn't one already) and have ansible's core module/configuration refer to those FQCNs to avoid using versions from collections.

I'm likely going to rename the pfsensible.core modules to not conflict with core modules, but I still think this is worth documenting and possibly fixing.

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansiballz

ANSIBLE VERSION

ansible 2.10.0.dev0
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/orion/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/orion/ansible/lib/ansible
  executable location = /home/orion/ansible/bin/ansible
  python version = 3.8.2 (default, Feb 28 2020, 00:00:00) [GCC 10.0.1 20200216 (Red Hat 10.0.1-0.8)]

CONFIGURATION

DEFAULT_HOST_LIST(/home/orion/ansible-pfsense/ansible.cfg) = ['/home/orion/ansible-pfsense/hosts']
DEFAULT_REMOTE_USER(/home/orion/ansible-pfsense/ansible.cfg) = root

OS / ENVIRONMENT

Fedora Rawhide

STEPS TO REPRODUCE

Install the pfsensible.core collection:

ansible-galaxy collection install pfsensible.core

Run this play:

---
- hosts: pfsense
  collections:
    - pfsensible.core
  gather_facts: yes
  tasks:
    - name: "Set admin user password"
      user:
        name: admin
        scope: system
        password: "{{ 'changeme' | password_hash('bcrypt') }}"

EXPECTED RESULTS

ansible collects system facts with the core setup module, then runs the pfsensible.core user module.

ACTUAL RESULTS

ansiballz ships and runs the pfsensible.core setup module. Users has not yet been tested.

$ ansible-playbook -vvv setup.yml
[WARNING]: You are running the development version of Ansible. You should only run Ansible from "devel" if you are modifying the Ansible engine, or trying out features
under development. This is a rapidly changing source of code and can become unstable at any point.
ansible-playbook 2.10.0.dev0
  config file = /home/orion/ansible-pfsense/ansible.cfg
  configured module search path = ['/home/orion/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/orion/ansible/lib/ansible
  executable location = /home/orion/ansible/bin/ansible-playbook
  python version = 3.8.2 (default, Feb 28 2020, 00:00:00) [GCC 10.0.1 20200216 (Red Hat 10.0.1-0.8)]
Using /home/orion/ansible-pfsense/ansible.cfg as config file
host_list declined parsing /home/orion/ansible-pfsense/hosts as it did not pass its verify_file() method
script declined parsing /home/orion/ansible-pfsense/hosts as it did not pass its verify_file() method
auto declined parsing /home/orion/ansible-pfsense/hosts as it did not pass its verify_file() method
Parsed /home/orion/ansible-pfsense/hosts inventory source with ini plugin

PLAYBOOK: setup.yml ********************************************************************************************************************************************************
1 plays in setup.yml

PLAY [pfsense] *************************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************
task path: /home/orion/ansible-pfsense/setup.yml:2
<pfsense-test> ESTABLISH SSH CONNECTION FOR USER: root
<pfsense-test> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/orion/.ansible/cp/53186e92e3 pfsense-test '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''                                                                                                                                                          
<pfsense-test> (0, b'/root\n', b"Warning: Permanently added the ED25519 host key for IP address '192.168.100.2' to the list of known hosts.\r\n")
<pfsense-test> ESTABLISH SSH CONNECTION FOR USER: root
<pfsense-test> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/orion/.ansible/cp/53186e92e3 pfsense-test '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346 && echo ansible-tmp-1587953727.7538044-11286-118086989917346="` echo /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346 `" ) && sleep 0'"'"''                                                          
<pfsense-test> (0, b'ansible-tmp-1587953727.7538044-11286-118086989917346=/root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346\n', b'')
<pfsense-test> Attempting python interpreter discovery
<pfsense-test> ESTABLISH SSH CONNECTION FOR USER: root
<pfsense-test> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/orion/.ansible/cp/53186e92e3 pfsense-test '/bin/sh -c '"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'/usr/bin/python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.5'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/libexec/platform-python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"''                                                                                                    
<pfsense-test> (0, b'PLATFORM\nFreeBSD\nFOUND\n/usr/local/bin/python3.7\nENDFOUND\n', b'')
<pfsense-test> Python interpreter discovery fallback (unsupported platform for extended discovery: freebsd)
Using module file /home/orion/.ansible/collections/ansible_collections/pfsensible/core/plugins/modules/setup.py
<pfsense-test> PUT /home/orion/.ansible/tmp/ansible-local-1128237slq1cp/tmpje7zguf7 TO /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346/AnsiballZ_setup.py                                                                                                                                                                       
<pfsense-test> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/orion/.ansible/cp/53186e92e3 '[pfsense-test]'                 
<pfsense-test> (0, b'sftp> put /home/orion/.ansible/tmp/ansible-local-1128237slq1cp/tmpje7zguf7 /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346/AnsiballZ_setup.py\n', b'')                                                                                                                                                     
<pfsense-test> ESTABLISH SSH CONNECTION FOR USER: root
<pfsense-test> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/orion/.ansible/cp/53186e92e3 pfsense-test '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346/ /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346/AnsiballZ_setup.py && sleep 0'"'"''                                                                                                                                                                   
<pfsense-test> (0, b'', b'')
<pfsense-test> ESTABLISH SSH CONNECTION FOR USER: root
<pfsense-test> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/orion/.ansible/cp/53186e92e3 -tt pfsense-test '/bin/sh -c '"'"'/usr/local/bin/python3.7 /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346/AnsiballZ_setup.py && sleep 0'"'"''                                             
<pfsense-test> (1, b'\r\n{"failed": true, "msg": "Unsupported parameters for (setup) module: gather_subset, gather_timeout Supported parameters include: dashboardavailablewidgetspanel, dashboardcolumns, disablealiaspopupdetail, dns_addresses, dns_gateways, dns_hostnames, dnsallowoverride, dnslocalhost, domain, hostname, interfacessort, language, logincss, loginshowhost, requirestatefilter, roworderdragging, statusmonitoringsettingspanel, systemlogsfilterpanel, systemlogsmanagelogpanel, timeservers, timezone, webguicss, webguifixedmenu, webguihostnamemenu, webguileftcolumnhyper", "invocation": {"module_args": {"gather_subset": ["all"], "gather_timeout": 10}}}\r\n', b'Shared connection to pfsense-test closed.\r\n')                                                                                                                                          
<pfsense-test> Failed to connect to the host via ssh: Shared connection to pfsense-test closed.
<pfsense-test> ESTABLISH SSH CONNECTION FOR USER: root
<pfsense-test> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/orion/.ansible/cp/53186e92e3 pfsense-test '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1587953727.7538044-11286-118086989917346/ > /dev/null 2>&1 && sleep 0'"'"''                                                                  
<pfsense-test> (0, b'', b'')
fatal: [pfsense-test]: FAILED! => {
    "ansible_facts": {},
    "changed": false,
    "failed_modules": {
        "setup": {
            "ansible_facts": {
                "discovered_interpreter_python": "/usr/local/bin/python3.7"
            },
            "failed": true,
            "invocation": {
                "module_args": {
                    "gather_subset": [
                        "all"
                    ],
                    "gather_timeout": 10
                }
            },
            "msg": "Unsupported parameters for (setup) module: gather_subset, gather_timeout Supported parameters include: dashboardavailablewidgetspanel, dashboardcolumns, disablealiaspopupdetail, dns_addresses, dns_gateways, dns_hostnames, dnsallowoverride, dnslocalhost, domain, hostname, interfacessort, language, logincss, loginshowhost, requirestatefilter, roworderdragging, statusmonitoringsettingspanel, systemlogsfilterpanel, systemlogsmanagelogpanel, timeservers, timezone, webguicss, webguifixedmenu, webguihostnamemenu, webguileftcolumnhyper",                                                                                                                                     
            "warnings": [
                "Platform freebsd on host pfsense-test is using the discovered Python interpreter at /usr/local/bin/python3.7, but future installation of another Python interpreter could change this. See https://docs.ansible.com/ansible/devel/reference_appendices/interpreter_discovery.html for more information."                               
            ]
        }
    },
    "msg": "The following modules failed to execute: setup\n"
}

PLAY RECAP *****************************************************************************************************************************************************************
pfsense-test               : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

[ansibot]

Files identified in the description:

• lib/ansible/executor/module_common.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[sivel]

The behavior you are seeing above is expected. You have defined collections within your play as the following:

  collections:
    - pfsensible.core

This implicitly becomes:

  collections:
    - pfsensible.core
    - ansible.legacy

Which puts the search order of pfsensible.core before ansible.legacy. As such, if pfsensible.core has a module that shadows the name of one within ansible itself, it will win.

If you want ansible.legacy (or ansible.builtin) to win, you would have to explicitly state that in your collections list, with ansible.legacy being listed with a higher search priority (Above pfsensible.core):

  collections:
    - ansible.legacy
    - pfsensible.core

Because of the this behavior, I've always felt like it was a mistake for collection authors to strip prefixes from their modules, that would cause their modules to shadow internal ansible module names.

I'm also a firm believer, that playbooks should ultimately be written to use the FQCN of the module, and shy away from using collections within a play.

[opoplawski]

Thanks, I pretty much agree with everything above. However, in line with using FQCNs everywhere, it seems appropriate that when ansible implicitly loads the setup module as part of the "Gathering facts" step that it explicitly calls ansible.legacy.setup (or whatever is appropriate for it).

[sivel]

Overriding modules is an acceptable use case.

[jimi-c]

Per the above discussion, we're going to go ahead and close this.

If you have any further questions, please let us know by stopping by one of the two mailing lists, as appropriate:

• https://groups.google.com/forum/#!forum/ansible-project - for user questions, tips, and tricks
• https://groups.google.com/forum/#!forum/ansible-devel - for strategy, future planning, and questions about writing code

Because this project is very active, we're unlikely to see comments made on closed tickets, but the mailing list is a great way to ask questions, or post if you don't think this particular issue is resolved.

Thank you!