Roles can't use collection modules when collections are listed at playbook level

[geerlingguy]

SUMMARY

If I put a task from my playbook which uses a module defined in a collection into a role, it breaks the playbook.

ISSUE TYPE

• Bug Report

COMPONENT NAME

• collections
• roles

ANSIBLE VERSION

ansible 2.9.6
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/Users/jgeerling/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.6 (default, Feb  9 2020, 13:28:08) [Clang 11.0.0 (clang-1100.0.33.17)]

CONFIGURATION

ANSIBLE_NOCOWS(/etc/ansible/ansible.cfg) = True
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 20
DEFAULT_HOST_LIST(/etc/ansible/ansible.cfg) = ['/etc/ansible/hosts']
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False

OS / ENVIRONMENT

macOS Catalina

STEPS TO REPRODUCE

# Create role.
mkdir roles && cd roles
ansible-galaxy init test_role
nano test_role/tasks/main.yml # add contents below 
cd ../

# Install collection (note that the k8s_log module is in pre-release).
ansible-galaxy collection install community.kubernetes

# Create and run playbook.
nano main.yml # add contents below
ansible-playbook main.yml

main.yml contents:

---
- hosts: localhost
  connection: local
  gather_facts: false

  collections:
    - community.kubernetes

  # This task works.
  pre_tasks:
    - name: Create a test namespace (pre_tasks).
      k8s_log:
        name: example-pod
        namespace: testing

  # But the same task doesn't work in a role.
  roles:
    - test_role

test_role/tasks/main.yml contents:

---
- name: Create a test namespace (role).
  k8s_log:
    name: example-pod
    namespace: testing

EXPECTED RESULTS

The playbook should run. The k8s_log task should work the same in the role tasks/main.yml file as it does in the rest of the playbook.

ACTUAL RESULTS

$ ansible-playbook -i 'localhost,' main.yml
ERROR! couldn't resolve module/action 'k8s_log'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/Users/jgeerling/Downloads/test/roles/test_role/tasks/main.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- name: Create a test namespace (role).
  ^ here

[ansibot]

Files identified in the description:

• lib/ansible/galaxy/data/default/collection/roles/.git_keep
• lib/ansible/module_utils/common/collections.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[geerlingguy]

Before someone tells me that I should start writing out the Fully Qualified Collection Name (FQCN) on each and every task I write in all my Ansible playbooks... no. I'm writing YAML, not Java! The collections keyword should work the same for any task in a play, whether that task is in the play, in an included task file, or in a role.

[sivel]

This is expected, please see #66667

[geerlingguy]

Can we reopen if we don't agree this is useful and/or expected behavior?

Or alternatively, can we just drop the collections keyword entirely and force everyone to always type in the FQCN for everything?

Having both coexist == hell once 2.10 is released and people try updating playbooks.

[Edit: Note—I'm a strong proponent of the former, this is far beyond attacking my ability to use hyphens in group names in terms of 'makes Ansible annoying to use'.]

[sivel]

There are no plans to change this behavior.

[geerlingguy]

In that PR's description:

In a playbook, you can control the collections Ansible searches for modules and action plugins to execute. However, any roles you call in your playbook define their own collections search order; they do not inherit the calling playbook's settings.

Why can't a role fall back on the calling playbook's settings? As it is, this is going to make my use of roles a lot more painful. I get adding collections to a role like my geerlingguy.mysql role on Ansible Galaxy, because that way someone using my role without also needing to use community.general's mysql_user module would not have to also add a collections keyword to their playbook accounting for it.

But I use individual roles in single playbooks a lot, and because of this change with collections, I'll have to add collections to both the playbook and role metadata all over the place. In the past, I could copy-and-paste a task from a play or task include file to a role and back, and it would work. Now, that is not the case.

All I'm wondering is why roles can't fall back to the play's collections if the role doesn't define any on its own?

[tolbrino]

I'm experiencing the same issue like @geerlingguy in the case of using roles in playbooks as a way to structure playbooks more clearly. Given the current state of this feature such a setup doesn't reduce the amount of work, instead the work required scales up with the number of roles being factored out.

From a user's perspective I'd like roles to be able to "bring" their own required collections as well as be able to "live" in the playbooks environment using the collections from that setup.

[geerlingguy]

I'm going to keep campaigning for this—using roles without this functionality is incredibly annoying, and yes I know there are workarounds; they all require a new level of verbosity and rework, when my playbooks do not require this.

Additionally, assuming you use Ansible Community Distribution (ACD), which will continue to ship and supposedly work without specifying collections in the top level playbook, this means that some roles may be designed (or grandfathered) to work in ACD playbooks, but the exact same role would no longer work if used with Ansible 'base' / 'core' (or other distributions) which require collections...

This leads to playbooks which will work fine in ACD, but break with any other distribution of Ansible.

[geerlingguy]

One difficulty, I think, is that the term "role"/"roles" can mean a lot of different things depending on context, and some of the usages aren't accounted for in the current plans/migration paths:

1. Role in Ansible Galaxy (which is an independent entity, installable via ansible-galaxy) — seems like it is slated for removal per https://github.com/ansible/galaxy_ng/issues/58#issuecomment-604034699 (though AFAICT this hasn't been announced elsewhere).
2. Role in a Collection in Ansible Galaxy/Automation Hub (new use case, currently no migration path from #1 to #2 until Allow roles to be converted to (or coexist with) collections galaxy#2253 is done).
3. Standalone role in a standalone playbook (e.g. in roles directory, which can be set via config variable ANSIBLE_ROLES_PATH). It doesn't seem like this is going away, but based on my tea-leaf-reading it seems like maybe it will? This would cause a lot of breakage since many (most, IME) playbooks have standalone roles that are packaged alongside them that are not published anywhere.
4. Role in a Collection in a standalone playbook (so far I've never seen anyone do this yet—it's a lot of overhead to build a role inside a collection, when it's way faster/simpler to just do ansible-galaxy init myrole in a playbook roles directory and be off to the races).

It's #3 that I'm most worried about in this particular issue—it becomes harder to build a playbook, then extract groups of tasks/vars/handlers into roles in that same playbook project. Now I have to extract things, then duplicate the collections: information in each role that I extract from the playbook, too (in addition to maybe also building roles inside collections, which means more empty nested folders to navigate in my IDE).

[s-hertel]

To keep thoughts collected in one place, geerlingguy/ansible-content-testing#4 (comment)