Molecule doesn't respect ANSIBLE_ROLES_PATH or ANSIBLE_COLLECTIONS_PATH

[nre-ableton]

Molecule calls ansible-galaxy <role/collection> install ..., but it doesn't pass along values for either ANSIBLE_ROLES_PATH or ANSIBLE_COLLECTIONS_PATH when doing so. As a result, roles/collections are always installed to ~/.cache/ansible-compat/<hash>.

Furthermore, in molecule 4.0.0 (specifically in #3514), the behavior was also changed to insert this cache dir ahead of any other entries. As such, when molecule runs, it looks something like this:

$ export ANSIBLE_ROLES_PATH=/caches/ansible/roles
$ export ANSIBLE_COLLECTIONS_PATH=/caches/ansible/collections
$ molecule converge
INFO     default scenario test matrix: dependency, create, prepare, converge
INFO     Performing prerun with role_name_check=0...
INFO     Running ansible-galaxy role install -vr requirements.yml --roles-path /home/nre/.cache/ansible-compat/ff4dc6/roles
INFO     Running ansible-galaxy collection install -vr requirements.yml -p /home/nre/.cache/ansible-compat/ff4dc6/collections
INFO     Set ANSIBLE_LIBRARY=/home/nre/.cache/ansible-compat/ff4dc6/modules:/home/nre/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/home/nre/.cache/ansible-compat/ff4dc6/collections:/caches/ansible/collections:/caches/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/home/nre/.cache/ansible-compat/ff4dc6/roles:/caches/ansible/roles:/caches/ansible/roles

This behavior causes trouble on our CI systems where we have a separate partition for caches and build all projects under the same directory name (/jenkins/workspace), which always hashes to the same value. We could possibly work around the first issue by calling ansible-galaxy ... install manually, but molecule doesn't look at data in these custom paths, so it doesn't work.

[zephyros-dev]

I wonder if this is why molecule always reinstall the dependency even after I specify the environment variables.

[zhan9san]

@nre-ableton @zephyros-dev

Could you help submit a PR to fix this issue? I can help review it.

I wonder if this is why molecule always reinstall the dependency even after I specify the environment variables.

Exactly.

Many user complain that molecule introduced conflicts with the existing roles or collections.
So we decided to make the dependencies installed by molecule completely isolated from them.

The order of ANSIBLE_ROLES_PATH environment variable matters, so we insert ~/.cache/ansible-compat/[AN-HASH-VALUE]/roles into the first position when we run molecule, which will re-set the ANSIBLE_ROLES_PATH environment variable. Each time we use molecule, the dependencies will always be installed into this folder, even ANSIBLE_ROLES_PATH environment variable is set explicitly.

[nre-ableton]

Could you help submit a PR to fix this issue? I can help review it.

I've only briefly looked at the code, and the parts that handle role/collection paths is indeed quite complex. I am not sure I'd be able to make a PR easily, but I can try to take a shot.

[nre-ableton]

@zhan9san please have a look at the draft PR #3681 when you have a bit of time. I've tested it locally and it seems to work fine. I'm not sure how to add tests, but it seems that there aren't tests for ANSIBLE_ROLES_PATH either. 🤷‍♀️