5.0.0: playbooks may fail to find current standalone galaxy role

[geerlingguy]

Summary

Since upgrading to 5.0.0, none of my roles seem to be able to complete their CI tests.

I have modified my pip3 install process in CI to also install either ansible-base or ansible, but even with that, I'm getting the error message:

WARNING  Listing 1 violation(s) that are fatal
internal-error: the role 'geerlingguy.mysql' was not found in /home/runner/work/ansible-role-mysql/ansible-role-mysql/geerlingguy.mysql/molecule/default/roles:/home/runner/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/runner/work/ansible-role-mysql/ansible-role-mysql/geerlingguy.mysql/molecule/default
WARNING  Replaced deprecated tag '106' with 'role-name' but it will become an error in the future.
Warning: default/converge.yml:7:7 [WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'
ERROR! the role 'geerlingguy.mysql' was not found in /home/runner/work/ansible-role-mysql/ansible-role-mysql/geerlingguy.mysql/molecule/default/roles:/home/runner/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/runner/work/ansible-role-mysql/ansible-role-mysql/geerlingguy.mysql/molecule/default

The error appears to be in '/home/runner/work/ansible-role-mysql/ansible-role-mysql/geerlingguy.mysql/molecule/default/converge.yml': line 7, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - role: geerlingguy.mysql
      ^ here


Error: internal-error the role 'geerlingguy.mysql' was not found in /home/runner/work/ansible-role-mysql/ansible-role-mysql/geerlingguy.mysql/molecule/default/roles:/home/runner/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/runner/work/ansible-role-mysql/ansible-role-mysql/geerlingguy.mysql/molecule/default
You can skip specific rules or tags by adding them to your configuration file:
# .ansible-lint
warn_list:  # or 'skip_list' to silence them completely
  - internal-error  # Unexpected internal error
Finished with 1 failure(s), 0 warning(s) on 24 files.

Issue Type

• Bug Report

Ansible and Ansible Lint details

(Latest as installed via pip3 in CI environment)

OS / ENVIRONMENT

GitHub Actions

STEPS TO REPRODUCE

See: https://github.com/geerlingguy/ansible-role-mysql/runs/1868407331?check_suite_focus=true#step:5:18

Desired Behaviour

ansible-lint should not require any special modifications to my role to make it lint-able?

Actual Behaviour

My roles are not able to be linted.

[ssbarnea]

That is because you are inside a standalone galaxy role repository which you did not package and install, so Ansible itself fails to find the role.

The linter now relies on ansible-playbook --syntax-check results.

For collections this was sorted as we detect their presence and install them in a special location before running.

I see that not as a regression but more of a side effect of doing a better testing.

[smutel]

Any workaround for this ? Sometimes we want to execute the lint on the playbook but not on the roles used by this playbook. In particular when the roles are developed by others ...

I tried the options below without any success ...
ansible-lint --exclude ./roles/ --offline <playbook>

[sio]

I'm encountering the same issue with ansible-lint 5.0.0, it fails to find roles mentioned in my molecule/default/playbook.yml

I do not use Galaxy, directory structure is simple roles/rolename/{molecule,defaults,tasks,handlers}/ and all roles are locally available. I understand that Molecule does some internal tricks with ANSIBLE_ROLES_PATH which ansible-lint is unaware of. Since playbook.yml/converge.yml is never meant to be a usable playbook outside of Molecule, it is ok for ansible-playbook --check to fail it, but I think it's not ok for ansible-lint to fail the whole roles/ directory if it contains some Molecule tests.

[ssbarnea]

@sio You are right, molecule does some magic with paths order to help ansible find not-installed-yet roles. Ansible-lint does something too, but not the same, I will try to address this case for the next patch.

[ssbarnea]

@geerlingguy Here is a question that you should answer yourself first: how do you expect this role to be consumed? Is this a galaxy standalone role or something else?

If is a standalone role we do expect it to be installed as a prefix.name folder by ansible-galaxy role install. Still, the prefix value is nowhere to be found inside the repository itself. The author field from inside galaxy_info happens to be your github/galaxy organization, but that only by chance.

If you read https://galaxy.ansible.com/docs/contributing/creating_role.html#role-metadata you can find no reference of using the author field for mentioning the prefix/name. In fact the example include "your name" which is clearly not a valid value for this.

This means that given a cloned role repository, we cannot infer the prefix to be added to the role but we do know for sure that in production the role would end-up being installed with a prefix, if installed from galaxy.

I do expect molecule playbooks to make use of full role import name, as they are supposed to match production usage, but we cannot determine that name.

One workaround that you could use would be to add a symlink roles/geerlingguy.mysql -> ../.., which will likely enable detection of the geerlingguy.mysql role, as the linter and also molecule both add roles/ folder to ansible role path when present.

In the absence of any feedback from Galaxy team, I am afraid I cannot do anything to remediate this, because the problem reported is purely related to galaxy-roles.

[decentral1se]

Also bit by this, strange that Ansible-lint now is looking for roles when run under Molecule...

[ssbarnea]

Also bit by this, strange that Ansible-lint now is looking for roles when run under Molecule...

Not really strange, is expected and caused by use of ansible syntax check. You either assure you have the roles or mock them (see mock_roles option). Mentioning roles dependencies inside a requirements.yml is another way to assure they are installed (in an isolated location, not the common ansible path).

[decentral1se]

Yeah it is strange because workflows where you don't expect the roles to be present are now broken. I don't want to edit 20 role molecule configurations to download the role itself or wire up a mocked role configuration for so many roles. For now, I am just running a cp of the role folder into a .ansible/roles which is just a weird hack. Making ansible-lint now implicity depend on another tools job is not ideal I would say. This error could catch out a lot of beginners.

[geerlingguy]

@ssbarnea - It seems like the fix isn't working, at least in the role I just tested it under: https://github.com/geerlingguy/ansible-role-mysql/runs/1922461996#step:5:17

[sio]

Same here, ansible-lint from git master can not find the role mentioned in molecule/playbook.yml

CI log: https://gitlab.com/sio/ci-with-molecule-git/-/jobs/1038564970#L430 (see line 430)
Repo: https://github.com/sio/homelab/blob/master/ansible/roles/canary/molecule/default/playbook.yml

[bbaassssiiee]

I use ansible-lint during development of standalone galaxy ansible roles, with ansible-lint in pre-commit. @ssbarnea closed this bug, but v5.0.7 still fails in the use-case described.

A workaround for pre-commit:

  - repo: https://github.com/ansible-community/ansible-lint.git
    rev: v4.3.7
    hooks:
      - id: ansible-lint
        files: \.(yaml|yml)$

[bbaassssiiee]

Mocking workaround: create a config file in the parent directory: ansible.cfg having:

[defaults]
roles = .

And create a script with the role in the parent directory ansible-lint-me.yml:

#!/usr/bin/env ansible-lint
---
# This playbook for ansible-lint
- name: ansible-lint
  hosts: localhost
  connection: local
  gather_facts: false
  become: false

  roles:
    - role_to_lint