ansible-galaxy list is not listing certain valid roles

[geerlingguy]

SUMMARY

I am trying to see what roles Ansible will pick up from a given roles directory, and it seems like it's only picking up roles that I've downloaded from Galaxy, not any other roles (like custom ones I've created via ansible-galaxy init).

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansible-galaxy

ANSIBLE VERSION

ansible 2.9.4
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/jgeerling/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.17 (default, Feb  9 2020, 19:49:15) [GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.33.17)]

CONFIGURATION

ANSIBLE_NOCOWS(/etc/ansible/ansible.cfg) = True
ANSIBLE_PIPELINING(/etc/ansible/ansible.cfg) = True
ANSIBLE_SSH_CONTROL_PATH(/etc/ansible/ansible.cfg) = /tmp/ansible-ssh-%%h-%%p-%%r
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 20
DEFAULT_HOST_LIST(/etc/ansible/ansible.cfg) = [u'/etc/ansible/hosts']
DEFAULT_ROLES_PATH(/etc/ansible/ansible.cfg) = [u'/Users/jgeerling/Dropbox/VMs/roles']
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False

OS / ENVIRONMENT

macOS Catalina, Ansible installed via pip3

STEPS TO REPRODUCE

$ mkdir testing-roles && cd testing-roles

$ $ ANSIBLE_ROLES_PATH=$(pwd) ansible-galaxy list
# /Users/jgeerling/Downloads/testing-roles
(no roles listed)

$ ansible-galaxy init testing
- Role testing was created successfully

$ ANSIBLE_ROLES_PATH=$(pwd) ansible-galaxy list
# /Users/jgeerling/Downloads/testing-roles
(still no roles listed)

EXPECTED RESULTS

I would expect the new testing role would be listed.

ACTUAL RESULTS

The new testing role is not listed.

ADDITIONAL INFO

After failing the above scenario, I installed a role from Galaxy, and it was listed:

$ ansible-galaxy install -p ./ geerlingguy.php
- downloading role 'php', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-php/archive/3.7.0.tar.gz
- extracting geerlingguy.php to /Users/jgeerling/Downloads/testing-roles/geerlingguy.php
- geerlingguy.php (3.7.0) was installed successfully

$ ANSIBLE_ROLES_PATH=$(pwd) ansible-galaxy list
# /Users/jgeerling/Downloads/testing-roles
- geerlingguy.php, 3.7.0

[geerlingguy]

Testing in a fresh Ubuntu environment:

$ docker run -it geerlingguy/docker-ubuntu1804-ansible /bin/bash

# apt-get update
# apt-get install -y python3-pip
# pip3 install ansible

# mkdir testing-roles && cd testing-roles
# ANSIBLE_ROLES_PATH=$(pwd) ansible-galaxy list
# /testing-roles

# ansible-galaxy init testing
- Role testing was created successfully

# ANSIBLE_ROLES_PATH=$(pwd) ansible-galaxy list
# /testing-roles

# ansible-galaxy install -p ./ geerlingguy.php
- downloading role 'php', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-php/archive/3.7.0.tar.gz
- extracting geerlingguy.php to /testing-roles/geerlingguy.php
- geerlingguy.php (3.7.0) was installed successfully

# ANSIBLE_ROLES_PATH=$(pwd) ansible-galaxy list
# /testing-roles
- geerlingguy.php, 3.7.0

So, same behavior as on my Mac environment; I even tried putting in the path directly (instead of $(pwd) and that made no difference.

[geerlingguy]

Interesting, using ANSIBLE_ROLES_PATH=$(pwd) strace -ffttv -e open -e stat ansible-galaxy list I found that:

# /testing-roles
22:52:17.578160 stat("/testing-roles/meta/main.yml", 0x7fff6623cf80) = -1 ENOENT (No such file or directory)
22:52:17.580177 stat("/testing-roles/meta/main.yaml", 0x7fff6623cf80) = -1 ENOENT (No such file or directory)
22:52:17.582333 stat("/testing-roles/geerlingguy.php/meta/main.yml", {st_dev=makedev(0, 285), st_ino=2665699, st_mode=S_IFREG|0664, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=594, st_atime=1581547861 /* 2020-02-12T22:51:01.840968000+0000 */, st_atime_nsec=840968000, st_mtime=1540320602 /* 2018-10-23T18:50:02+0000 */, st_mtime_nsec=0, st_ctime=1581547859 /* 2020-02-12T22:50:59.688968000+0000 */, st_ctime_nsec=688968000}) = 0
22:52:17.594287 stat("/testing-roles/geerlingguy.php/meta/main.yaml", 0x7fff6623cf80) = -1 ENOENT (No such file or directory)
22:52:17.595648 stat("/testing-roles/geerlingguy.php/meta/.galaxy_install_info", {st_dev=makedev(0, 285), st_ino=2665728, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=54, st_atime=1581547861 /* 2020-02-12T22:51:01.851968000+0000 */, st_atime_nsec=851968000, st_mtime=1581547859 /* 2020-02-12T22:50:59.697968000+0000 */, st_mtime_nsec=697968000, st_ctime=1581547859 /* 2020-02-12T22:50:59.697968000+0000 */, st_ctime_nsec=697968000}) = 0
- geerlingguy.php, 3.7.0
22:52:17.604541 stat("/root/.ansible/tmp/ansible-local-1556d0zrxymb", {st_dev=makedev(0, 285), st_ino=2665696, st_mode=S_IFDIR|0700, st_nlink=2, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=4096, st_atime=1581547934 /* 2020-02-12T22:52:14.751725000+0000 */, st_atime_nsec=751725000, st_mtime=1581547934 /* 2020-02-12T22:52:14.751725000+0000 */, st_mtime_nsec=751725000, st_ctime=1581547934 /* 2020-02-12T22:52:14.751725000+0000 */, st_ctime_nsec=751725000}) = 0
22:52:17.606447 stat("/root/.ansible/tmp/ansible-local-1556d0zrxymb", {st_dev=makedev(0, 285), st_ino=2665696, st_mode=S_IFDIR|0700, st_nlink=2, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=4096, st_atime=1581547934 /* 2020-02-12T22:52:14.751725000+0000 */, st_atime_nsec=751725000, st_mtime=1581547934 /* 2020-02-12T22:52:14.751725000+0000 */, st_mtime_nsec=751725000, st_ctime=1581547934 /* 2020-02-12T22:52:14.751725000+0000 */, st_ctime_nsec=751725000}) = 0

Basically, for the local role(s) it's looking for the file /testing-roles/meta/main.yml and not finding it. For Galaxy-installed roles it's looking for /testing-roles/geerlingguy.php/meta/main.yaml and still saying ENOENT (No such file or directory) but that file does exist.

[geerlingguy]

Well the plot thickens...

# ansible-galaxy init foo    
- Role foo was created successfully

# ANSIBLE_ROLES_PATH=$(pwd) ansible-galaxy list
# /testing-roles
- foo, (unknown version)
- geerlingguy.php, 3.7.0

So maybe this is just an issue with roles named testing?

[geerlingguy]

That was a fun rabbit hole—I can confirm that, as long as the role name does not consist of the word test or testing, it is listed.

Tested the following names:

Role name	Is listed with ansible-galaxy list?
testing	❌
test	❌
tests	✅
foo	✅
zzztest	✅

[geerlingguy]

Debugging with strace:

• ansible-galaxy init testing: https://gist.github.com/geerlingguy/d361d073a95e0497b92d12982934e5ff
• ansible-galaxy install -p ./ geerlingguy.php: https://gist.github.com/geerlingguy/78b31ab4ac4fcd828e3e04b80dd4453b
• ansible-galaxy list: https://gist.github.com/geerlingguy/e829bb25a0a37e8c07258a7fb733cfc5

[geerlingguy]

I tried inside the directory fizzbuzz instead of testing-roles, and list did work, meaning it seems to be a directory-naming-related bug if the role has a substring of part of the path.

[geerlingguy]

According to @jctanner it seems the issue is in this bit of code: https://github.com/ansible/ansible/blob/devel/lib/ansible/galaxy/role.py#L69-L71