ansible fail to find module in "library" path if its a directory

[danielmotaleite]

I'm trying to use hashivault module and it works fine when installed in the system module path:

/usr/lib/python2.7/dist-packages/ansible/modules/hashivault

But i want to make this more portable and not require users to install software, so i tried to set the library to the extras/plugins/modules , but it fails to find the module

As per #15432, i tried to load multiple paths:

library=extras/plugins/modules:extras/plugins/modules/hashivault

but ansible still fails to find the module

the error is this:

The conditional check 'config_in_vault == False' failed. The error was: An unhandled exception occurred while templating '{{ lookup('hashivault', 'openvpn/users/' + user.name + 
'/generated_date', 'value') | default(False, true) }}'. Error was a <type 'exceptions.ImportError'>, original message: No module named hashivault

the lookup code imports the hashivault using this code:
from ansible.modules.hashivault import hashivault_read

even trying to install the module in $HOME/.ansible/plugins will fail, i can only make this work if the module is installed in the system and strace show that ansible is not even trying to find it in other places.

ISSUE TYPE

• Bug Report

COMPONENT NAME

modules

ANSIBLE VERSION

ansible 2.6.4
  config file = /home/danielleite/git/TechOps/provisioners/ansible/ansible.cfg
  configured module search path = [u'/home/danielleite/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]

CONFIGURATION

ANSIBLE_PIPELINING(/home/dleite/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/home/dleite/ansible/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=30m
DEFAULT_CALLBACK_PLUGIN_PATH(/home/dleite/ansible/ansible.cfg) = [u'/home/dleite/ansible/extras/plugins/callback']
DEFAULT_CALLBACK_WHITELIST(/home/dleite/ansible/ansible.cfg) = ['timer']
DEFAULT_FORKS(/home/dleite/ansible/ansible.cfg) = 10
DEFAULT_GATHERING(/home/dleite/ansible/ansible.cfg) = smart
DEFAULT_HASH_BEHAVIOUR(/home/dleite/ansible/ansible.cfg) = merge
DEFAULT_HOST_LIST(/home/dleite/ansible/ansible.cfg) = [u'/home/dleite/ansible/hosts.test']
DEFAULT_LOOKUP_PLUGIN_PATH(/home/dleite/ansible/ansible.cfg) = [u'/home/dleite/ansible/extras/plugins/lookup']
DEFAULT_MANAGED_STR(/home/dleite/ansible/ansible.cfg) = This file is managed by Ansible.%n
template: {file}
date: %Y-%m-%d %H:%M
user: {uid}
host: {host}
DEFAULT_MODULE_PATH(/home/dleite/ansible/ansible.cfg) = [u'/home/dleite/ansible/extras/plugins/modules', u'/home/dleite/ansible/extras/plugins/modules/hashivault']
DEFAULT_MODULE_UTILS_PATH(/home/dleite/ansible/ansible.cfg) = [u'/home/dleite/ansible/extras/plugins/utils']
DEFAULT_ROLES_PATH(/home/dleite/ansible/ansible.cfg) = [u'/home/dleite/ansible/roles/external', u'/home/dleite/ansible/roles/internal']
DEFAULT_STDOUT_CALLBACK(/home/dleite/ansible/ansible.cfg) = debug
DEFAULT_VAULT_IDENTITY_LIST(/home/dleite/ansible/ansible.cfg) = ['~/.vault-token', '~/.vault-token.staging']
DEFAULT_VAULT_PASSWORD_FILE(/home/dleite/ansible/ansible.cfg) = /home/danielleite/.vault-token
DISPLAY_SKIPPED_HOSTS(/home/dleite/ansible/ansible.cfg) = False
HOST_KEY_CHECKING(/home/dleite/ansible/ansible.cfg) = False
MAX_FILE_SIZE_FOR_DIFF(/home/dleite/ansible/ansible.cfg) = 327680
RETRY_FILES_ENABLED(/home/dleite/ansible/ansible.cfg) = False

OS / ENVIRONMENT

Ubuntu 18.04.1 LTS using
http://ppa.launchpad.net/ansible/ansible/ubuntu

STEPS TO REPRODUCE

Download the https://github.com/TerryHowe/ansible-modules-hashivault/ and install it in the user $HOME/.ansible path
make sure you have the lookup hashivault.py in the lookup seach path
try to do a {{ lookup('hashivault', 'openvpn/test', 'value') | default('', true) }}
even without vault and any config, ansible should load the lookup module, but then errors about missing module

EXPECTED RESULTS

It should try to load hashivault and error about missing parameters

ACTUAL RESULTS

It fails to load the module because its a directory and have not any hashivault.py file as module

[pid  4303] stat("/usr/lib/python2.7/dist-packages/ansible/modules/hashivault", ) = -1 ENOENT (No such file or directory)
[pid  4303] openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/ansible/modules/hashivault.x86_64-linux-gnu.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  4303] openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/ansible/modules/hashivault.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  4303] openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/ansible/modules/hashivaultmodule.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  4303] openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/ansible/modules/hashivault.py", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  4303] openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/ansible/modules/hashivault.pyc", O_RDONLY) = -1 ENOENT (No such file or directory)

It do not try to load it from any other place, even when it reports this
configured module search path = [u'/home/dleite/ansible/extras/plugins/modules', u'/home/dleite/ansible/extras/plugins/modules/hashivault']

The strace shows that in the beginning, ansible do a stat on the correct directories, but fail to load it later on:

openat(AT_FDCWD, "/home/dleite/ansible/extras/plugins/modules/hashivault", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 7
stat("/home/dleite/ansible/extras/plugins/modules/hashivault", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_auth_enable.py", {st_mode=S_IFREG|0664, st_size=3242, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/__init__.pyc", {st_mode=S_IFREG|0664, st_size=158, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_unseal.py", {st_mode=S_IFREG|0664, st_size=2596, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_rekey_status.py", {st_mode=S_IFREG|0664, st_size=2464, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_init.py", {st_mode=S_IFREG|0664, st_size=3545, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_mount_tune.py", {st_mode=S_IFREG|0664, st_size=3810, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_secret_enable.py", {st_mode=S_IFREG|0664, st_size=3377, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_rekey_init.py", {st_mode=S_IFREG|0664, st_size=3596, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_policy_list.py", {st_mode=S_IFREG|0664, st_size=2529, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_read.pyc", {st_mode=S_IFREG|0664, st_size=4139, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_read.py", {st_mode=S_IFREG|0775, st_size=4000, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_delete.py", {st_mode=S_IFREG|0775, st_size=2941, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_secret_list.py", {st_mode=S_IFREG|0664, st_size=2504, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_token_lookup.py", {st_mode=S_IFREG|0664, st_size=3401, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_policy_get.py", {st_mode=S_IFREG|0664, st_size=2685, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_rekey.py", {st_mode=S_IFREG|0664, st_size=2834, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_token_create.py", {st_mode=S_IFREG|0664, st_size=6186, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_secret_disable.py", {st_mode=S_IFREG|0664, st_size=2908, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_audit_list.py", {st_mode=S_IFREG|0664, st_size=2456, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_policy_delete.py", {st_mode=S_IFREG|0664, st_size=2800, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_audit_enable.py", {st_mode=S_IFREG|0664, st_size=3185, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_auth_list.py", {st_mode=S_IFREG|0664, st_size=2498, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_rekey_cancel.py", {st_mode=S_IFREG|0664, st_size=2645, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_userpass_delete.py", {st_mode=S_IFREG|0664, st_size=2640, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_write.py", {st_mode=S_IFREG|0775, st_size=3883, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_seal.py", {st_mode=S_IFREG|0664, st_size=2500, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_userpass_create.py", {st_mode=S_IFREG|0664, st_size=3069, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_policy_set.py", {st_mode=S_IFREG|0664, st_size=2793, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_status.py", {st_mode=S_IFREG|0664, st_size=2493, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/hashivault_list.py", {st_mode=S_IFREG|0664, st_size=3784, ...}) = 0
stat("/home/dleite/ansible/extras/plugins/modules/hashivault/__init__.py", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0

[ansibot]

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[sivel]

There are indeed problems with that repository and trying to achieve your goal.

Typically speaking, a lookup plugin, should not try to import modules. Because of the way that project is laid out, it will be functional unless it is installed into the ansible tree, which is also something I would not recommend.

In the end, this is a problem with the external repository, and not something that ansible will try to support.

We have a feature request (#28770) that in conjunction with changes to how ansible-modules-hashivault works could make this achievable in the future.

If you have further questions please stop by IRC or the mailing list:

• IRC: #ansible on irc.freenode.net
• mailing list: https://groups.google.com/forum/#!forum/ansible-project