allow 'ignore_missing' for fileglob lookups

[philfry]

SUMMARY

There are situations where it's useful to hide the Unable to find ... in expected paths (use -vvvvv to see paths) message. This PR adds the ability to pass ignore_missing to the fileglob plugin. See below for example.

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

fileglob

ANSIBLE VERSION

ansible 2.8.0.dev0 (fileglob-ignore_missing e0389d0aab) last updated 2018/10/19 10:44:40 (GMT +200)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/phil/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/phil/projects/3rdparty/ansible/lib/ansible
  executable location = /home/phil/projects/3rdparty/ansible/bin/ansible
  python version = 2.7.15 (default, Sep 21 2018, 23:26:48) [GCC 8.1.1 20180712 (Red Hat 8.1.1-5)]

ADDITIONAL INFORMATION

echo -e "foo\nbar\nbaz" > inventory
for f in foo baz; do install -D /dev/null files/${f}/iptables; done
cat > test.yml <<EOF
---
- hosts: all
  gather_facts: no
  tasks:
    - name: "without ignore_missing"
      debug: var=item
      loop: '{{lookup("fileglob",
        "files/"+inventory_hostname+"/iptables",
        wantlist=true)}}'
    - name: "with ignore_missing"
      debug: var=item
      loop: '{{lookup("fileglob",
        "files/"+inventory_hostname+"/iptables",
        wantlist=true, ignore_missing=true)}}'
...
EOF
ansible-playbook -i inventory test.yml 

output:

PLAY [all] ********************************************************************************

TASK [without ignore_missing] *************************************************************
 [WARNING]: Unable to find 'files/bar' in expected paths (use -vvvvv to see paths)

ok: [foo] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables"
}
ok: [baz] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables"
}

TASK [with ignore_missing] ****************************************************************
ok: [foo] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables"
}
ok: [baz] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables"
}

PLAY RECAP ********************************************************************************
bar                        : ok=0    changed=0    unreachable=0    failed=0    skipped=2   
baz                        : ok=2    changed=0    unreachable=0    failed=0    skipped=0
foo                        : ok=2    changed=0    unreachable=0    failed=0    skipped=0

[ansibot]

Hi @philfry, thank you for submitting this pull-request!

click here for bot help

[bcoca]

this is redundant with the 'errors' option available to all lookups. https://docs.ansible.com/ansible/latest/plugins/lookup.html#using-lookup-plugins

[philfry]

this is redundant with the 'errors' option available to all lookups.

The errors option does not eleminate the warnings for me. Am I missing something?

---
- hosts: all
  gather_facts: no
  tasks:
    - name: "with errors=strict"
      debug: var=item
      loop: '{{lookup("fileglob",
        "files/"+inventory_hostname+"/iptables",
        wantlist=true, errors="strict")}}'
    - name: "with errors=ignore"
      debug: var=item
      loop: '{{lookup("fileglob",
        "files/"+inventory_hostname+"/iptables",
        wantlist=true, errors="ignore")}}'
    - name: "with errors=warn"
      debug: var=item
      loop: '{{lookup("fileglob",
        "files/"+inventory_hostname+"/iptables",
        wantlist=true, errors="warn")}}'
...

ansible-playbook -i inventory test2.yml 

PLAY [all] ***************************************************************************************************************************************************************************

TASK [with errors=strict] ************************************************************************************************************************************************************
 [WARNING]: Unable to find 'files/bar' in expected paths (use -vvvvv to see paths)

ok: [foo] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables"
}
ok: [baz] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables"
}

TASK [with errors=ignore] ************************************************************************************************************************************************************
 [WARNING]: Unable to find 'files/bar' in expected paths (use -vvvvv to see paths)

ok: [foo] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables"
}
ok: [baz] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables"
}

TASK [with errors=warn] **************************************************************************************************************************************************************
 [WARNING]: Unable to find 'files/bar' in expected paths (use -vvvvv to see paths)

ok: [foo] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/foo/iptables"
}
ok: [baz] => (item=/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables) => {
    "item": "/home/phil/projects/3rdparty/ansible/tmp/files/baz/iptables"
}

PLAY RECAP ***************************************************************************************************************************************************************************
bar                        : ok=0    changed=0    unreachable=0    failed=0   
baz                        : ok=3    changed=0    unreachable=0    failed=0   
foo                        : ok=3    changed=0    unreachable=0    failed=0   

[bcoca]

ah, nvmd thought you wanted to handle an error, not a warning

[philfry]

affects_2.8, affects_2.9, affects_2.10, affects_2.11, …
I'd really like to see that merged, as the unnescessary warnings are messing up lots of playbook output.

[philfry]

rebased (again)

[netsandbox]

Please see also https://groups.google.com/g/ansible-project/c/C668U45mz7U why this option is maybe needed.

[jborean93]

Are you able to add an integration test for this, once in and passing I am happy to merge.

[jborean93]

Sorry for the delay but 2.14 is now the latest devel version.

[philfry]

Are you able to add an integration test for this, once in and passing I am happy to merge.

I'm not sure. As far as I understood I cannot register the outputs (and thus warnings) from lookups. So something like this:

- debug: msg="hello world"
  loop: '{{q("fileglob", "/path/to/nowhere")}}'
  register: foo

or

- set_fact: bar={{q("fileglob", "/path/to/nowhere")}}'

won't result in having either foo.warnings or bar.warnings defined. Or am I mistaken?

[bcoca]

true, register only takes in module/action output. You cannot get the warning but you can inspect the result in an assert/fail task:

- assert:
     that:
       - found|length > 0
  vars:
     found: '{{q("fileglob", "/path/to/nowhere")}}'

[philfry]

Well I guess that test is already implemented 😀
https://github.com/ansible/ansible/blob/devel/test/integration/targets/lookup_fileglob/non_existent/play.yml