Async with find weirdly fails when ansible_python_interpreter=python3

[noonedeadpunk]

SUMMARY

When find module is launched with async, it just fails when playbook/role is launched with -e ansible_python_interpreter=python3.
It works ok under the wollowing conditions though:

• When ansible_python_interpreter=python2
• When ansible_python_interpreter=/usr/bin/python3
• Without async usage but with ansible_python_interpreter=python3

ISSUE TYPE

• Bug Report

COMPONENT NAME

async_command or find

ANSIBLE VERSION

(ansible) root@lodgeit:~# ansible --version
ansible 2.9.6
  config file = None
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ubuntu/ansible/lib/python3.6/site-packages/ansible
  executable location = /home/ubuntu/ansible/bin/ansible
  python version = 3.6.9 (default, Nov  7 2019, 10:44:02) [GCC 8.3.0]
(ansible) root@lodgeit:~#

CONFIGURATION

(ansible) root@lodgeit:~# ansible-config dump --only-changed
(ansible) root@lodgeit:~# 

OS / ENVIRONMENT

Reproduces on Fedora 30, Ubuntu 18.04, Debian 10, etc.

STEPS TO REPRODUCE

- hosts: localhost
  connection: local
  tasks:
    - find:
        paths: /
        hidden: yes
        patterns: '.shosts,shosts.equiv'
      register: shosts_find
      async: 300
      poll: 0
      changed_when: False

    - async_status:
        jid: "{{ shosts_find.ansible_job_id }}"
      failed_when: False
      changed_when: False
      register: job_result
      until: job_result.finished | bool

EXPECTED RESULTS

ok: [localhost] => {
    "job_result": {
        "ansible_job_id": "922640901297.25973",
        "attempts": 2,
        "changed": false,
        "examined": 141540,
        "failed": false,
        "failed_when_result": false,
        "files": [],
        "finished": 1,
        "matched": 0,
        "msg": "/proc/25978/task/25978/fd/4 was skipped as it does not seem to be a valid file or it cannot be accessed\n/proc/25978/task/25978/fdinfo/4 was skipped as it does not seem to be a valid file or it cannot be accessed\n/proc/25978/fd/4 was skipped as it does not seem to be a valid file or it cannot be accessed\n/proc/25978/fdinfo/4 was skipped as it does not seem to be a valid file or it cannot be accessed\n"
    }
}

ACTUAL RESULTS

(ansible) root@lodgeit:~# ansible-playbook test.yml -i localhost, -e ansible_python_interpreter=python3 -vvv
ansible-playbook 2.9.6
  config file = None
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ubuntu/ansible/lib/python3.6/site-packages/ansible
  executable location = /home/ubuntu/ansible/bin/ansible-playbook
  python version = 3.6.9 (default, Nov  7 2019, 10:44:02) [GCC 8.3.0]
No config file found; using defaults
Parsed localhost, inventory source with host_list plugin

PLAYBOOK: test.yml **********************************************************************************************************
1 plays in test.yml

PLAY [localhost] ************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************
task path: /home/ubuntu/test.yml:2
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1585770446.459262-35501733551671 `" && echo ansible-tmp-1585770446.459262-35501733551671="` echo /root/.ansible/tmp/ansible-tmp-1585770446.459262-35501733551671 `" ) && sleep 0'                                                                                                           
Using module file /home/ubuntu/ansible/lib/python3.6/site-packages/ansible/modules/system/setup.py
<localhost> PUT /home/ubuntu/.ansible/tmp/ansible-local-16773aahoatej/tmp1pc76nro TO /root/.ansible/tmp/ansible-tmp-1585770446.459262-35501733551671/AnsiballZ_setup.py                                                                                   
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1585770446.459262-35501733551671/ /root/.ansible/tmp/ansible-tmp-1585770446.459262-35501733551671/AnsiballZ_setup.py && sleep 0'                                                    
<localhost> EXEC /bin/sh -c 'python3 /root/.ansible/tmp/ansible-tmp-1585770446.459262-35501733551671/AnsiballZ_setup.py && sleep 0'                                                                                                                       
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1585770446.459262-35501733551671/ > /dev/null 2>&1 && sleep 0'                                                                                                                       
ok: [localhost]
META: ran handlers

TASK [find] *****************************************************************************************************************
task path: /home/ubuntu/test.yml:5
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174 `" && echo ansible-tmp-1585770447.67535-145702848030174="` echo /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174 `" ) && sleep 0'                                                                                                           
Using module file /home/ubuntu/ansible/lib/python3.6/site-packages/ansible/modules/files/find.py
<localhost> PUT /home/ubuntu/.ansible/tmp/ansible-local-16773aahoatej/tmp_voe8ryn TO /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/AnsiballZ_find.py                                                                                    
<localhost> PUT /home/ubuntu/.ansible/tmp/ansible-local-16773aahoatej/tmp9bn42nky TO /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/async_wrapper.py                                                                                     
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/ /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/AnsiballZ_find.py /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/async_wrapper.py && sleep 0'                                                                                                 
<localhost> EXEC /bin/sh -c 'ANSIBLE_ASYNC_DIR='"'"'~/.ansible_async'"'"' python3 /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/async_wrapper.py 128314508930 300 /root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/AnsiballZ_find.py _ && sleep 0'                                                                                                    
ok: [localhost] => {
    "ansible_job_id": "128314508930.16855",
    "changed": false,
    "finished": 0,
    "results_file": "/home/ubuntu/.ansible_async/128314508930.16855",
    "started": 1
}

TASK [async_status] *********************************************************************************************************
task path: /home/ubuntu/test.yml:14
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1585770448.4405618-151811452492619 `" && echo ansible-tmp-1585770448.4405618-151811452492619="` echo /root/.ansible/tmp/ansible-tmp-1585770448.4405618-151811452492619 `" ) && sleep 0'                                                                                                     
Using module file /home/ubuntu/ansible/lib/python3.6/site-packages/ansible/modules/utilities/logic/async_status.py
<localhost> PUT /home/ubuntu/.ansible/tmp/ansible-local-16773aahoatej/tmpndrufjm9 TO /root/.ansible/tmp/ansible-tmp-1585770448.4405618-151811452492619/AnsiballZ_async_status.py                                                                          
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1585770448.4405618-151811452492619/ /root/.ansible/tmp/ansible-tmp-1585770448.4405618-151811452492619/AnsiballZ_async_status.py && sleep 0'                                         
<localhost> EXEC /bin/sh -c 'python3 /root/.ansible/tmp/ansible-tmp-1585770448.4405618-151811452492619/AnsiballZ_async_status.py && sleep 0'                                                                                                              
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1585770448.4405618-151811452492619/ > /dev/null 2>&1 && sleep 0'                                                                                                                     
ok: [localhost] => {
    "ansible_job_id": "128314508930.16855",
    "attempts": 1,
    "changed": false,
    "cmd": "/root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/AnsiballZ_find.py",
    "failed_when_result": false,
    "finished": 1,
    "invocation": {
        "module_args": {
            "_async_dir": "/home/ubuntu/.ansible_async",
            "jid": "128314508930.16855",
            "mode": "status"
        }
    },
    "msg": "[Errno 2] No such file or directory: '/root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/AnsiballZ_find.py': '/root/.ansible/tmp/ansible-tmp-1585770447.67535-145702848030174/AnsiballZ_find.py'",
    "outdata": "",
    "stderr": "",
    "stderr_lines": []
}
META: ran handlers
META: ran handlers

PLAY RECAP ******************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[ansibot]

Files identified in the description:
None

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[sivel]

It's never recommended to use relative paths or just the command name for the ansible_python_interpreter. It should always be an absolute path to the binary. Part of this is due to certain features invoking commands through Popen with shell=False. Additionally, we set what you provide as the shebang in scripts. So if you just provide python3, then the shebang becomes #!python3 which is invalid.

This is the case with async.

With the above in mind, I'm going to proceed with closing this.

If you have further questions please stop by IRC or the mailing list:

• IRC: #ansible on irc.freenode.net
• mailing list: https://groups.google.com/forum/#!forum/ansible-project

[noonedeadpunk]

Weird that it doesn't fail with ansible_python_interpreter=python2 then. Ok, thanks for the answer and spending your time.