'could not find job' on async_status with delegate_to localhost

[jojoob]

Summary

async_status is looking at a different place for the async_dir as it actually is when the actual async task and async_status are delegated to localhost, HOME environment variable is explicitly set and ANSIBLE_ASYNC_DIR is not explicitly set.

I think the bug lies somewhere in the _remote_expand_user function of the async_module which has its own logic to expand the ~ in the path while async_wrapper uses os.path.expanduser to do that.

I first discovered the issue while using ansible-semaphore: semaphoreui/semaphore#943

Issue Type

Bug Report

Component Name

async_status

Ansible Version

$ ansible --version
ansible [core 2.15.4]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/jojoob/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /home/jojoob/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.5 (main, Sep  2 2023, 14:16:33) [GCC 13.2.1 20230801] (/usr/bin/python)
  jinja version = 3.1.2
  libyaml = True

Configuration

# if using a version older than ansible-core 2.12 you should omit the '-t all'
$ ansible-config dump --only-changed -t all
CALLBACKS_ENABLED(/etc/ansible/ansible.cfg) = ['profile_tasks']
CONFIG_FILE() = /etc/ansible/ansible.cfg
DEFAULT_LOAD_CALLBACK_PLUGINS(/etc/ansible/ansible.cfg) = True
DEFAULT_STDOUT_CALLBACK(/etc/ansible/ansible.cfg) = yaml
EDITOR(env: EDITOR) = vim
PAGER(env: PAGER) = less

OS / Environment

arch Linux and Ubuntu 20.04

Steps to Reproduce

Test Playbook (async-delegate.yml):

- hosts: all
  gather_facts: no
  tasks:
    - command: sleep 5
      async: 7
      poll: 0
      register: async_task
      delegate_to: localhost
    - async_status:
        jid: "{{async_task.ansible_job_id}}"
      register: job_result
      until: job_result.finished
      retries: 7
      delay: 1
      delegate_to: localhost

Command: HOME=/tmp/foo ansible-playbook -i example.host, async-delegate.yml --verbose

Expected Results

The playbook should run without errors, retrying max. 7 times until the sleep command is finished. And this is the case for the following variants:

• ansible-playbook -i example.host, async-delegate.yml
• ANSIBLE_ASYNC_DIR=/tmp/bar HOME=/tmp/foo ansible-playbook -i example.host, async-delegate.yml

Using /etc/ansible/ansible.cfg as config file

PLAY [all] *************************************************************

TASK [command] ******************************************************
Donnerstag 28 September 2023  08:38:54 +0200 (0:00:00.012)       0:00:00.012 *** 
changed: [example.host -> localhost] => changed=true 
  ansible_job_id: j628130553180.21703
  finished: 0
  results_file: /home/jojoob/.ansible_async/j628130553180.21703
  started: 1

TASK [async_status] ****************************************************
Donnerstag 28 September 2023  08:38:54 +0200 (0:00:00.409)       0:00:00.422 *** 
FAILED - RETRYING: [example.host -> localhost]: async_status (7 retries left).
FAILED - RETRYING: [example.host -> localhost]: async_status (6 retries left).
FAILED - RETRYING: [example.host -> localhost]: async_status (5 retries left).
FAILED - RETRYING: [example.host -> localhost]: async_status (4 retries left).
FAILED - RETRYING: [example.host -> localhost]: async_status (3 retries left).
changed: [example.host -> localhost] => changed=true 
  ansible_job_id: j628130553180.21703
  attempts: 6
  cmd:
  - sleep
  - '5'
  delta: '0:00:06.003162'
  end: '2023-09-28 08:39:00.579409'
  finished: 1
  msg: ''
  rc: 0
  results_file: /home/jojoob/.ansible_async/j628130553180.21703
  start: '2023-09-28 08:38:54.576247'
  started: 1
  stderr: ''
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

Actual Results

$ HOME=/tmp/foo ansible-playbook -i example.host, async-delegate.yml --verbose
Using /etc/ansible/ansible.cfg as config file

PLAY [all] *************************************************************

TASK [command] ******************************************************
Donnerstag 28 September 2023  08:26:04 +0200 (0:00:00.012)       0:00:00.012 *** 
changed: [example.host -> localhost] => changed=true 
  ansible_job_id: j923045085645.20378
  finished: 0
  results_file: /tmp/foo/.ansible_async/j923045085645.20378
  started: 1

TASK [async_status] ****************************************************
Donnerstag 28 September 2023  08:26:04 +0200 (0:00:00.401)       0:00:00.413 *** 
fatal: [example.host -> localhost]: FAILED! => changed=false 
  ansible_job_id: j923045085645.20378
  attempts: 1
  finished: 1
  msg: could not find job
  results_file: /home/jojoob/.ansible_async/j923045085645.20378
  started: 1
  stderr: ''
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct

[ansibot]

Files identified in the description:

• lib/ansible/modules/async_status.py
• lib/ansible/plugins/action/async_status.py

If these files are incorrect, please update the component name section of the description or use the component bot command.