Including tasks with first_found fails to find files within same task directory

[anoopcs9]

Summary

Running a playbook, as described further down, which includes tasks from different yaml file within the same task directory using first_found causes the following error terminating the execution:

fatal: [localhost]: FAILED! => {"msg": "No file was found when using first_found."}

Please note that this is a regression from v2.16.2 where the reproducer works as expected.

Issue Type

Bug Report

Component Name

first_found

Ansible Version

ansible [core 2.16.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.12/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.12.1 (main, Jan 23 2024, 13:14:28) [GCC 8.5.0 20210514 (Red Hat 8.5.0-21)] (/usr/bin/python3.12)
  jinja version = 3.1.2
  libyaml = True

Configuration

CONFIG_FILE() = /etc/ansible/ansible.cfg

OS / Environment

CentOS Stream 8

Steps to Reproduce

1. Implement the following playbook

---
# shift.yml
- hosts: localhost
  roles:
    - shift

---
# roles/shift/tasks/main.yml
- name: Include tasks from patch role
  include_role:
    name: patch
    tasks_from: inside/main.yml

---
# roles/patch/tasks/inside/main.yml
- name: Include files
  include_tasks: "{{ include_file}}"
  with_first_found:
    - files:
      - one.yml
      - two.yml
      - three.yml
  loop_control:
    loop_var: include_file

---
# roles/patch/tasks/inside/two.yml
- debug:
    msg: "All good"

Expected Results

PLAY [localhost] ******************************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************
ok: [localhost]

TASK [Include tasks from patch role] **********************************************************************************************************************************************************

TASK [patch : Include files] ******************************************************************************************************************************************************************
included: /tmp/roles/patch/tasks/inside/two.yml for localhost => (item=/tmp/roles/patch/tasks/inside/two.yml)

TASK [patch : debug] **************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "All good"
}

PLAY RECAP ************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Actual Results

PLAY [localhost] ******************************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************
ok: [localhost]

TASK [Include tasks from patch role] **********************************************************************************************************************************************************

TASK [patch : Include files] ******************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "No file was found when using first_found."}

PLAY RECAP ************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibot]

Files identified in the description:

• lib/ansible/plugins/lookup/first_found.py

If these files are incorrect, please update the component name section of the description or use the component bot command.

[sivel]

Caused by a9919dd

[s-hertel]

This worked because the role uses an entrypoint in a subdirectory and the role autodetection logic doesn't handle that at all (meaning as a result, none of the normal role paths were searched).

The docs suggest the task's directory is searched regardless https://docs.ansible.com/ansible/latest/playbook_guide/playbook_pathing.html#resolving-local-relative-paths:

Like 1, in the current task file’s directory.

but I think the documentation was more accurate for roles before that was added in #73818. Maybe the fix is just to change always append the source's current directory?

diff --git a/lib/ansible/parsing/dataloader.py b/lib/ansible/parsing/dataloader.py
index db695ba8dd..da198e05aa 100644
--- a/lib/ansible/parsing/dataloader.py
+++ b/lib/ansible/parsing/dataloader.py
@@ -316,11 +316,10 @@ class DataLoader:
                 if (is_role or self._is_role(path)) and b_pb_base_dir.endswith(b'/tasks'):
                     search.append(os.path.join(os.path.dirname(b_pb_base_dir), b_dirname, b_source))
                     search.append(os.path.join(b_pb_base_dir, b_source))
-                else:
-                    # don't add dirname if user already is using it in source
-                    if b_source.split(b'/')[0] != dirname:
-                        search.append(os.path.join(b_upath, b_dirname, b_source))
-                    search.append(os.path.join(b_upath, b_source))
+                # don't add dirname if user already is using it in source
+                if b_source.split(b'/')[0] != dirname:
+                    search.append(os.path.join(b_upath, b_dirname, b_source))
+                search.append(os.path.join(b_upath, b_source))

[anoopcs9]

Thanks for taking a quick look at the issue.

Since I am not sure about the release criteria and time taken to get it packaged in distribution, is there any workaround that I can use in the meanwhile? Our CI workflow heavily uses first_found in a playbook structure as described in the reproducer steps. It is currently stuck unless we restrict ansible to a version < 2.16.3.

[jborean93]

You could try using {{ role_path }}/tasks/inside/... as a potential workaround. I haven't tested it but theoretically should work as it's an absolute path.

[anoopcs9]

If not obvious, issue is also seen when tasks are included from a different sub directory within the same role as outlined below:

---
# shift.yml
- hosts: localhost
  roles:
    - shift

---
# roles/shift/tasks/main.yml
- name: Include tasks from current role
  include_tasks: within/main.yml

---
# roles/shift/tasks/within/main.yml
- name: Include files
  include_tasks: "{{ include_file }}"
  with_first_found:
    - files:
      - one.yml
      - two.yml
      - three.yml
  loop_control:
    loop_var: include_file

---
# roles/shift/tasks/within/two.yml
- debug:
    msg: "All good"