Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

with_first_found will always find a task before vars in a role #14857

Closed
JamesBelchamber opened this issue Mar 8, 2016 · 9 comments
Closed
Labels
bug This issue/PR relates to a bug.

Comments

@JamesBelchamber
Copy link

Issue Type:
  • Bug Report
Ansible Version:
ansible 2.0.1.0
Ansible Configuration:

None

Environment:

OS X

Summary:

When with_first_found is used to look up a file in vars/ or files/ and a file with the same name exists in tasks/, the file in tasks/ will be returned instead.

Steps To Reproduce:

The galaxy role DavidWittman.filebeat exhibits this problem, as described in DavidWittman/ansible-filebeat#1. This is the relevant part of the role layout:

├── tasks
│   ├── debian.yml
│   ├── main.yml
│   └── redhat.yml
└── vars
    ├── Debian.yml
    ├── RedHat.yml
    └── main.yml

And here is the play:

- include_vars: "{{ item }}"
  with_first_found:
    - "{{ ansible_os_family }}.yml"
    - default.yml

Please bear in mind that on OS X, the default filesystem is HFS+ which is case-insensitive.

Expected Results:

vars/RedHat.yml should be returned on a EL server.

Actual Results:

tasks/redhat.yml is returned and an error is reported (because it is not a vars file).

TASK [DavidWittman.filebeat : include_vars] ************************************
fatal: [x.x.x.x]: FAILED! => {"failed": true, "msg": "/usr/local/etc/ansible/roles/DavidWittman.filebeat/tasks/RedHat.yml must be stored as a dictionary/hash"}
@JamesBelchamber
Copy link
Author

This is caused by the path_dwim_relative function. When with_first_found is used within a role, the role's path is passed to path_dwim_relative, which (correctly) identifies it as a role and appends the role's tasks/ directory to the search list. This means that tasks/ is essentially searched in every iteration. The code is here.

I don't really know why this behaviour is desirable so I don't know how I would go about patching this bug. It's possible that the 'patch' is really to let path_dwim_relative handle the list of places to search (instead of the list that's iterated through here), and document the order of places that with_first_found will search by default.

@bcoca
Copy link
Member

bcoca commented Mar 8, 2016

this is because first_found looks at 'all' dirs no matter what the task is, it should be more discriminatory and use 'vars' in this case and not 'files' nor 'templates' which find the match in 'tasks' before even trying 'vars'

@eb4x
Copy link
Contributor

eb4x commented May 21, 2016

Ran into this problem today aswell.

@bcoca
Copy link
Member

bcoca commented Jul 23, 2016

please confirm that merge above fixes this

@eb4x
Copy link
Contributor

eb4x commented Jul 26, 2016

An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/ansible-2.2.0-py2.7.egg/ansible/executor/task_executor.py", line 92, in run
item_results = self._run_loop(items)
File "/Library/Python/2.7/site-packages/ansible-2.2.0-py2.7.egg/ansible/executor/task_executor.py", line 259, in _run_loop
res = self._execute(variables=task_vars)
File "/Library/Python/2.7/site-packages/ansible-2.2.0-py2.7.egg/ansible/executor/task_executor.py", line 454, in _execute
result = self._handler.run(task_vars=variables)
File "/Library/Python/2.7/site-packages/ansible-2.2.0-py2.7.egg/ansible/plugins/action/include_vars.py", line 41, in run
source = self._find_needle('vars', source)
File "/Library/Python/2.7/site-packages/ansible-2.2.0-py2.7.egg/ansible/plugins/action/init.py", line 843, in _find_needle
result = self._loader.path_dwim_relative_stack(path_stack, dirname, needle)
File "/Library/Python/2.7/site-packages/ansible-2.2.0-py2.7.egg/ansible/parsing/dataloader.py", line 284, in path_dwim_relative_stack
if source.startswith('~') or source.startswith(os.path.sep):
AttributeError: 'NoneType' object has no attribute 'startswith'

@bcoca
Copy link
Member

bcoca commented Jul 26, 2016

@eb4x that should only happen if you search for empty or null string (fixing this path now).
@JamesBelchamber I'm going to assume everything is working for you and will close the ticket unless you tell me otherwise

eb4x added a commit to eb4x/ansible-bug that referenced this issue Jul 26, 2016
@eb4x
Copy link
Contributor

eb4x commented Jul 26, 2016

I've created a minimal test case you can pull, https://github.com/eb4x/ansible-bug.git

TASK [example : include_vars]
failed: localhost => {"failed": true, "item": "ansible-bug/roles/example/tasks/RedHat.yml", "message": "ansible-bug/roles/example/tasks/RedHat.yml must be stored as a dictionary/hash"}

Which makes it seem like nothing has really changed. include_vars is still pulling from tasks instead of vars directory first.

@bcoca
Copy link
Member

bcoca commented Aug 8, 2016

so it worked for me cause of 'test bias', now it should work for everyone via ce282e9.

@bcoca bcoca closed this as completed Aug 8, 2016
bcoca added a commit that referenced this issue Aug 8, 2016
named matched only cause i set it to the same in my tests
fixes #14857
@xenithorb
Copy link

xenithorb commented Aug 27, 2016

Sorry to pry but:

  1. I keep running into problems like this where I have to leave workarounds in my roles to actual ansible bugs. Am I using too recent of a version in Fedora? Is this issue present in all lesser versions to 2.1.1 as well?
  2. What exactly is the best practice way to workaround this? Understand that my mentality is that while I could patch my local copy of ansible, I want my role to be compatible to other unsuspecting users.
    • It seemed the most logical and elegant thing was to not have files with the same name, so I decided to go with {{ ansible_os_family }}.yml for vars, and the tasks are prefixed with t_{{ ansible_os_family }}.yml and are named such in their respective folders. And then I left a comment referencing this issue. Is that correct?
  3. Even though this is now fixed, it's worth mentioning that the error message wasn't helpful until I came across this bug. I didn't catch on until I looked closely and saw that include_vars: was complaining about a file in tasks/ versus vars/

@ansibot ansibot added bug This issue/PR relates to a bug. and removed bug_report labels Mar 7, 2018
@ansible ansible locked and limited conversation to collaborators Apr 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue/PR relates to a bug.
Projects
None yet
Development

No branches or pull requests

6 participants