Lookup plugin 'template' doesn't search the same paths as 'file'

[pinkeen]

Ansible Version

± % pip show ansible                                                                                                                                                                                                                          !3182

---
Name: ansible
Version: 1.7.2
Location: /usr/lib/python2.7/site-packages
Requires: paramiko, jinja2, PyYAML, setuptools, pycrypto

The problem

File lookup plugin searches for file in the playbook directory and in the role's files directory.

When using the template plugin the playbook directory isn't searched and lookup returns empty string if file is not found.

{{ lookup('file', 'config/test.j2') }} - returns the file contents
{{ lookup('template', 'config/test.j2') }} - return empty string
{{ lookup('template', '/home/somebody/project/config/test.j2') }} - absolute path works as expected

The dir structure is

project/
  * group_vars/
    * some_group.yml - this group sets the variable via lookup
  * roles/
    * the_actual_role_that_uses_the_variable/
    ...
  * config/
    * test.j2
  * playbook.yml

[mpdehaan]

I believe there may be a misunderstanding there:

"File lookup plugin searches for file in the playbook directory and in the role's template directory."

File lookup plugins look in "files/", the template one I believe looks in "templates/".

This is by design, because of the way templates should operate on templates.

[pinkeen]

You're right, this was an error in description, of course I meant the "files/" dir. The issue still stands though. The template plugin should behave the same way as the file plugin and search for the file in playbook directory (in addition to "templates/" of course). I think that when you use lookup() you expect that each plugin will behave the same way when it comes to the path you specify.

[bcoca]

Not all lookups behave that way, some of the file based lookups have special logic to look into the files/ directory.

[pinkeen]

Ok, sure, I agree that behaviour should not be identical. In my humble opinion all the lookups which read some kind of file should search similar paths and the playbook directory might be one of them. When I use the file lookup and then decide to exchange it with template then I expect that both will allow me to store the files somewhere outside the role. AFAIK the template action searches the root project directory too.

All I advocate that both file and template lookups behave "analogously" when searching for source file.

Also there is a second issue here: template lookup fails silently if file is not found.

[josef-spak]

Just had a colleague of mine run into this as well, while trying to use the virt (extras) module which has something like virt: xml="{{ lookup('template', 'container-template.xml.j2') }}" ... in one of the examples.

In my opinion people who didn't run into this behavior before do not expect it.

The lack of any error message makes it worse.

Edit: This was semi-fixed in 1.8.2 and fully fixed in 1.8.3 (error message now reports the path, not just the basename). Current devel reports the absolute path.

So in the template lookup still checks the role's files/ directory only, but at least the error message is decent now, if your Ansible is up to date.

[sebastianneubauer]

I have the same problem with ansible version 1.9.1, so what is the current status, and is there a workaround?
in playbook.yml

...
- role: my_role
  configuration: "{{ lookup('template', 'templates/application.cfg.j2') }}"
...

with the directory structure

/home/path_to_playbook/:
   playbook.yml
   roles/my_role/tasks/main.yml
   templates/application.cfg.j2

I get the error:

Failed to template {{ lookup('template', 'templates/application.cfg.j2') }}: unable to read /home/path_to_playbook/roles/my_role/tasks/templates/application.cfg.j2

So it searches in the roles path instead of the playbook path for templates/application.cfg.j2

It would make much more sense if the template is searched in the playbook folder and not in the role folder, as "{{ lookup('template', 'templates/application.cfg.j2') }}" is written in the playbook and not in the role, so let's say it is evaluated too late: not in the playbook but in the role.

Can I force ansible to evaluate this variable already in the playbook and only pass the lookuped string to the role? Because that is what I really want: produce template and pass the result to the role...

How could I at least get the absolute path to the playbook template folder /home/path_to_playbook/templates/application.cfg.j2? If I pass this absolute path it should work I guess? Is there something like {{ ansible_path_to_playbook }}?

[bennojoy]

@sebastianneubauer you coudl use {{ lookup('template', playbook_dir + '/templates/application.cfg.j2') }}

[sebastianneubauer]

@bennojoy : thanks, "playbook_dir" seems to work! Do you know by chance, if this is somewhere documented or is it an implementation detail which might change in future versions without notice?

[bcoca]

check current devel, the template lookup should now look at both role and playbook level dirs

[jimi-c]

Hi, this situation is no longer present in the devel branch (which will be ansible 2.0)

If you continue seeing any problems related to this issue, or if you have any further questions, please let us know by stopping by one of the two mailing lists, as appropriate:

• https://groups.google.com/forum/#!forum/ansible-project - for user questions, tips, and tricks
• https://groups.google.com/forum/#!forum/ansible-devel - for strategy, future planning, and questions about writing code

Because this project is very active, we're unlikely to see comments made on closed tickets, but the mailing list is a great way to ask questions, or post if you don't think this particular issue is resolved.

Thank you!