Jinja expressions are not evaluated in (not only) credential variables

[mordekasg]

OS / Environment

Debian 12

Ansible version

ansible [core 2.14.17]
  config file = /home/test/host/ansible/ansible.cfg
  configured module search path = ['/home/test/host/ansible/local_modules']
  ansible python module location = /home/test/env.ansible/lib/python3.11/site-packages/ansible
  ansible collection location = /home/test/host/ansible/collections:/usr/share/ansible/collections
  executable location = /home/test/env.ansible/bin/ansible
  python version = 3.11.2 (main, May  2 2024, 11:59:08) [GCC 12.2.0] (/home/test/env.ansible/bin/python)
  jinja version = 3.1.4
  libyaml = True

Environment variables for strategy plugin

ANSIBLE_STRATEGY=mitogen_linear
ANSIBLE_STRATEGY_PLUGINS=/home/test/env.ansible/src/mitogen/ansible_mitogen/plugins/strategy

Mitogen version

v0.3.12

Problem

Jinja expressions are not evaluated by mitogen plugin

How to reproduce

Description

Ansible host: srv1 has credentials configured using Jinja expressions from vault/hostvars variables.
Ansible host: srv2 has credentials configured directly as host variables.

Vault file

Path: vault.test.yml

ANSIBLE_LOCALHOST_SRVS_USER: "testusr"
ANSIBLE_LOCALHOST_SRVS_PASSWORD: "okmqwe123"
ANSIBLE_LOCALHOST_SRVS_BECOME_USER: "root"
ANSIBLE_LOCALHOST_SRVS_BECOME_PASSWORD: "okmqwe123"

Inventory file

Path: inventory/local/local.yml

all:
  children:
    test_srvs:
      hosts:
        srv1:
          host_ip: 127.0.0.1
          ansible_host: "{{ host_ip }}"
          ansible_user: "{{ ANSIBLE_LOCALHOST_SRVS_USER }}"
          ansible_password: "{{ ANSIBLE_LOCALHOST_SRVS_PASSWORD }}"
          ansible_become_user: "{{ ANSIBLE_LOCALHOST_SRVS_BECOME_USER }}"
          ansible_become_password: "{{ ANSIBLE_LOCALHOST_SRVS_BECOME_PASSWORD }}"
          ansible_ssh_user: "{{ ansible_user }}"
          ansible_ssh_password: "{{ ansible_password }}"
        srv2:
          ansible_host: 127.0.0.1
          ansible_user: testusr
          ansible_password: okmqwe123
          ansible_become_user: root
          ansible_become_password: okmqwe123
          ansible_ssh_user: testusr
          ansible_ssh_password: okmqwe123

Commands

regular user

ansible -e @vault.test.yml -i inventory/local/local.yml -m ping test_srvs

superuser - become

ansible -b -e @vault.test.yml -i inventory/local/local.yml -m ping test_srvs

Output

regular user

srv1 | UNREACHABLE! => {
    "changed": false,
    "msg": "EOF on stream; last 100 lines received:\nhostname contains invalid characters\r",
    "unreachable": true
}
srv2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

superuser - become

srv1 | UNREACHABLE! => {
    "changed": false,
    "msg": "EOF on stream; last 100 lines received:\nhostname contains invalid characters\r",
    "unreachable": true
}
srv2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Commands with verbose

regular user

ansible -e @vault.test.yml -i inventory/local/local.yml -m ping srv1 -vvvv

superuser - become

ansible -b -e @vault.test.yml -i inventory/local/local.yml -m ping srv1 -vvvv

Output

[mux 871642] 20:15:32.461360 D mitogen.parent: command line for Connection(None): ssh -o "LogLevel ERROR" -l "{{ ansible_user }}" -o "Compression yes" -o "ServerAliveInterval 30" -o "ServerAliveCountMax 10" -o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -o "GlobalKnownHostsFile /dev/null" -C -o ControlMaster=no -o ControlPersist=60s "{{ host_ip }}" /usr/bin/python3 [...]

Expected result

srv1 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
srv2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Source of the problem

Ansible: ansible/ansible@6d2d476

Similar issues

Ansible

• Paramiko plugin: Jinja expressions are not evaluated in credential variables if paramiko plugin is used ansible/ansible#78509
• Ansible: avoid poluting vars with incorrect settings ansible/ansible#76590

Mitogen

• Templating broken when constructing value for ansible_ssh_common_args #905
• ansible_port variable is not evaluated #978
• ansible_user and ansible_ssh_host variables not evaluated by mitogen #1022
• The attribute 'remote_user' of the object 'connection._play_context' contains the variable name instead of its value. #1040
• wait_for_connection to localhost explicitly specified in the inventory and ansible_python_interpreter set to ansible_playbook_python fails with 'timed out waiting for ping module test: 'int' object has no attribute 'template'"' #1079
• feat(ansible): Handle templated ssh_user #1116

[mordekasg]

Tested on latest release: 0.3.9.
Result is still the same.

[golebiewsky]

I have the same issue 👍

[moreati]

With PR #1114 merged into master the Ansible connection password (e.g. ansible_password, ansible_ssh_pass) is templated. Now that I've found a reasonable way to do this, others should follow.

[moreati]

Ansible connection password (e.g. ansible_password, ansible_ssh_pass) is templated

Now released in 0.3.12

[mordekasg]

Ansible connection password (e.g. ansible_password, ansible_ssh_pass) is templated

Now released in 0.3.12

That`s great news!

I have tested on 0.3.12 and templated ansible_password, ansible_ssh_pass are working.

According to this issue: #1116, after new version release, also templated ansible_user, ansible_ssh_user will be working.

I have added PR: #1148, which will allow to template ansible_become_password attribute.

After those 2 PR`s released, feel free to close this issue!

[marbud0]

That`s great news!

I have tested on 0.3.12 and templated ansible_password, ansible_ssh_pass are working.

According to this issue: #1116, after new version release, also templated ansible_user, ansible_ssh_user will be working.

I have added PR: #1148, which will allow to template ansible_become_password attribute.

After those 2 PR`s released, feel free to close this issue!

This problem also affects the ansible_host variable.. It's kind of hinted to in the subject, but not showed in the example.. Can we use this issue to cover this or should I file a new issue?

[mordekasg]

This problem also affects the ansible_host variable.. It's kind of hinted to in the subject, but not showed in the example.. Can we use this issue to cover this or should I file a new issue?

I updated my post and extended example to some extent.

It`s great idea to use this issue to cover also ansible_host variable.