set_fact ansible_ssh_common_args fails

[corbinstuard]

ISSUE TYPE

• Bug Report

COMPONENT NAME

set_fact

ANSIBLE VERSION

ansible 2.5.0b2 (stable-2.5 980b6a00e2) last updated 2018/02/22 12:15:16 (GMT -400)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/seit31/ansible/lib/ansible
  executable location = /home/seit31/ansible/bin/ansible
  python version = 2.7.5 (default, Nov 20 2015, 02:00:19) [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]

CONFIGURATION

ANSIBLE_PIPELINING(/etc/ansible/ansible.cfg) = False
ANSIBLE_SSH_ARGS(/etc/ansible/ansible.cfg) = -C -o ControlMaster=auto -o ControlPersist=600s
ANSIBLE_SSH_CONTROL_PATH(/etc/ansible/ansible.cfg) = /tmp/%%h-%%p-%%r
ANSIBLE_SSH_CONTROL_PATH_DIR(/etc/ansible/ansible.cfg) = /tmp
DEFAULT_FILTER_PLUGIN_PATH(/etc/ansible/ansible.cfg) = [u'/etc/ansible/filter_plugins']
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 5
DEFAULT_LOG_PATH(/etc/ansible/ansible.cfg) = /var/log/ansible.log
DEFAULT_ROLES_PATH(/etc/ansible/ansible.cfg) = [u'/etc/ansible/roles', u'/usr/share/ansible/roles']
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
PERSISTENT_CONNECT_TIMEOUT(/etc/ansible/ansible.cfg) = 30
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False

OS / ENVIRONMENT

N/A

SUMMARY

Attempting to set ansible_ssh_common_args mid-play using set_fact prints a warning and doesn't save the variable:

[WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

STEPS TO REPRODUCE

  - name: set ssh jump host args
    set_fact:
      ansible_ssh_common_args: "-o ProxyCommand='ssh -W %h:%p -q root@{{ hostvars[groups['router'][0]]['ansible_host'] }}'"

EXPECTED RESULTS

ACTUAL RESULTS

[ansibot]

Files identified in the description:

• lib/ansible/modules/utilities/logic/set_fact.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @dagwieers
click here for bot help

[corbinstuard]

cc @agaffney

[agaffney]

This appears to be happening as a side effect of the code to prevent malicious overriding of connection vars via facts returned from a compromised host. This happens in the _execute() function in lib/ansible/executor/task_executor.py when the returned facts are passed through clean_vars().

This could be fixed with an exception for set_fact (or maybe even all action plugins) in that code, but I'm not sure if that's the "right" way to do it.

[agaffney]

It looks like this may be a regression in 2.5/devel. I cannot reproduce this in 2.4.3, but I can reproduce with devel from a few weeks ago. However, with devel, I get the warning but it still sets the var.

[agaffney]

@corbinstuard are you positive that the var doesn't actually get saved? It looks like this bug is just the warning, but the value still gets set elsewhere in the code before the warning is triggered.

[corbinstuard]

No, I assumed that was the case since my script was failing but have since verified:

- name: test
  hosts: all
  gather_facts: false
  
  tasks:
  - name: set ssh jump host args
    set_fact:
      ansible_ssh_common_args: "-o ProxyCommand='ssh -W %h:%p -q root@{{ hostvars[groups['router'][0]]['ansible_host'] }}'"
 
  - name: test   
    debug:
      var: ansible_ssh_common_args

PLAY [test] **********************************************************************************************************

TASK [set ssh jump host args] ****************************************************************************************
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [dps1-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [dps3-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [gw-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [df-test]
ok: [hal-test]
ok: [dps2-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [dumbclient1-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [vtc-test]
ok: [dumbclient3-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [dumbclient2-test]
ok: [dumbclient4-test]
ok: [router-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [l2gw-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [l2mlx1-test]
 [WARNING]: Removed restricted key from module data: ansible_ssh_common_args = -o ProxyComma ... 0.104'

ok: [l2mlx2-test]

TASK [test] **********************************************************************************************************
ok: [dps1-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [gw-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [df-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [dps2-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [vtc-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [dps3-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [dumbclient1-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [hal-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [dumbclient2-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [dumbclient3-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [dumbclient4-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [router-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [l2gw-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [l2mlx1-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}
ok: [l2mlx2-test] => {
    "ansible_ssh_common_args": "-o ProxyCommand='ssh -W %h:%p -q root@10.1.0.104'"
}

[bcoca]

so the problem was due to x2 processing of set_fact/include_vars, the cleaning is supposed to happen only when inserting ansible_facts, but these 2 plugins 'overload it' and short circuit it at the same time.

see PR linked above, it should remove the 2nd processing of the ansible_facts key and avoid the warning, but still remove these keys from other 'facts' modules.

[willthames]

@corbinstuard can you check if #38316 would meet your needs as an alternative?

[kladiv]

Hi, still present in ansible 2.5.0

[juliohm1978]

I'm still getting the same warning using ansible 2.5.0 and 2.5.1

ansible 2.5.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

include_vars task prints warning revealing passwords from a protected vault!

TASK [basic-config : include_vars] ***********************************************************************************************************************************************************
 [WARNING]: Removed restricted key from module data: ansible_become_pass = ***REDACTED***

Any way I can disable warnings? Even a globally is acceptable at this point. These warnings are clearly exposing sensitive information.

[juliohm1978]

Quick update: Just tested the new 2.5.2 release and the warning no longer appears. Congrats!