backup files in host_vars override vars.yml #17382

rodmanning · 2016-09-04T02:08:36Z

ISSUE TYPE

Bug Report

COMPONENT NAME

host_vars configuration

ANSIBLE VERSION

ansible 2.1.1.0
  config file = /home/rod/deltoid/servers/ansible/ansible.cfg
  configured module search path = Default w/o overrides

CONFIGURATION

Added vault password file path to ansible.cfg, nothing else.

OS / ENVIRONMENT

Arch linux (x86_64)

SUMMARY

When reading host_vars, ansible reads vars from a file name "vars.yml~" with priority over a file name "vars.yml". The "vars.yml~" file was created by editing the original file with emacs, which saves the backup by appending a "~" to the filename.

STEPS TO REPRODUCE

Create host_vars in a folder as follows:

ansible-folder
- host_vars
  - my-server-01
    - vars.yml
        ansible_user: root
        ansible_port: 22
    - vars.yml~
        ansible_user: someone_else
        ansible_port: 9999

When connecting to a host (e.g. ansible -i hosts all -m ping), ansible will use the vars saved in vars.yml~.

EXPECTED RESULTS

Expected to connect with 'root' on port 22, etc.

Using /home/rod/deltoid/servers/ansible/ansible.cfg as config file
SSH password: 
Loaded callback minimal of type stdout, v2.0
<dlt-tx-02> ESTABLISH SSH CONNECTION FOR USER: root
<dlt-tx-02> SSH: EXEC sshpass -d13 ssh -C -vvv -o ControlMaster=no -o ControlPersist=no -o Port=22 -o User=root -o ConnectTimeout=10 -o ControlPath=/dev/null dlt-tx-02 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1472954078.5-25255107732476 `" && echo ansible-tmp-1472954078.5-25255107732476="` echo $HOME/.ansible/tmp/ansible-tmp-1472954078.5-25255107732476 `" ) && sleep 0'"'"''
<dlt-tx-02> PUT /tmp/tmpldCjDh TO /root/.ansible/tmp/ansible-tmp-1472954078.5-25255107732476/ping
<dlt-tx-02> SSH: EXEC sshpass -d13 sftp -o BatchMode=no -b - -C -vvv -o ControlMaster=no -o ControlPersist=no -o Port=22 -o User=root -o ConnectTimeout=10 -o ControlPath=/dev/null '[dlt-tx-02]'
<dlt-tx-02> ESTABLISH SSH CONNECTION FOR USER: root
<dlt-tx-02> SSH: EXEC sshpass -d13 ssh -C -vvv -o ControlMaster=no -o ControlPersist=no -o Port=22 -o User=root -o ConnectTimeout=10 -o ControlPath=/dev/null -tt dlt-tx-02 '/bin/sh -c '"'"'LANG=en_AU.UTF-8 LC_ALL=en_AU.UTF-8 LC_MESSAGES=en_AU.UTF-8 /usr/bin/python /root/.ansible/tmp/ansible-tmp-1472954078.5-25255107732476/ping; rm -rf "/root/.ansible/tmp/ansible-tmp-1472954078.5-25255107732476/" > /dev/null 2>&1 && sleep 0'"'"''
dlt-tx-02 | SUCCESS => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "data": null
        }, 
        "module_name": "ping"
    }, 
    "ping": "pong"
}

ACTUAL RESULTS

Ansible generated an ssh command for the incorrect user/port.

Using /home/rod/deltoid/servers/ansible/ansible.cfg as config file
SSH password: 
Loaded callback minimal of type stdout, v2.0
<dlt-tx-02> ESTABLISH SSH CONNECTION FOR USER: someone_else
<dlt-tx-02> SSH: EXEC sshpass -d13 ssh -C -vvv -o ControlMaster=no -o ControlPersist=no -o Port=9999 -o User=someone_else -o ConnectTimeout=10 -o ControlPath=/dev/null dlt-tx-02 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1472954135.94-221184064468334 `" && echo ansible-tmp-1472954135.94-221184064468334="` echo $HOME/.ansible/tmp/ansible-tmp-1472954135.94-221184064468334 `" ) && sleep 0'"'"''
dlt-tx-02 | UNREACHABLE! => {
    "changed": false, 
    "msg": "Failed to connect to the host via ssh.", 
    "unreachable": true
}

The text was updated successfully, but these errors were encountered:

georgebuckerfield · 2016-10-17T23:17:31Z

I've been able to replicate this. I think it's due to the behaviour of the _load_inventory_file() function in lib/ansible/vars/__init__.py.
The list of files to be loaded is sorted using sort(), which leaves a filename appended with a ~ character to be loaded later:
[u'localhost.yml', u'localhost.yml~']

You can avoid those files being loaded with an extra condition on the for loop that follows, but I don't know if that's desirable behaviour.
e.g.

# do not parse hidden files or dirs, e.g. .svn/
paths = [os.path.join(path, name) for name in names if not name.startswith('.') and not name.endswith('~')]

ceving · 2017-01-31T09:20:23Z

Same bug here with group_vars.

rodmanning · 2017-03-26T03:44:34Z

Thanks @george-buckerfield, that's exactly the behaviour I'd be wanting. I've looked at that part of the code and agree with your take on it.

My take on desirable behaviour: saving with a "~" extension for backup files is the default behaviour for both vim & emacs, so makes sense to exclude these when parsing hostvars and groupvars.

Alternatively might be to reverse the sort order, so that backup files were parsed first and over-written by variables in the actual file. (Personally think that's less transparent than just not parsing it at all though.)

Does anyone want a pull request for this, or is more discussion needed about whether or not this is desireable behaviour?

georgebuckerfield · 2017-03-26T10:31:23Z

@rodmanning I had a PR ready to go a while back, so I've submitted it. Happy to modify it based on any feedback though.

fixes ansible#17382 alternate to ansible#22979 deal with cases in which group/host have . in name updated as per feedbck only be strict about extension when doing dirs also avoid ~ endings

fixes #17382 alternate to #22979 deal with cases in which group/host have . in name updated as per feedbck only be strict about extension when doing dirs also avoid ~ endings

fixes #17382 alternate to #22979 deal with cases in which group/host have . in name updated as per feedbck only be strict about extension when doing dirs also avoid ~ endings (cherry picked from commit 602a2bc)

ansibot added triage affects_2.1 This issue/PR affects Ansible v2.1 labels Sep 7, 2016

bcoca added affects_2.0 This issue/PR affects Ansible v2.0 affects_1.9 This issue/PR affects Ansible v1.9 affects_2.2 This issue/PR affects Ansible v2.2 and removed triage labels Sep 9, 2016

ansibot added the c:vars/hostvars label Jan 30, 2017

georgebuckerfield mentioned this issue Mar 26, 2017

Fixes 17382: Parsing vim/emacs backup files #22979

Closed

ghost mentioned this issue Apr 6, 2017

Text backups (xxx.yml~) in host_vars override variables #11017

Closed

bcoca mentioned this issue Apr 7, 2017

Fix vars file selection #23415

Merged

bcoca closed this as completed in #23415 Apr 20, 2017

bcoca added a commit that referenced this issue Apr 20, 2017

fix vars file selection

602a2bc

fixes #17382 alternate to #22979 deal with cases in which group/host have . in name updated as per feedbck only be strict about extension when doing dirs also avoid ~ endings

ansibot added bug This issue/PR relates to a bug. and removed bug_report labels Mar 7, 2018

ansible locked and limited conversation to collaborators Apr 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

backup files in host_vars override vars.yml #17382

backup files in host_vars override vars.yml #17382

rodmanning commented Sep 4, 2016

georgebuckerfield commented Oct 17, 2016

ceving commented Jan 31, 2017

rodmanning commented Mar 26, 2017

georgebuckerfield commented Mar 26, 2017

backup files in host_vars override vars.yml #17382

backup files in host_vars override vars.yml #17382

Comments

rodmanning commented Sep 4, 2016

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

georgebuckerfield commented Oct 17, 2016

ceving commented Jan 31, 2017

rodmanning commented Mar 26, 2017

georgebuckerfield commented Mar 26, 2017