Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Text backups (xxx.yml~) in host_vars override variables #11017

Closed
webmaster128 opened this issue May 15, 2015 · 14 comments · Fixed by #11058
Closed

Text backups (xxx.yml~) in host_vars override variables #11017

webmaster128 opened this issue May 15, 2015 · 14 comments · Fixed by #11058
Labels
bug This issue/PR relates to a bug.

Comments

@webmaster128
Copy link

Issue Type:

Bug Report

Ansible Version:

ansible 1.9.1 from ppa:ansible/ansible

Ansible Configuration:

default

Environment:

Ubuntu 15.04

Summary:

Text backup files ending on tilde as created e.g. by joe or gedit, in host_vars/<hostname>/ override the actual variable data.

Steps To Reproduce:
  • Write samle data into host_vars/<hostname>/test.yml
  • Edit host_vars/<hostname>/test.yml using gedit, gedit will create host_vars/<hostname>/test.yml~
  • Ensure test.yml and test.yml~ differ

Now ansible will override a variable defined in test.yml using the data from test.yml~.

Expected Results:

Only *.yml files are recognized by ansible in host_vars.

Actual Results:

*.yml~ files are recognized by ansible in host_vars.

@bcoca
Copy link
Member

bcoca commented May 15, 2015

host/group_vars import have no extension limitations or requirements, they just have to match host/group name.
so I'm totally wrong, it should only load files with the following extensions:
"", ".yml", ".yaml", ".json", so from the code I cannot see how its loading yml~

@webmaster128
Copy link
Author

Could you add a link to the code you are referring to? Thanks

@bcoca
Copy link
Member

bcoca commented May 15, 2015

@webmaster128
Copy link
Author

Let's do the math together.

Let

YAML_FILENAME_EXTENSIONS = [ "", ".yml", ".yaml", ".json" ]
basepath = "/home/me/serverconfig/host_vars/example.com"
paths_to_check = [ "".join([basepath, ext]) for ext in YAML_FILENAME_EXTENSIONS ]

then

paths_to_check =
['/home/me/serverconfig/host_vars/example.com',
'/home/me/serverconfig/host_vars/example.com.yml',
'/home/me/serverconfig/host_vars/example.com.yaml',
'/home/me/serverconfig/host_vars/example.com.json']

Now
_load_vars_from_path is called on
/home/me/serverconfig/host_vars/example.com
which is a directory.

Then _load_vars_from_folder is called on
/home/me/serverconfig/host_vars/example.com
which will return all files inside, that are not "hidden". But "hidden is defines as starting with "." (https://github.com/ansible/ansible/blob/devel/lib/ansible/utils/__init__.py#L1620)

@bcoca
Copy link
Member

bcoca commented May 15, 2015

ah, yes, you did not mention it was a folder, if using a folder for group
vars, everything not hidden gets slurped.

Brian Coca

@webmaster128
Copy link
Author

Is that desired? Does a user need/want to parse /home/me/serverconfig/host_vars/example.com/holiday.JPEG as YAML?

Assuming it is desired that all files in a host's host_vars are parsed as variables, one could argue about what a hidden file is on Linux. Nautilus on Ubuntu for example does not display text backup files ending on ~ while ls does.

@srgvg
Copy link
Contributor

srgvg commented May 16, 2015

When loading vars from a dir, only hidden file or subdirs are skipped (https://github.com/ansible/ansible/blob/devel/lib/ansible/utils/__init__.py#L1619-L1620)

When checking the base paths to load files from, https://github.com/ansible/ansible/blob/devel/lib/ansible/utils/__init__.py#L1521, C.YAML_FILENAME_EXTENSIONS is respected, however when using a directory in group/host_vars, this happens only for the directory name.

So that check should also happen when scanning a dirctory at https://github.com/ansible/ansible/blob/devel/lib/ansible/utils/__init__.py#L1619-L1620

@bcoca you marked this issues a a bug_report, so I assume such a fix is acceptable?

@bcoca
Copy link
Member

bcoca commented May 18, 2015

@svg, considering it, want to sync with rest of core before deciding

@abadger
Copy link
Contributor

abadger commented May 21, 2015

@bcoca, @jimi-c, and I discussed this and we all think that it would be a bugfix to make loading from a directory obey the same rules as when checking the base paths.

srgvg added a commit to srgvg/ansible that referenced this issue May 23, 2015
inventory vars: make loading from a directory obey the same rules as
when checking the base paths, looking at the file name extensions
as defined in CONSTANTS.YAML_FILENAME_EXTENSIONS

Fixes Github issue ansible#11017
@opqdonut
Copy link
Contributor

Could you reopen this? I'm experiencing the same problem on ansible 2.0.1.0

Reproducing:

# in an empty dir
% mkdir -p host_vars/localhost/
% echo "foo: 1" > host_vars/localhost/vars.yml 
% echo "foo: 2" > host_vars/localhost/vars.yml~
% ansible -m debug -a var="hostvars['localhost']" localhost | grep foo                                   
        "foo": 2, 

@arnodel
Copy link

arnodel commented Sep 14, 2016

Same here in ansible 2.1.1
I use emacs as my primary text editor and it makes backup files by appending a ~ to file names. This means I need to something like find . -name '*~' -exec rm {} \; before running any ansible!

@tpatja
Copy link

tpatja commented Jan 26, 2017

Reproducible consistently with 2.2.0.0. Debugging the cause of failure in a large playbook just took me an hour because of this bug.

@opqdonut's test case is minimal and reproduces the issue every time, so fixing should be simple.

Please re-open the issue. Thank you.

@tmu
Copy link

tmu commented Mar 7, 2017

Confirmed with 2.2.1.0, potentially super-dangerous bug.

Basically, my structure is following:

  • environment/dev/hosts
  • environment/dev/group_vars/all/vars.yml
  • environment/dev/group_vars/all/secret.yml
  • ...

Similarly, for staging environment (and production)

  • environment/staging/hosts
  • environment/staging/group_vars/all/vars.yml
  • environment/staging/group_vars/all/secret.yml
  • ...

Now, Ansible picked values from Emacs temporary file

  • environment/dev/group_vars/all/vars.yml~
    which had values from production as the original was copied from production env directories and edited.

You can clearly envision how this can potentially cause catastrophic situations.

@ghost
Copy link

ghost commented Apr 6, 2017

Confirmed with 2.2.2.0, This bug is pretty problematic. I think it is currently tracked as issue 17382 and there is an open PR.

@ansibot ansibot added bug This issue/PR relates to a bug. and removed bug_report labels Mar 6, 2018
@ansible ansible locked and limited conversation to collaborators Apr 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue/PR relates to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants