mkdtemp calls do not use expanduser to expand ~ paths

[phobologic]

ISSUE TYPE

• Bug Report

COMPONENT NAME

plugins.shell.ShellBase.mkdtemp

ANSIBLE VERSION

root@ip-10-128-14-156:~# ansible --version
ansible 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

CONFIGURATION

OS / ENVIRONMENT

Ubuntu 14.04

Linux ip-10-128-14-156 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

This is a local run of ansible.

SUMMARY

In 2.2.1.0, #18551 broke the use of plugins.shell.ShellBase.mkdtemp since it doesn't use os.path.expanduser on the path when no system path is being used (ie: it uses ~/ as the base of the path) when creating the directory, but not when files are being put into that temporary directory - so it creates the tempdir in /etc/ansible/~, but when it goes to put the files there it expands it to $HOME/ correctly.

STEPS TO REPRODUCE

In 2.2.0.0, with verbose output turned on, the initial call of the setup module results in this output:

TASK [setup] *******************************************************************
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/core/system/setup.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1485884989.86-75562667700797 `" && echo ansible-tmp-1485884989.86-75562667700797="` echo $HOME/.ansible/tmp/ansible-tmp-1485884989.86-75562667700797 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpCTqQuC TO /.ansible/tmp/ansible-tmp-1485884989.86-75562667700797/setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /.ansible/tmp/ansible-tmp-1485884989.86-75562667700797/ /.ansible/tmp/ansible-tmp-1485884989.86-75562667700797/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /.ansible/tmp/ansible-tmp-1485884989.86-75562667700797/setup.py; rm -rf "/.ansible/tmp/ansible-tmp-1485884989.86-75562667700797/" > /dev/null 2>&1 && sleep 0'
ok: [127.0.0.1]

On 2.2.1.0 the output instead ends up being:

TASK [setup] *******************************************************************
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/core/system/setup.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1485841982.53-82953584593844 `" && echo ansible-tmp-1485841982.53-82953584593844="` echo ~/.ansible/tmp/ansible-tmp-1485841982.53-82953584593844 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpD3gLtN TO ~/.ansible/tmp/ansible-tmp-1485841982.53-82953584593844/setup.py
fatal: [127.0.0.1]: FAILED! => {
    "failed": true,
    "msg": "failed to transfer file to ~/.ansible/tmp/ansible-tmp-1485841982.53-82953584593844/setup.py: [Errno 2] No such file or directory: '~/.ansible/tmp/ansible-tmp-1485841982.53-82953584593844/setup.py'"
}
        to retry, use: --limit @/etc/ansible/run.retry

EXPECTED RESULTS

The path should be expanded to $HOME effectively.

ACTUAL RESULTS

See STEPS TO REPRODUCE.

[phobologic]

BTW, on the broken system, after the run, I see a ~ directory get created in /etc/ansible:

root@ip-10-128-14-156:~# ls -l /etc/ansible/
total 48
drwx------  3 root   root    4096 Jan 31 04:52 ~
drwxrwxr-x  3 ubuntu ubuntu  4096 Jan 31 04:11 ami
-rw-r--r--  1 ubuntu ubuntu 14388 Jan 16 17:03 ansible.cfg
drwxrwxr-x  2 ubuntu ubuntu  4096 Jan 31 04:52 filter_plugins
-rwxr-x---  1 root   root    1507 Jan 31 04:11 hosts
drwxr-xr-x  2 ubuntu ubuntu  4096 Jan 30 21:55 playbook
drwxrwxr-x 13 ubuntu ubuntu  4096 Jan 31 04:11 roles
-rw-r--r--  1 root   root      10 Jan 31 21:22 run.retry
-rw-r--r--  1 ubuntu ubuntu   597 Jan 31 04:11 run.yml

[bcoca]

this should already have been fixed by #20486

[phobologic]

@bcoca thanks - is there a document for testing with master on my system, or does downloading the git repo and doing a python setup.py install just work? I can verify this on my system if there's an easy way to run master there, thanks!

[bcoca]

that will work, but instead of installing try downloading and using source hacking/env-setup , its less permanent

[MiLk]

@bcoca It has been broken again when fixing #20706
I already spent a lot of time on this when I initially reported the issue, let me know if you want me to spend more time next week to try to find a way to solve it in ansible.
Someone provided an easy way to reproduce the issue here: #20332 (comment)

@phobologic You can also use pip install git+git://github.com/ansible/ansible.git@stable.

[phobologic]

Thanks - for now we've worked around it (by configuring, through environment variables, a good remote temp), but I'll try to find some time to dig back into it.

[bcoca]

This was fixed by adding a call to _remote_expand_user in before calling mkdtemp in #20486, it temporarily regressed when the function was updated, but that was also fixed. As such i'm going to close this issue, feel free to reopen or ping us if you find it not to be the case.