synchronize with password authentication does not work with python 2 #56629
Comments
|
Files identified in the description: If these files are inaccurate, please update the |
ansibot
added
affects_2.8
|
👍 Also seeing this issue on RHEL7: My task looks like this: |
samdoran
added
P3
|
After some more testing to determine if I could figure out a workaround: |
* origin/issue587: issue #587: docs: update Changelog. issue #587: disable SSH key setup, it breaks unit tests issue #587: attempt to fix Mac Azure job issue #587: Add 2.8.0/Py2.7 job to Azure issue #587: use deadsnakes PPA Python because VSTS version is junk issue #587: workaround for ansible/ansible#56629 issue #587: enable spawn_reverse_shell and tidy up Azure step names issue #587: ensure Azure worker has a working SSH configuration issue #587: import spawn_reverse_shell.py script. issue #587: support pausing ansible_tests if flag file exists issue #587: "state: absent" was removed in 2.8.0 issue #587: consistent become_exe() behaviour for older Ansibles. issue #587: update stub_connections/ test to use new doas var issue #587: update MODULE FAILURE message format for post >2.7 issue #587: fix syntax error due to presence of comment issue #587: update mitogen_doas doc to match varible change. issue #587: disable deprecation_warnings for CI. issue #587: mitogen_doas should not become_exe for doas_path issue #587: 2.8 whitespace handling was improved. issue #587: 2.8 PlayContext lacks sudo_flags attribute. issue #587: 2.8 PluginLoader.get() introduced new collection_list kwarg issue #587: 2.8 PlayContext.connection no longer contains connection name issue #587: bump max Ansible version
|
I also see this error on CentOS 7 (7.6.1810 (Core)): I had success with Python3 and Ansible 2.8: |
Some fixes for the local runner for the unified CI: - ansible 2.8.0 introduces a breakage in the synchronization module [1]. Freezing the version below 2.8.0 temporarily, until the issue is resolved. - using an input.yml file extracted from one of the Jenkins jobs to run steps locally has an issue when the `os_project_name` variable isn't set - more recent versions of ansible complain about the indentation of the python code in heat-generator/tasks/main.yml [1] ansible/ansible#56629
|
I'm having this issue too. |
|
I'm also having this issue. I believe that adding sshpass as a requirement to the Please note: The hosts do switch between issue 1 and issue 2. In issue 1 the variable is set such that: But in issue 2, it's set such that: Issue (sshpass is installed): Playbook snippet: Result: Honestly, not sure why this one is failing. When I go in and do a manual run via the user it becomes, it works. Issue (sshpass is NOT installed) Playbook Snippet: Result: |
|
From my own testing, this may have something to do with I noticed that, even using a sudoers entry for rsync, it attempts to still use the password. Removing the password from the tuple just results in a |
|
As a temporary work around to get deploys working, I grabbed the ansible 2.7 synchronize module, added the _local_rsync_password parameter to it, and dropped it in a library directory where all our playbooks are located. This basically restored the ansible 2.7 behavior which is good enough for now until this issue is resolved. |
|
That does appear to work with password-less sudo or using the root user; however using a passworded sudoer user I was still unable to connect. Were you able to connect utilizing a user that has to pass a password @bitchkat ? |
|
@notoriousbsd no, this was explicitly re-enabling the ansible 2.7 behavior which didn't support running rsync via sudo with a password. https://docs.ansible.com/ansible/2.7/modules/synchronize_module.html says "Currently, synchronize is limited to elevating permissions via passwordless sudo. This is because rsync itself is connecting to the remote machine and rsync doesn’t give us a way to pass sudo credentials in." We have an sudoers entry that allows running via sudo without a password: |
|
Yes, we had to do this as well. I just made the playbook add the line to the sudoers file, and remove it after it's done, but that's a fine work around for me. |
|
As Currently I prefer fixing version on |
|
The recent change of I think this is the root cause. |
|
Same issue ... what's news ? |
|
Any news on this? |
|
X? |
|
On my Ubuntu 18.04 I fixed it via adding |
|
Is there any news on that topic? I cannot change python version to python3, because of SLES 12 and the missing support of python 3.5. And with python 2.x it comes to the behaviour as described for the synchronize module: FAILED! => {"changed": false, "cmd": "sshpass", "msg": "[Errno 2] No such file or directory", "rc": 2} |
doesn't seem to work for me ;( |
|
It seems that this issue can arise when Ansible on the control host is running via Python 2 interpreter. To avoid passing the password to sshpass in the command line it uses pipes and I'm running Ubuntu 18.04 LTS on the control machine, and it seems that it still runs Ansible using python2 interpreter, so it is unable to pass the password to sshpass and the process fails. It is relatively simple to patch 463,464c463,464
< _sshpass_pipe = os.pipe()
< cmd = ['sshpass', '-d' + to_native(_sshpass_pipe[0], errors='surrogate_or_strict')] + cmd
---
> cmd = ['sshpass', '-p' + rsync_password] + cmd
>
577,592c577
< # If we are using password authentication, write the password into the pipe
< if rsync_password:
< def _write_password_to_pipe(proc):
< os.close(_sshpass_pipe[0])
< try:
< os.write(_sshpass_pipe[1], to_bytes(rsync_password) + b'\n')
< except OSError as exc:
< # Ignore broken pipe errors if the sshpass process has exited.
< if exc.errno != errno.EPIPE or proc.poll() is None:
< raise
<
< (rc, out, err) = module.run_command(
< cmd, pass_fds=_sshpass_pipe,
< before_communicate_callback=_write_password_to_pipe)
< else:
< (rc, out, err) = module.run_command(cmd)
---
> (rc, out, err) = module.run_command(cmd)
Also I used module option P.S. I think it could be nice and quite easy to add option to pass password to sshpass on the command line to accomodate python2 users. I can contribute the PR if needed. |
|
@Rogach |
|
@sugitk Unfortunately ansible team is opposed to such PR being merged, citing security issues. (see: https://groups.google.com/forum/#!topic/ansible-devel/jk7ixOTb6rQ) Thus it seems that the only way to proceed is to patch local ansible installation yourself. |
|
It's possible that what needs to happen is in This would bring It should be as simple as: diff --git a/lib/ansible/module_utils/basic.py b/lib/ansible/module_utils/basic.py
index 4756381ac6..e0a616b5f6 100644
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@@ -2454,7 +2454,8 @@ class AnsibleModule(object):
``use_unsafe_shell=False`` no path or variable expansion will be done.
:kw pass_fds: When running on python3 this argument
dictates which file descriptors should be passed
- to an underlying ``Popen`` constructor.
+ to an underlying ``Popen`` constructor. On python2,
+ this toggles `close_fds` to False
:kw before_communicate_callback: This function will be called
after ``Popen`` object will be created
but before communicating to the process.
@@ -2565,6 +2566,8 @@ class AnsibleModule(object):
)
if PY3 and pass_fds:
kwargs["pass_fds"] = pass_fds
+ elif PY2 and pass_fds:
+ kwargs["close_fds"] = False
# store the pwd
prev_dir = os.getcwd()
diff --git a/test/units/module_utils/basic/test_run_command.py b/test/units/module_utils/basic/test_run_command.py
index e41a4960b4..2fd721e5de 100644
--- a/test/units/module_utils/basic/test_run_command.py
+++ b/test/units/module_utils/basic/test_run_command.py
@@ -13,6 +13,7 @@ from io import BytesIO
import pytest
from ansible.module_utils._text import to_native
+from ansible.module_utils.six import PY2
class OpenBytesIO(BytesIO):
@@ -197,3 +198,21 @@ class TestRunCommandOutput:
# bytes because it's returning native strings
assert stdout == to_native(u'Žarn§')
assert stderr == to_native(u'لرئيسية')
+
+
+@pytest.mark.parametrize('stdin', [{}], indirect=['stdin'])
+def test_run_command_fds(mocker, rc_am):
+ subprocess = mocker.patch('ansible.module_utils.basic.subprocess')
+ subprocess.Popen.side_effect = AssertionError # short circuit execution, we don't what happens after
+
+ try:
+ rc_am.run_command('/bin/true', pass_fds=(1,))
+ except SystemExit:
+ pass
+
+ if PY2:
+ assert subprocess.Popen.call_args[1]['close_fds'] == False
+ assert 'pass_fds' not in subprocess.Popen.call_args[1]
+ else:
+ assert subprocess.Popen.call_args[1]['close_fds'] == True
+ assert subprocess.Popen.call_args[1]['pass_fds'] == (1,)We may also want to extend the docstring for I believe doing this in |
sivel
added
the
easyfix
|
I'll get a PR together to fix this. Thanks for doing the hard work, @sivel! |
|
Hi, kubepray uses synchronize module to copy the files but ansible_ssh_pass doesn't pass to rsync and had to install sshpass on my clients to fix this but despite of that was having "broken pipe" issue but I applied that patched and it fix the issue. May I know when is this PR going to be merged? Secondly, is there anyway ansible_ssh_pass can be used with rsync without installing sshpass? Thanks in advance |
|
hi, i have the same problem with ansible version 2.7.8 and i fix by update to ansible 2.9.4 and run with python3. $ ansible --version I update the ansible to version 2.9.4 and run playbook with pytnhon 3 interpreter option "-e ansible_python_interpreter=python3" $ sudo apt update $ansible --version ansible 2.9.4 ansible-playbook -i inventory -b ZabbixAgent/main.yml -e ansible_python_interpreter=python3 after doing this step the problem was solved to me. |
and others
SUMMARY
In Ansible 2.8 (in 1403744),
synchronizemodule wraps rsync command in sshpass automatically if password authentication is enabled. It is a great improvement, however, the way it was implemented works only with Python 3 (due to usage ofpass_fdsofsubprocess.Popen, I suppose).Previously there was a way to force synchronize to work with password-based authentication with Python 2 by passing proper
rsync_opts, now it fails with "Broken pipe" error.ISSUE TYPE
COMPONENT NAME
synchronize
ANSIBLE VERSION
CONFIGURATION
OS / ENVIRONMENT
Ubuntu 18.04 (4.15.0-50-generic)
STEPS TO REPRODUCE
EXPECTED RESULTS
I expected
synchronizemodule to succeed.ACTUAL RESULTS
synchronizemodule fails with:It works when I set python interpreter to
python3like so:The text was updated successfully, but these errors were encountered: