Improve ssh.py organisation and state machine #12276

Closed
wants to merge 2 commits into
from

Conversation

Projects
None yet
7 participants
@amenonsen
Contributor

amenonsen commented Sep 8, 2015

I had to reorganise the internals of ssh.py to make the connection handling clearer. I've tested this with various combinations of ANSIBLE_PIPELINING=y/n, ANSIBLE_SCP_IF_SSH=y/n, become: y/n, become_method: su/sudo, etc. but more testing would be appreciated.

@mgedmin

View changes

lib/ansible/plugins/connections/ssh.py
+ '''
+ Takes a binary (ssh, scp, sftp) and optional extra arguments and returns
+ a command line as an array that can be passed to subprocess.Popen after
+ appending any extra commands to it.

This comment has been minimized.

@mgedmin

mgedmin Sep 8, 2015

Contributor

"extra commands"? Perhaps you meant arguments?

@mgedmin

mgedmin Sep 8, 2015

Contributor

"extra commands"? Perhaps you meant arguments?

This comment has been minimized.

@amenonsen

amenonsen Sep 8, 2015

Contributor

Well, I was thinking of "ssh options hostname 'some command with sudo and sh and whatnot'". Arguments would work too. I'll change it.

@amenonsen

amenonsen Sep 8, 2015

Contributor

Well, I was thinking of "ssh options hostname 'some command with sudo and sh and whatnot'". Arguments would work too. I'll change it.

@mgedmin

View changes

lib/ansible/plugins/connections/ssh.py
+ p = subprocess.Popen(cmd, stdin=slave, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ stdin = os.fdopen(master, 'w', 0)
+ os.close(slave)
+ except:

This comment has been minimized.

@mgedmin

mgedmin Sep 8, 2015

Contributor

A bare except: clause is usually frowned upon in Python land, because it tends to catch things like KeyboardInterrupt that you don't want to catch (and also hides any bugs that shouldn't be hidden).

Wouldn't except (OSError, IOError): suffice here?

@mgedmin

mgedmin Sep 8, 2015

Contributor

A bare except: clause is usually frowned upon in Python land, because it tends to catch things like KeyboardInterrupt that you don't want to catch (and also hides any bugs that shouldn't be hidden).

Wouldn't except (OSError, IOError): suffice here?

This comment has been minimized.

@amenonsen

amenonsen Sep 8, 2015

Contributor

Thanks for the suggestion, I'll do it that way.

@amenonsen

amenonsen Sep 8, 2015

Contributor

Thanks for the suggestion, I'll do it that way.

@bcoca bcoca added this to the v2 milestone Sep 8, 2015

@halberom

This comment has been minimized.

Show comment
Hide comment
@halberom

halberom Sep 8, 2015

Contributor

tested with become using sudo.

works correctly with

---
- hosts: centos66
  remote_user: vagrant
  gather_facts: no
  tasks:
    - name: should fail
      shell: cat /etc/shadow
      ignore_errors: yes

- hosts: centos66
  remote_user: vagrant
  become_user: root
  become: yes
  gather_facts: no
  tasks:
    - name: should work
      shell: cat /etc/shadow
      ignore_errors: yes

- hosts: centos66
  remote_user: vagrant
  gather_facts: no
  tasks:
    - name: should work
      shell: cat /etc/shadow
      ignore_errors: yes
      become_user: root
      become: yes

However in another playbook, it fails on the very first task after gathering facts.

# inv/group_vars/tag_Loc_vagrant 
---
# variables for the vagrant location
ansible_ssh_user: vagrant
ansible_ssh_private_key_file: '~/.vagrant.d/insecure_private_key'
ansible_become: yes
ansible_become_user: root
$ ansible-playbook --version
ansible-playbook 2.0.0 (ssh_locking 41225dd194) last updated 2015/09/08 18:50:43 (GMT +100)

TASK [lnbis_stdbase : ensure resolv.conf is setup] *****************************
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r)
<192.168.35.16> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441737311.75-174302916007763" && echo "$HOME/.ansible/tmp/ansible-tmp-1441737311.75-174302916007763")
CONNECTION: pid 35743 waiting for lock on 9
CONNECTION: pid 35743 acquired lock on 9
CONNECTION: pid 35743 released lock on 9
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r)
<192.168.35.16> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.35.16 /bin/sh -c 'sudo -H -n -S -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-vasgavmzsunypsypwexpzgijhlupybsk; rc=flag; [ -r /etc/resolv.conf ] || rc=2; [ -f /etc/resolv.conf ] || rc=1; [ -d /etc/resolv.conf ] && rc=3; python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc}  "/etc/resolv.conf && exit 0; (python -c '"'"'"'"'"'"'"'"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (python -c '"'"'"'"'"'"'"'"'import sha; BLOCKSIZE = 65536; hasher = sha.sha();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (echo '"'"'"'"'"'"'"'"'0  '"'"'"'"'"'"'"'"'/etc/resolv.conf)'"'"''
CONNECTION: pid 35743 waiting for lock on 9
CONNECTION: pid 35743 acquired lock on 9
CONNECTION: pid 35743 released lock on 9
<192.168.35.16> PUT /var/folders/js/2jgvltb55bj5_yd53djlzxzm0000gn/T/tmpf91Hr5 TO /home/vagrant/.ansible/tmp/ansible-tmp-1441737311.75-174302916007763/source
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r)
<192.168.35.16> SSH: EXEC sftp -b - -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r [192.168.35.16]
CONNECTION: pid 35743 waiting for lock on 9
CONNECTION: pid 35743 acquired lock on 9
CONNECTION: pid 35743 released lock on 9
fatal: [resolve-dev-vagrant-001]: FAILED! => {"failed": true, "msg": "ERROR! Connection error waiting for privilege escalation: "}

vs devel

$ ansible-playbook --version
ansible-playbook 2.0.0 (devel 1d7b493db3) last updated 2015/09/08 19:12:00 (GMT +100)

TASK [lnbis_stdbase : ensure resolv.conf is setup] *****************************
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r")
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383" && echo "$HOME/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383")
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 /bin/sh -c 'sudo -H -n -S -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-jwvmncnklgyywbjgmhfqidezwzibfkgs; rc=flag; [ -r /etc/resolv.conf ] || rc=2; [ -f /etc/resolv.conf ] || rc=1; [ -d /etc/resolv.conf ] && rc=3; python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc}  "/etc/resolv.conf && exit 0; (python -c '"'"'"'"'"'"'"'"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (python -c '"'"'"'"'"'"'"'"'import sha; BLOCKSIZE = 65536; hasher = sha.sha();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (echo '"'"'"'"'"'"'"'"'0  '"'"'"'"'"'"'"'"'/etc/resolv.conf)'"'"''
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> PUT /var/folders/js/2jgvltb55bj5_yd53djlzxzm0000gn/T/tmpnommN5 TO /home/vagrant/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383/source
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198" && echo "$HOME/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198")
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> PUT /var/folders/js/2jgvltb55bj5_yd53djlzxzm0000gn/T/tmpdq7aJS TO /home/vagrant/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198/copy
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 /bin/sh -c 'sudo -H -n -S -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-avatuboikwvxqnalyawduhcinmghknnl; LANG=C LC_MESSAGES=C LC_CTYPE=C /usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198/copy; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198/" > /dev/null 2>&1'"'"''
changed: [resolve-dev-vagrant-001] => {"changed": true, "checksum": "98082f4964610ca817d3cf979f84f2a22245103c", "dest": "/etc/resolv.conf", "gid": 0, "group": "root", "md5sum": "bd73bebbc2fb85053dbb7f3a1bfa628d", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:net_conf_t:s0", "size": 376, "src": "/home/vagrant/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383/source", "state": "file", "uid": 0}
Contributor

halberom commented Sep 8, 2015

tested with become using sudo.

works correctly with

---
- hosts: centos66
  remote_user: vagrant
  gather_facts: no
  tasks:
    - name: should fail
      shell: cat /etc/shadow
      ignore_errors: yes

- hosts: centos66
  remote_user: vagrant
  become_user: root
  become: yes
  gather_facts: no
  tasks:
    - name: should work
      shell: cat /etc/shadow
      ignore_errors: yes

- hosts: centos66
  remote_user: vagrant
  gather_facts: no
  tasks:
    - name: should work
      shell: cat /etc/shadow
      ignore_errors: yes
      become_user: root
      become: yes

However in another playbook, it fails on the very first task after gathering facts.

# inv/group_vars/tag_Loc_vagrant 
---
# variables for the vagrant location
ansible_ssh_user: vagrant
ansible_ssh_private_key_file: '~/.vagrant.d/insecure_private_key'
ansible_become: yes
ansible_become_user: root
$ ansible-playbook --version
ansible-playbook 2.0.0 (ssh_locking 41225dd194) last updated 2015/09/08 18:50:43 (GMT +100)

TASK [lnbis_stdbase : ensure resolv.conf is setup] *****************************
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r)
<192.168.35.16> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441737311.75-174302916007763" && echo "$HOME/.ansible/tmp/ansible-tmp-1441737311.75-174302916007763")
CONNECTION: pid 35743 waiting for lock on 9
CONNECTION: pid 35743 acquired lock on 9
CONNECTION: pid 35743 released lock on 9
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r)
<192.168.35.16> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.35.16 /bin/sh -c 'sudo -H -n -S -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-vasgavmzsunypsypwexpzgijhlupybsk; rc=flag; [ -r /etc/resolv.conf ] || rc=2; [ -f /etc/resolv.conf ] || rc=1; [ -d /etc/resolv.conf ] && rc=3; python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc}  "/etc/resolv.conf && exit 0; (python -c '"'"'"'"'"'"'"'"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (python -c '"'"'"'"'"'"'"'"'import sha; BLOCKSIZE = 65536; hasher = sha.sha();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (echo '"'"'"'"'"'"'"'"'0  '"'"'"'"'"'"'"'"'/etc/resolv.conf)'"'"''
CONNECTION: pid 35743 waiting for lock on 9
CONNECTION: pid 35743 acquired lock on 9
CONNECTION: pid 35743 released lock on 9
<192.168.35.16> PUT /var/folders/js/2jgvltb55bj5_yd53djlzxzm0000gn/T/tmpf91Hr5 TO /home/vagrant/.ansible/tmp/ansible-tmp-1441737311.75-174302916007763/source
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r)
<192.168.35.16> SSH: EXEC sftp -b - -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r [192.168.35.16]
CONNECTION: pid 35743 waiting for lock on 9
CONNECTION: pid 35743 acquired lock on 9
CONNECTION: pid 35743 released lock on 9
fatal: [resolve-dev-vagrant-001]: FAILED! => {"failed": true, "msg": "ERROR! Connection error waiting for privilege escalation: "}

vs devel

$ ansible-playbook --version
ansible-playbook 2.0.0 (devel 1d7b493db3) last updated 2015/09/08 19:12:00 (GMT +100)

TASK [lnbis_stdbase : ensure resolv.conf is setup] *****************************
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: default arguments: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.35.16> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r")
<192.168.35.16> SSH: ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set: (-o)(IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key")
<192.168.35.16> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<192.168.35.16> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set: (-o)(User=vagrant)
<192.168.35.16> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383" && echo "$HOME/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383")
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 /bin/sh -c 'sudo -H -n -S -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-jwvmncnklgyywbjgmhfqidezwzibfkgs; rc=flag; [ -r /etc/resolv.conf ] || rc=2; [ -f /etc/resolv.conf ] || rc=1; [ -d /etc/resolv.conf ] && rc=3; python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc}  "/etc/resolv.conf && exit 0; (python -c '"'"'"'"'"'"'"'"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (python -c '"'"'"'"'"'"'"'"'import sha; BLOCKSIZE = 65536; hasher = sha.sha();
afile = open("'"'"'"'"'"'"'"'"'/etc/resolv.conf'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
    hasher.update(buf)
    buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (echo '"'"'"'"'"'"'"'"'0  '"'"'"'"'"'"'"'"'/etc/resolv.conf)'"'"''
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> PUT /var/folders/js/2jgvltb55bj5_yd53djlzxzm0000gn/T/tmpnommN5 TO /home/vagrant/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383/source
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198" && echo "$HOME/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198")
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> PUT /var/folders/js/2jgvltb55bj5_yd53djlzxzm0000gn/T/tmpdq7aJS TO /home/vagrant/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198/copy
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 192.168.35.16 /bin/sh -c 'sudo -H -n -S -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-avatuboikwvxqnalyawduhcinmghknnl; LANG=C LC_MESSAGES=C LC_CTYPE=C /usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198/copy; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1441737578.58-4008187864198/" > /dev/null 2>&1'"'"''
changed: [resolve-dev-vagrant-001] => {"changed": true, "checksum": "98082f4964610ca817d3cf979f84f2a22245103c", "dest": "/etc/resolv.conf", "gid": 0, "group": "root", "md5sum": "bd73bebbc2fb85053dbb7f3a1bfa628d", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:net_conf_t:s0", "size": 376, "src": "/home/vagrant/.ansible/tmp/ansible-tmp-1441737578.46-67285424196383/source", "state": "file", "uid": 0}
@amenonsen

This comment has been minimized.

Show comment
Hide comment
@amenonsen

amenonsen Sep 9, 2015

Contributor

@halberom many thanks for testing. This is caused by a confusion about what sudoable actually means; I had thought it (meant "this task wants privilege escalation", and therefore) wouldn't be set for put_file() and fetch_file() (i.e. scp/sftp), but subsequent discussions with @bcoca made it clear that this isn't the case. I'll fix it.

Contributor

amenonsen commented Sep 9, 2015

@halberom many thanks for testing. This is caused by a confusion about what sudoable actually means; I had thought it (meant "this task wants privilege escalation", and therefore) wouldn't be set for put_file() and fetch_file() (i.e. scp/sftp), but subsequent discussions with @bcoca made it clear that this isn't the case. I'll fix it.

@amenonsen

This comment has been minimized.

Show comment
Hide comment
@amenonsen

amenonsen Sep 10, 2015

Contributor

@halberom I've pushed a revised set of changes. Does this work for you?

Contributor

amenonsen commented Sep 10, 2015

@halberom I've pushed a revised set of changes. Does this work for you?

@halberom

This comment has been minimized.

Show comment
Hide comment
@halberom

halberom Sep 10, 2015

Contributor

hmmm, I now get

$ ansible-playbook --version
ansible-playbook 2.0.0 (ssh_locking 9e95d22c43) last updated 2015/09/10 09:37:42 (GMT +100)
TASK [setup] *******************************************************************
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441874514.78-29123847111331" && echo "$HOME/.ansible/tmp/ansible-tmp-1441874514.78-29123847111331")
CONNECTION: pid 77872 waiting for lock on 9
CONNECTION: pid 77872 acquired lock on 9
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/Users/glynch/git/ansible/lib/ansible/executor/process/worker.py", line 119, in run
    executor_result = TaskExecutor(host, task, job_vars, new_play_context, self._new_stdin, self._loader, shared_loader_obj).run()
  File "/Users/glynch/git/ansible/lib/ansible/executor/task_executor.py", line 119, in run
    res = self._execute()
  File "/Users/glynch/git/ansible/lib/ansible/executor/task_executor.py", line 328, in _execute
    result = self._handler.run(task_vars=variables)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/normal.py", line 26, in run
    results = self._execute_module(tmp=tmp, task_vars=task_vars)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/__init__.py", line 357, in _execute_module
    tmp = self._make_tmp_path()
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/__init__.py", line 173, in _make_tmp_path
    result = self._low_level_execute_command(cmd, None, sudoable=False)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/__init__.py", line 461, in _low_level_execute_command
    rc, stdin, stdout, stderr = self._connection.exec_command(cmd, tmp, in_data=in_data, sudoable=sudoable)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/connections/ssh.py", line 268, in exec_command
    raise e
UnboundLocalError: local variable 'awaiting' referenced before assignment
Contributor

halberom commented Sep 10, 2015

hmmm, I now get

$ ansible-playbook --version
ansible-playbook 2.0.0 (ssh_locking 9e95d22c43) last updated 2015/09/10 09:37:42 (GMT +100)
TASK [setup] *******************************************************************
<192.168.35.16> ESTABLISH SSH CONNECTION FOR USER: vagrant
<192.168.35.16> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/Users/glynch/.vagrant.d/insecure_private_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/Users/glynch/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.35.16 (umask 22 && mkdir -p "$HOME/.ansible/tmp/ansible-tmp-1441874514.78-29123847111331" && echo "$HOME/.ansible/tmp/ansible-tmp-1441874514.78-29123847111331")
CONNECTION: pid 77872 waiting for lock on 9
CONNECTION: pid 77872 acquired lock on 9
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/Users/glynch/git/ansible/lib/ansible/executor/process/worker.py", line 119, in run
    executor_result = TaskExecutor(host, task, job_vars, new_play_context, self._new_stdin, self._loader, shared_loader_obj).run()
  File "/Users/glynch/git/ansible/lib/ansible/executor/task_executor.py", line 119, in run
    res = self._execute()
  File "/Users/glynch/git/ansible/lib/ansible/executor/task_executor.py", line 328, in _execute
    result = self._handler.run(task_vars=variables)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/normal.py", line 26, in run
    results = self._execute_module(tmp=tmp, task_vars=task_vars)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/__init__.py", line 357, in _execute_module
    tmp = self._make_tmp_path()
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/__init__.py", line 173, in _make_tmp_path
    result = self._low_level_execute_command(cmd, None, sudoable=False)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/action/__init__.py", line 461, in _low_level_execute_command
    rc, stdin, stdout, stderr = self._connection.exec_command(cmd, tmp, in_data=in_data, sudoable=sudoable)
  File "/Users/glynch/git/ansible/lib/ansible/plugins/connections/ssh.py", line 268, in exec_command
    raise e
UnboundLocalError: local variable 'awaiting' referenced before assignment
@halberom

This comment has been minimized.

Show comment
Hide comment
@halberom

halberom Sep 10, 2015

Contributor

Works now, only tried for become_method: sudo, become_user: root though. And I've not tested against multiple hosts.

Contributor

halberom commented Sep 10, 2015

Works now, only tried for become_method: sudo, become_user: root though. And I've not tested against multiple hosts.

@jimi-c

This comment has been minimized.

Show comment
Hide comment
@jimi-c

jimi-c Sep 10, 2015

Member

@amenonsen pinged in IRC, noting here as well.

This serializes very badly when hosts are unreachable (basically no other connections can be attempted until the unreachable host times out). Would it break things to move the initial lock down below the initial Popen?

https://gist.github.com/jimi-c/58a45c314841b0c2392b

This reduces the delay from unreachable hosts considerably, however I'm pretty sure this won't work as the lock needs to be in place for the known hosts check (if I remember right).

Member

jimi-c commented Sep 10, 2015

@amenonsen pinged in IRC, noting here as well.

This serializes very badly when hosts are unreachable (basically no other connections can be attempted until the unreachable host times out). Would it break things to move the initial lock down below the initial Popen?

https://gist.github.com/jimi-c/58a45c314841b0c2392b

This reduces the delay from unreachable hosts considerably, however I'm pretty sure this won't work as the lock needs to be in place for the known hosts check (if I remember right).

@amenonsen

This comment has been minimized.

Show comment
Hide comment
@amenonsen

amenonsen Sep 11, 2015

Contributor

You're right, it won't work to just move the lock down, but I can acquire the lock once I see a 'Connection established' message from ssh. I'm working on that now, I just have to figure out the corresponding entry-point for a multiplexed connection. Will resubmit shortly.

Contributor

amenonsen commented Sep 11, 2015

You're right, it won't work to just move the lock down, but I can acquire the lock once I see a 'Connection established' message from ssh. I'm working on that now, I just have to figure out the corresponding entry-point for a multiplexed connection. Will resubmit shortly.

@amenonsen

This comment has been minimized.

Show comment
Hide comment
@amenonsen

amenonsen Sep 13, 2015

Contributor

So I wrote the code to implement delayed lock acquisition, and then realised that it can't possibly be made to work—once we start multiple ssh processes in parallel, they'll happily trample over each other's host key prompts no matter what locks we hold (obviously; I don't know why it took me so long to realise that).

So the choices are to implement the locking and take the performance hit, or forget the locking and live with the status quo: @jimi-c on IRC said that he preferred the latter (due to the magnitude of the performance hit with serialising connections to eventually-unreachable hosts).

I've removed the locking code and resubmitted a version that has only the code reorganisation and the state machine cleanups. I believe this represents a significant improvement to the current code, and am hoping that it will be considered for v2 on their own merits even without locking.

For the record, this code fixes various minor problems in the old code: (a) it doesn't discard an entire chunk of program output when it sees the prompt on one line therein, (b) it won't mistake an error containing the echo BECOME-SUCCESS… command for actual success, (c) it won't write initial data into a password prompt if the password is wrong, (d) it can deal with partial reads cleanly, (e) it stops examining output line by line once it's done with the privilege escalation prompt, etc. I think it's also significantly more maintainable due to the improved structure and detailed comments.

Contributor

amenonsen commented Sep 13, 2015

So I wrote the code to implement delayed lock acquisition, and then realised that it can't possibly be made to work—once we start multiple ssh processes in parallel, they'll happily trample over each other's host key prompts no matter what locks we hold (obviously; I don't know why it took me so long to realise that).

So the choices are to implement the locking and take the performance hit, or forget the locking and live with the status quo: @jimi-c on IRC said that he preferred the latter (due to the magnitude of the performance hit with serialising connections to eventually-unreachable hosts).

I've removed the locking code and resubmitted a version that has only the code reorganisation and the state machine cleanups. I believe this represents a significant improvement to the current code, and am hoping that it will be considered for v2 on their own merits even without locking.

For the record, this code fixes various minor problems in the old code: (a) it doesn't discard an entire chunk of program output when it sees the prompt on one line therein, (b) it won't mistake an error containing the echo BECOME-SUCCESS… command for actual success, (c) it won't write initial data into a password prompt if the password is wrong, (d) it can deal with partial reads cleanly, (e) it stops examining output line by line once it's done with the privilege escalation prompt, etc. I think it's also significantly more maintainable due to the improved structure and detailed comments.

@amenonsen

This comment has been minimized.

Show comment
Hide comment
@amenonsen

amenonsen Sep 17, 2015

Contributor

Update: I've found one situation where the old code definitely does "hang on sudo", and it's common enough that it's plausible that it's at the root of some of the hanging bugs that people have reported in the past. If you run ssh with -v, then the sudo prompt (which isn't newline-terminated) often shows up with an ssh debug2 message appended to it, and the old code checked using .endswith(prompt), which failed to detect the prompt (and thus kept waiting).

Contributor

amenonsen commented Sep 17, 2015

Update: I've found one situation where the old code definitely does "hang on sudo", and it's common enough that it's plausible that it's at the root of some of the hanging bugs that people have reported in the past. If you run ssh with -v, then the sudo prompt (which isn't newline-terminated) often shows up with an ssh debug2 message appended to it, and the old code checked using .endswith(prompt), which failed to detect the prompt (and thus kept waiting).

@amenonsen amenonsen changed the title from Implement locking/unlocking in ssh.py to Improve ssh.py organisation and state machine Sep 19, 2015

amenonsen added some commits Sep 4, 2015

Reorganise ssh.py to cleanly separate responsibilities
The main exec_command/put_file/fetch_file methods now _build_command and
call _run to handle input from/output to the ssh process. The purpose is
to bring connection handling together in one place so that the locking
doesn't have to be split across functions.

Note that this doesn't change the privilege escalation and connection IO
code at all—just puts it all into one function.

Most of the changes are just moving code from one place to another (e.g.
from _connect to _build_command, from _exec_command and _communicate to
_run), but there are some other notable changes:

1. We test for the existence of sshpass the first time we need to use
   password authentication, and remember the result.
2. We set _persistent in _build_command if we're using ControlPersist,
   for later use in close(). (The detection could be smarter.)
3. Some apparently inadvertent inconsistencies between put_file and
   fetch_file (e.g. argument quoting, sftp -b use) have been removed.

Also reorders functions into a logical sequence, removes unused imports
and functions, etc.

Aside: the high-level EXEC/PUT/FETCH description should really be logged
from ConnectionBase, while individual subclasses log transport-specific
details.
Implement ssh connection handling as a state machine
The event loop (even after it was brought into one place in _run in the
previous commit) was hard to follow. The states and transitions weren't
clear or documented, and the privilege escalation code was non-blocking
while the rest was blocking.

Now we have a state machine with four states: awaiting_prompt,
awaiting_escalation, ready_to_send (initial data), and awaiting_exit.
The actions in each state and the transitions between then are clearly
documented.

The check_incorrect_password() method no longer checks for empty strings
(since they will always match), and check_become_success() uses equality
rather than a substring match to avoid thinking an echoed command is an
indication of successful escalation. Also adds a check_missing_password
connection method to detect the error from sudo -n/doas -n.
@jimi-c

This comment has been minimized.

Show comment
Hide comment
@jimi-c

jimi-c Sep 23, 2015

Member

This was merged in, but due to the way I did it Github isn't picking it up, so closing manually.

Thanks!

Member

jimi-c commented Sep 23, 2015

This was merged in, but due to the way I did it Github isn't picking it up, so closing manually.

Thanks!

@jimi-c jimi-c closed this Sep 23, 2015

@whereismyjetpack

This comment has been minimized.

Show comment
Hide comment
@whereismyjetpack

whereismyjetpack Sep 27, 2015

this commit seems to be causing issues in my environment, specifically line 132 of ssh.py.

-q is being appended to sftp, even though -q is not an option.

example running the ping module:
ansible git -m ping Hipchat plugin is disabled, please configure Timer plugin is active. git | FAILED! => { "failed": true, "msg": "ERROR! failed to transfer file to /root/.ansible/tmp/ansible-tmp-1443314208.6-177508567201448/ping:\n\nsftp: illegal option -- q\nusage: sftp [-1Cv] [-B buffer_size] [-b batchfile] [-F ssh_config]\n [-o ssh_option] [-P sftp_server_path] [-R num_requests]\n [-S program] [-s subsystem | sftp_server] host\n sftp [user@]host[:file ...]\n sftp [user@]host[:dir[/]]\n sftp -b batchfile [user@]host\n" }

changing the if block to the following seems to fix the issue:

        if self._play_context.verbosity > 3:
            self._command += ['-vvv']
        elif binary == 'ssh':
            self._command += ['-q']

this seems sane enough, only append -q for the 'ssh' binary -- however I'm not sure if there is a better way to do this.

Please let me know if you need any more information
Thanks,
Dann

this commit seems to be causing issues in my environment, specifically line 132 of ssh.py.

-q is being appended to sftp, even though -q is not an option.

example running the ping module:
ansible git -m ping Hipchat plugin is disabled, please configure Timer plugin is active. git | FAILED! => { "failed": true, "msg": "ERROR! failed to transfer file to /root/.ansible/tmp/ansible-tmp-1443314208.6-177508567201448/ping:\n\nsftp: illegal option -- q\nusage: sftp [-1Cv] [-B buffer_size] [-b batchfile] [-F ssh_config]\n [-o ssh_option] [-P sftp_server_path] [-R num_requests]\n [-S program] [-s subsystem | sftp_server] host\n sftp [user@]host[:file ...]\n sftp [user@]host[:dir[/]]\n sftp -b batchfile [user@]host\n" }

changing the if block to the following seems to fix the issue:

        if self._play_context.verbosity > 3:
            self._command += ['-vvv']
        elif binary == 'ssh':
            self._command += ['-q']

this seems sane enough, only append -q for the 'ssh' binary -- however I'm not sure if there is a better way to do this.

Please let me know if you need any more information
Thanks,
Dann

@jimi-c

This comment has been minimized.

Show comment
Hide comment
@jimi-c

jimi-c Sep 27, 2015

Member

@whereismyjetpack pushed that change in commit 4cd810a. If you continue seeing problems with this, open another issue.

Thanks!

Member

jimi-c commented Sep 27, 2015

@whereismyjetpack pushed that change in commit 4cd810a. If you continue seeing problems with this, open another issue.

Thanks!

@amenonsen

This comment has been minimized.

Show comment
Hide comment
@amenonsen

amenonsen Sep 27, 2015

Contributor

@whereismyjetpack that's a bit odd, my sftp (from openssh 6.6p1) accepts (and the manpage documents) a -q option. Same with scp.

Contributor

amenonsen commented Sep 27, 2015

@whereismyjetpack that's a bit odd, my sftp (from openssh 6.6p1) accepts (and the manpage documents) a -q option. Same with scp.

@whereismyjetpack

This comment has been minimized.

Show comment
Hide comment
@whereismyjetpack

whereismyjetpack Sep 27, 2015

we have an older, RHEL 6 control host and the version of sftp that ships with RHEL 6 does not have that flag. I'm not sure if we want to make older versions of openssh obsolete, I presume RHEL 6 is still pretty prevalent. (Production phase does not end until November 30, 2020)

we have an older, RHEL 6 control host and the version of sftp that ships with RHEL 6 does not have that flag. I'm not sure if we want to make older versions of openssh obsolete, I presume RHEL 6 is still pretty prevalent. (Production phase does not end until November 30, 2020)

amenonsen added a commit to amenonsen/ansible that referenced this pull request Nov 28, 2015

Use connection locking to protect against competing host key prompts
We acquire the connection lock before executing ssh, and release it as
soon as the unknown host key prompt is negotiated, or we can be sure it
won't be issued at all. This fixes the problem where many prompts pile
up and compete for input.

Unfortunately, we can only lock against other connections: there's no
output_lockfile that can prevent output from other subsystems. See the
FIXME note.

This problem was the original motivation for PR #12276, but although
the bulk of that PR was merged, the locking changes were not.

Fixes #13318

@ansibot ansibot added bug and removed bugfix_pull_request labels Mar 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment