Fabric hangs when using SSH proxy and paramiko 1.12 #1020

mkobos opened this Issue Nov 10, 2013 · 27 comments


None yet
mkobos commented Nov 10, 2013

[Maintainer edit: #1042 seems to be a dupe with its own big set of affected users.]

I'm using the most current Fabric version available in pypi: Fabric 1.8.0 with Paramiko 1.12 on Ubuntu 13.04. When I execute fab deploy using the following sample fabfile

#!/usr/bin/env python

from fabric.api import *

# Respect the proxy SSH host setting.
env.use_ssh_config = True 
env.hosts = ["my_host_name"]

def deploy():
    run("ls .")

Fabric executes the ls commnad, prints Done. and hangs indefinitely. When I interrupt it with ^C, it prints:

File "/usr/local/bin/fab", line 9, in <module>
  load_entry_point('Fabric==1.8.0', 'console_scripts', 'fab')()
File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 760, in main
File "/usr/local/lib/python2.7/dist-packages/fabric/network.py", line 597, in disconnect_all
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 350, in close
File "/usr/local/lib/python2.7/dist-packages/paramiko/transport.py", line 618, in close
File "/usr/local/lib/python2.7/dist-packages/paramiko/transport.py", line 1395, in stop_thread
File "/usr/lib/python2.7/threading.py", line 960, in join
File "/usr/lib/python2.7/threading.py", line 358, in wait

Note that it happens only when I use SSH proxy, i.e. this is the important part of the ~/.ssh/config file:

Host my_host_name
  User my_login
  ForwardAgent yes
  ProxyCommand ssh -W %h:%p my_proxy_host

A solution to this problem turned out to be downgrading paramiko library installed in my system to version 1.11.1 (downgrading to a little more current version 1.11.2 doesn't fix the problem).

lazzurs commented Nov 29, 2013


I can +1 this bug when using the proxycommand setting in my ssh config the latest version of paramiko hangs and does not close the connection but 1.11.1 does work fine.

However I would suggest this should be a bug for paramiko.

cholick commented Dec 17, 2013

Another +1 for this issue - it's creating a problem for me as well.

xanbuck commented Dec 17, 2013


dragon3 commented Dec 18, 2013

+1 ๐Ÿ˜„







xsb commented Feb 5, 2014


mgedmin commented Feb 11, 2014

Here's a paramiko bug report that claims to fix this: paramiko/paramiko#252


While I am poking at this in Paramiko, can anybody else who +1'd this verify that Paramiko 1.11.1 works OK and 1.11.2 is subject to the problem? Thanks!

@bitprophet bitprophet referenced this issue in paramiko/paramiko Feb 12, 2014

fix hang if using proxycommand #252


Definitely seems to be a patch submitted as part of 1.11.2, however that patch itself is fixing a different deadlock problem so we can't simply revert it. I'd like more info from y'all if I can get it, please!

  • Is this happening every time you run Fabric or is it intermittent?
  • Does it still happen when you switch from using ProxyCommand+ssh to using the Fabric gateway option (http://docs.fabfile.org/en/1.8/usage/env.html#gateway)?
    • NOTE: Using this instead of ProxyCommand is recommended regardless, as it's cleaner & leverages a native sshd forwarding feature. ProxyCommand is more for when you need to proxy over other protocols/programs.
mgedmin commented Feb 12, 2014

This happens every time.

This doesn't happen if I use fab --gateway.

(Unfortunately env.gateway is not a suitable workaround for me: fab -g fridge -H fridge task fails with a "paramiko.SSHException: SSH session not active". I'll file a bug.)


Thanks, and yea, was just asking to help pin down exactly what is affected by the threading problem.


Also now I am wondering how y'all are getting around #1046 o_O I hate new features, so much.

mgedmin commented Feb 12, 2014

Also now I am wondering how y'all are getting around #1046

Easy: pip install Fabric==1.8.0 ;)


Well that's the thing, I went back to check and the typo present in #1046 appeared to be consistent back to ~Fabric 1.5 :) Moot now, anyways.

I can confirm this issue on my end pretty consistently, so that is good, will let me test potential fixes.


Dear all,

Please try using this branch (it's based off of the 1.11 line, FYI) and see how it addresses the problem for you. It may cause things overall to be slower than normal, though that may be my paranoia at work.

Makes things terminate cleanly for me in a base case setup (ProxyCommand of ssh -W to machine A, run 'ls /' on machine B).

See Paramiko #252 discussion for details on what I dug up, if you care, but that branch is the result.


Pushed more related fixes to same branch just now. cc @mkobos @lazzurs @mcholick @abuckles @dragon3 @DavidWittman @samyscoub @giuseppe7 @zroadhouse-wsm and anybody else I missed. GITHUB NOTIFICATION PARANOIA ENABLED

dragon3 commented Feb 14, 2014

@bitprophet Thank you for notification!
I tried "fix-proxycommand-infinite-loop-252" branch,
and I confirmed all connection terminated cleanly, my problem was fixed! ๐Ÿ˜ƒ


@bitprophet Verified that 1.11.1 works and that 1.12.0 does not.


@bitprophet fix-proxycommand-infinite-loop-252 addressed my issues as well.



Awesome, thanks guys. Merged the branch for Paramiko 1.11.x and 1.12.x, will release those lines today if I can, then you can just grab latest Paramiko in your desired line and it will include this change.

New Fabric will be out around the same time but besides the typo fix mentioned above, nothing had to change to support the Paramiko-level fixes.

@bitprophet bitprophet closed this Feb 14, 2014
dragon3 commented Feb 15, 2014

I tried Fabric 1.8.2 + Paramiko 1.12.2, they work fine!!
@bitprophet Thank you!

mkobos commented Feb 15, 2014

I checked my script with the most current versions of Fabric (1.8.2) and Paramiko (1.12.2) and it works fine now. Thanks!

lepture commented Sep 29, 2014

It still hangs!


@lepture Can you open a new ticket with some details please? E.g. are you using the example snippets up top here; if not, what's your config & fabfile look like; etc. Also, please try to use multiple Paramiko versions (something below 1.15 for sure, as that made moderate changes) and see how that affects things.

(Again - in a new ticket, please & thanks!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment