Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
SSH config handling can be slow #110
The timing of the ProxyCommand support was impeccable, but I noticed that when I started using my SSH config, my Fabric runs were excruciatingly slow. I traced it to the use of socket.getfqdn() in config.SSHConfig._expand_variables. This is an admittedly convoluted fix, but it's much, much faster for me.
You do not have a local caching resolver (e.g bind)?
gethostbyname() will only return ipv4 addresses so this patch won't be compatible with ipv6.
The correct approach would probably be to fix your ipv6 configuration/have a local dns caching mechanism. While this might seem as a regression/bug this slowness should actually be apparent for other applications doing dns resolution too.
Olle, you're exactly right, it's the IPv6 lookups. I should have included more background.
I'm on a Mac, running 10.8.2, with no IPv6 anywhere in the network (unfortunately). Even if I disable IPv6 in network preferences or via the networksetup command, getaddrinfo with AF_UNSPEC will still try IPv6 lookups, and that of course is exactly how it's being called in the socket module.
I'm not seeing the problem with other applications, so I think that usage is the problem. For example, if I run the same command from my Fabric task via ssh directly, the timing is almost the same whether I use "AddressFamily inet" in my ~/.ssh/config or not.
I've revised my change to eliminate the IPv4-only functions and use getaddrinfo, trying IPv4 first, then IPv6. I'll understand if you don't want to embed that "improvement" over the standard library's logic, to work around one platform's weaknesses with IPv6, but on the other hand, if its results are correct, it will speed things up for a lot of people.
It's up to @bitprophet to merge the patches, I'm just a happy contributer. :)
The socket.has_ipv6 actually only returns true or false based on if the socket library has been compiled with ipv6 support, not if you actually have ipv6 addressing available/enabled.
It's unfortunate that getaddrinfo does not seem to honour the os configuration for python. While, to me, the code looks nice, it think it will break what's specified in http://www.ietf.org/rfc/rfc3484.txt and the ability to change the priority of lookups in /etc/gai.conf (for linux).
Perhaps a patch considering if AddressFamily has been set and using the appropriate lookup?
inet -> ipv4
Might not get the speed up you want by default, but at least paramiko will follow what the sysadmin have configured and it won't break the RFC.