Implement tunnelling

[bitprophet]

Description

It should be possible to tunnel all the commands through a single entry point in the network.

---

Originally submitted by Anonymous () on 2009-07-27 at 05:22pm EDT

Attachments

• monkeypatch.py
• fabric.patch

Relations

• Related to Consider forking Paramiko #275: Consider forking Paramiko
• Duplicated by Tunnelling SSH over HTTP Proxies #344: Tunnelling SSH over HTTP Proxies
• Related to Add Tunneling Context to Fab #78: Add Tunneling Context to Fab
• Related to SSH key forwarding #72: SSH key forwarding

[bitprophet]

Anonymous () posted:

---

The incredible monkeypatch in the attached file allows some kind of tunnelling. First of all, there would be no need to change the network.connect() function if authentication happened without user interaction (no password prompt). I think I cannot use the same connect while forwarding because I need to create a direct-tcpip connection and recreate it every time I have an error while trying to connect, I suspect paramiko doesn't support some reset packet from SSH, it sends a warning about not supporting type 3.

The second monkeypatch done is to SSHClient to make it accept an already created socket rather than creating one of its own. The system is actually passing an already setup direct-tcpip Channel as a socket, and it can do that because Channel and socket apparently share the same interface so the Transport doesn't freak out.

Then I monkeypatch the ConnectionCache to make it look for a gateway option in the env dictionary, since fab doesn't support it from command line it has to be added in a config file, the format is the same for hosts username@host:port. When the gateway option is present it would then try to connect to the gateway and then to forward connect from it to the clients.

And the last monkeypatch is related to #36 and replaces the connections dictionary in the files where it matters.

It definitely needs more testing and I'm not even sure how to test all this stuff, specifically the actual forwarding... There are so many layers involved that it's a big project to get any of this in the corresponding project on its own...

Hope it helps, also it might contain bugs (mainly not imported names), it's just a reference right now.

---

on 2009-07-28 at 09:17pm EDT

[bitprophet]

Valentino Volonghi (dialtone) posted:

---

Side note... The patch is from me :)

---

on 2009-07-28 at 09:21pm EDT

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

See http://pypi.python.org/pypi/paraproxy -- apparently this is a drop-in patch to paramiko that allows ProxyCommand to work (so I assume it reads ssh_config too, haven't tested it myself yet.)

Kudos to IRC user Favoretti for the tip.

IRC log pasted by goosemo

---

on 2011-06-06 at 05:10pm EDT

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

A user (not sure if it's the same IRC user?) also posted this ML thread with patches, probably the same approach -- have not looked yet.

---

on 2011-06-08 at 02:05am EDT

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

Read up on ProxyCommand real quick since I wasn't entirely sure if it was the same thing as using gateways; it sounds like it is one of the more common ways of gatewaying/proxying through one SSH server to another. See e.g. this blog post about using e.g. netcat on the first hop (again, this technique is used in a number of places.)

---

on 2011-07-03 at 06:03pm EDT

[bitprophet]

Erwin Bolwidt (ebolwidt) posted:

---

I tried the monkeypatch and it still works. (The quick way: add monkeypath.py to fabric folder, add "import monkeypatch" to main.py, and add an option to state.py so you can specify the SSH gateway that you want on the command line:

• make_option('-G', '--gateway',

•    default=None,
  

•    help="SSH gateway"
  

• ),

)

I don't get any errors about not supporting type 3, so I guess paramiko resolved that since this issue was opened.
I do get errors during shutdown; my guess is that the connection to the gateway is closed before the connection through the gateway to the target host is closed, which is the wrong way around and causes the thread that runs the connection to the target host to complain.

If that can be resolved, then the monkeypatch could be integrated with the code easily (it only patches fabric, not paramiko, and is quite clean)

---

on 2011-07-14 at 02:33am EDT

[bitprophet]

Erwin Bolwidt (ebolwidt) posted:

---

To make one point: the monkeypatch approach is a lot like the approach taken by capistrano. It allows the SSH gateway host to be specified in any way you like.
The other approach using paraproxy is less flexible in my opinion, as it requires you to specify the SSH gateway host by adding ProxyCommand directories to your SSH configuration files. The monkeypatch works differently and doesn't require that.

---

on 2011-07-14 at 02:36am EDT

[bitprophet]

Erwin Bolwidt (ebolwidt) posted:

---

I transformed the monkeypatch into clean code modifications. See attached git diff file and the master branch of my github fork https://github.com/ebolwidt/fabric (tag "monkeypatch_normalized_sshgateway")

On Unix, it works perfectly for me. Example:

fabric --gateway some_ssh.host.com --host target_ssh.host.com who_am_i

Running run("who am i") has the advantage of printing the hostname that you are logging in from, so you can test whether the gateway server is used correctly. Any two ssh hosts to which you have access suffices; the target host doesn't have to be unreachable from the source host. Of course you only need the gateway if the target host is not directly reachable, such as in a DMZ, but for testing there is no need for this.

On my Windows installation with Python 2.7.1 I sometimes have a problem with hanging processes if I use the gateway feature.
Symptoms: the disconnect from the target host doesn't complete, and if I ctrl-C the process from the command-line, it keeps running instead (using 13% CPU consistently) and keeps the network connection to the gateway open (but not to the target host from the gateway).
By running StraceNT on the process I can see that the process is busy with multi-threaded mutexes but no useful system calls being performed.
I'm inclined to think that this has to do with a broken installation on my end; my Windows setup is a bit hackety.

I would appreciate if someone else could try this feature on Windows to see if it works for them.

---

on 2011-07-18 at 03:54am EDT

[parente]

I tried Erwin Bolwidt's patch on my OS X 10.6.8 box. It worked but with the same hang he reported on his Windows machine.

[favoretti]

ML patch came from a colleague of mine. Favoretti would be me then. :)

If I can add something in favor of paraproxy is that using paraproxy doesn't really make it less flexible.
It will just honor whatever SSH proxy options are present in the config of the user running fabric at that particular moment.

If there are no proxies defined in the config - well, we don't use anything.Why not to combine ability to specify gateway and ability of using netcat-based multi-level proxying?

[mdz]

FWIW, I'd really prefer the paraproxy/ProxyCommand approach, because we use this technique for all ssh sessions, not just fabric. Having to store this config in two places (fabric and ssh_config) is more complex and error-prone.

Until this issue is resolved, is there a way to use paraproxy with fabric without modifying fabric?