Add Tunneling Context to Fab #78

Open
bitprophet opened this Issue Aug 19, 2011 · 13 comments

Projects

None yet

4 participants

@bitprophet
Member

Description

This would be very helpful for me because it would make it very easy
to for example connect to a remote MySQL server using my command line
MySQL client.

We could use a "with" context statement, something like:

with tunnel(local=3307, remote=3306):
    local('mysql --port=3007 --host=localhost' mydb < db/dbdump.sql')

This would eliminate the need to upload mysql dump file to the server
just to be able to run import.

Another application could be for administering Cherokee web servers.

Cherokee web admin by default is only accessible from the server that
it's running on. So you want to access the admin you have to tunnel
into the server and access the admin interface using a local port.
This could also be simplified with this functionality.

with tunnel(local=9090, remote=9090):
   sudo('cherokee-admin')
   prompt('Stop cherokee admin?')

That last line would keep the tunnel open until it's closed by providing input.


Originally submitted by Taras Mankovski (tarasm) on 2009-11-02 at 09:30am EST

Relations

  • Related to #38: Implement tunnelling
@bitprophet bitprophet was assigned Aug 19, 2011
@bitprophet
Member

Jeff Forcier (bitprophet) posted:


(modified description so code blocks were indented :))


on 2009-11-02 at 09:35am EST

@munhitsu
munhitsu commented Jan 9, 2012

It would be awesome if tunnel would also support the other way around. Meaning listening on remote and forwarding it into localhost/other locally available host:port.

@bitprophet
Member

@munhitsu I'm not sure how that's a use case for Fabric specifically, though. Can you elaborate?

@munhitsu

Imagine a DMZ setup which does not have an outbound http/proxy access.
Whole deployment goes through fabric.

In such case we could use fabric to tunnel local proxy through ssh, to be temporarily accessible for the host being just provisioned in DMZ. Right now I'm opening a separate ssh tunnel on second console, so that fabric works within "context" of this tunnel.

example usage:

with rev_tunnel(local=8080, remote=8080):
    sudo("http_proxy='http://localhost:8080' apt-get install -y puppet")
@bitprophet
Member

OK, so it is a fairly standard reverse tunnel setup after all I guess, and your desire re: Fabric is for it to handle doing the tunnel vs you having to run eg local(ssh -R ...).

I went back and forth on whether this is truly worth going into core, but it really would make a decent amount of sense for it to be supported in Fabric proper; the other solutions are hacky (eg some thread or subprocess running ssh -- how to do this well, make sure it shuts down when Fab does, etc) and I do see the validity of the use case (sharing local resources with the remote end during execution.)

Main roadblock is that I am not sure the SSH lib supports this yet; we'll have to figure that out and somebody would have to implement it if it doesn't. (I think that would be a good addition to said library, however, vs solidifying aforementioned ssh -R workaround.)

EDIT: #38 discusses implementation and patching and such. Might be best to actually close this and simply note there that when implemented, it should be possible to trigger with a context manager if possible.

@munhitsu

I fully agree that having it implemented so that we enter, leave contexts, or even worse multiple levels of contexts is tricky.

Regarding EDIT: Anything that keeps us closer to having this functionality is a good idea.

@bitprophet
Member

For now I'll leave this open, just to keep things granular. Assigned it to 1.4 so I don't forget, though. Chances are I will try to knock this out right as I am doing #38. Will update here when it's in master, thanks.

@bitprophet
Member

This is actually not as related to #38 as I thought, since its changes are solely about gatewaying the SSH traffic itself, not tunneling additional ports through the SSH connection. That will require a different (or at least, additional) solution. Punting for now, sorry :) (Meaning: still open, just not going to arrive in 1.5.)

@haridsv
Contributor
haridsv commented Apr 7, 2015

I recently needed this functionality to sync to a remote rsync server whose port is not directly reachable and found that the paramiko's forward.py demo code has sample code that I could use, so I came up with a solution that worked well for me and I submitted it as a patch for forward.py here: paramiko/paramiko#504/

We could add ForwardServer from that patch and have a local_tunnel() that simply returns an instance of ForwardServer. As per recommendation from @bitprophet on the pull request, I will work on a patch for Fabric.

@haridsv
Contributor
haridsv commented Apr 7, 2015

I actually didn't realize, but there is already a patch for local_tunnel(), though I am not fully sure about its status. What should I do?

@bitprophet
Member

@haridsv If you're able, testing it out & mentioning that you successfully used it, on that patch (#939), would help, otherwise just have patience :) thanks!

@naphthalene

Is there any plan to resolve this issue? There are two outstanding pull requests that together resolve this, but they haven't been reviewed? Any way we could speed this along?
#939, #1218

I've been using the code in #1218 with no problems.

@bitprophet
Member

#939 is still in a release bucket, I just punted it from the upcoming 1.11 because I needed to cut some stuff, but it'll get priority for the next feature cycle. (And it looks like #1218 supersedes #939 so I'll probably end up merging that & crediting #939 in the changelog.)

@renskiy renskiy referenced this issue in renskiy/fabricio Aug 5, 2016
Open

use docker-py to communicate with the Docker #21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment