-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement tunnelling #38
Comments
Anonymous () posted: The incredible monkeypatch in the attached file allows some kind of tunnelling. First of all, there would be no need to change the network.connect() function if authentication happened without user interaction (no password prompt). I think I cannot use the same connect while forwarding because I need to create a direct-tcpip connection and recreate it every time I have an error while trying to connect, I suspect paramiko doesn't support some reset packet from SSH, it sends a warning about not supporting type 3. The second monkeypatch done is to SSHClient to make it accept an already created socket rather than creating one of its own. The system is actually passing an already setup direct-tcpip Channel as a socket, and it can do that because Channel and socket apparently share the same interface so the Transport doesn't freak out. Then I monkeypatch the ConnectionCache to make it look for a gateway option in the env dictionary, since fab doesn't support it from command line it has to be added in a config file, the format is the same for hosts username@host:port. When the gateway option is present it would then try to connect to the gateway and then to forward connect from it to the clients. And the last monkeypatch is related to #36 and replaces the connections dictionary in the files where it matters. It definitely needs more testing and I'm not even sure how to test all this stuff, specifically the actual forwarding... There are so many layers involved that it's a big project to get any of this in the corresponding project on its own... Hope it helps, also it might contain bugs (mainly not imported names), it's just a reference right now. on 2009-07-28 at 09:17pm EDT |
Valentino Volonghi (dialtone) posted: Side note... The patch is from me :) on 2009-07-28 at 09:21pm EDT |
Jeff Forcier (bitprophet) posted: See http://pypi.python.org/pypi/paraproxy -- apparently this is a drop-in patch to paramiko that allows ProxyCommand to work (so I assume it reads ssh_config too, haven't tested it myself yet.) Kudos to IRC user Favoretti for the tip. on 2011-06-06 at 05:10pm EDT |
Jeff Forcier (bitprophet) posted: A user (not sure if it's the same IRC user?) also posted this ML thread with patches, probably the same approach -- have not looked yet. on 2011-06-08 at 02:05am EDT |
Jeff Forcier (bitprophet) posted: Read up on ProxyCommand real quick since I wasn't entirely sure if it was the same thing as using gateways; it sounds like it is one of the more common ways of gatewaying/proxying through one SSH server to another. See e.g. this blog post about using e.g. netcat on the first hop (again, this technique is used in a number of places.) on 2011-07-03 at 06:03pm EDT |
Erwin Bolwidt (ebolwidt) posted: I tried the monkeypatch and it still works. (The quick way: add monkeypath.py to fabric folder, add "import monkeypatch" to main.py, and add an option to state.py so you can specify the SSH gateway that you want on the command line:
) I don't get any errors about not supporting type 3, so I guess paramiko resolved that since this issue was opened. If that can be resolved, then the monkeypatch could be integrated with the code easily (it only patches fabric, not paramiko, and is quite clean) on 2011-07-14 at 02:33am EDT |
Erwin Bolwidt (ebolwidt) posted: To make one point: the monkeypatch approach is a lot like the approach taken by capistrano. It allows the SSH gateway host to be specified in any way you like. on 2011-07-14 at 02:36am EDT |
Erwin Bolwidt (ebolwidt) posted: I transformed the monkeypatch into clean code modifications. See attached git diff file and the master branch of my github fork https://github.com/ebolwidt/fabric (tag "monkeypatch_normalized_sshgateway") On Unix, it works perfectly for me. Example: fabric --gateway some_ssh.host.com --host target_ssh.host.com who_am_i Running run("who am i") has the advantage of printing the hostname that you are logging in from, so you can test whether the gateway server is used correctly. Any two ssh hosts to which you have access suffices; the target host doesn't have to be unreachable from the source host. Of course you only need the gateway if the target host is not directly reachable, such as in a DMZ, but for testing there is no need for this. On my Windows installation with Python 2.7.1 I sometimes have a problem with hanging processes if I use the gateway feature. I would appreciate if someone else could try this feature on Windows to see if it works for them. on 2011-07-18 at 03:54am EDT |
I tried Erwin Bolwidt's patch on my OS X 10.6.8 box. It worked but with the same hang he reported on his Windows machine. |
ML patch came from a colleague of mine. Favoretti would be me then. :) If I can add something in favor of paraproxy is that using paraproxy doesn't really make it less flexible. If there are no proxies defined in the config - well, we don't use anything.Why not to combine ability to specify gateway and ability of using netcat-based multi-level proxying? |
FWIW, I'd really prefer the paraproxy/ProxyCommand approach, because we use this technique for all ssh sessions, not just fabric. Having to store this config in two places (fabric and ssh_config) is more complex and error-prone. Until this issue is resolved, is there a way to use paraproxy with fabric without modifying fabric? |
Ah, I see paramiko is no longer used, so paraproxy doesn't apply |
Used SSH lib is a fork of paramiko. Now IMO hacking paraproxy in would be just perfect. Just pull it in as a whole. :) |
+1 to what @mdz suggests, but i'd also like to see a ProxyCommand-type thing as a subclass-able ContextManager, allowing for things like:
Or as an arg to the proxy class itself:
|
Ah! Thanks @bitprophet! It wasn't terribly clear to me at first, but now i see it. Thanks! |
+1 for @favoretti The paraproxy approach would still be of much use for me as I already maintain an .ssh/config with a lot of ProxyCommands. If ssh is a fork of paramiko, it should hot be so hard to make it paraproxy-aware and it would suddenly agwin be usable for me (having an SSH gateway between office and production networks) Thanks, |
+1 Just to make it explicit to anyone who might not have read carefully enough to figure out how to use the monkeypatch now, while waiting for this to make its way into a release:
And that's all. Remote commands will now proxy correctly. If Awesome work! |
I really would like this. I wasn't able to get the monkey patch to work - it kept complaining about a missing password - and I need different proxies for different hosts anyway. However, the following replacement for run() seems to do the trick: @needs_host
def proxy_run(cmd):
host, port = env.host_string, 22
if ':' in host:
host, port = env.host_string.split(':')
local('ssh -p %s -A %s "%s" 2>&1 | sed "s/^/[%s] /"' % (port, host, cmd, env.host_string)) |
I +1 this with my entire being :) At work, in order to access the servers from the customers, we have to do a hop through an intermediate network. Of course we do this with tunnels, but having proxycommand support built into fabric would be a HUGE help. We have ways to log in directly to the customer's network but that is not completely reliable. |
+1 For reference, some of the existing ssh config is like such, with one host entry for the target system and then one host entry that applies the ProxyCommand (note ssh -W) when needed: Host bastionhost Host bastion-* 10.5.5.* !internal- Host bastion-theserver |
Here's a patch that implements the ProxyCommand in fabric's ssh library: bitprophet/ssh#34 |
Here's the change in fabric to take use the ssh change: #698 |
@clarete I just edited your comment to reflect this, but in future please use "#NNN" to refer to other tickets, not a hyperlink. See the 'Github Flavored Markdown' link above text fields for details. Thanks! |
Hi @bitprophet! Sure I'll follow your advice in my next comments! Sorry for that :) |
Hi, |
@MiLk: |
Paramiko clearinghouse ticket for the socket-add feature (which is the base required to implement direct-tcpip tunneling in Fabric, and is related to but distinct from ProxyCommand's requirements) is Paramiko #77. The ProxyCommand ticket over there is Paramiko #97. |
@clarete's link a few comments up will be my starting point for ProxyCommand stuff (he also provides the equivalent changes at the Paramiko level, see above). Starting point for the direct-tcpip approach is this compare view to @ebolwidt's fork. I'm about 75% done with a blog post outlining & comparing both solutions. |
Blog post 1st draft is up: Gateway Solutions in Paramiko & Fabric. Will now actually do code work and either write a followup post or edit that one, if the plan changes. |
Hi @bitprophet! Thank you for the very nice article! It explains the problem and the two possible solutions in a very clear and elegant way! :) Just two things I'd say about it are the following:
|
I have the direct-tcpip approach working at the Fabric level now (after merging the core 'overridden socket' feature in Paramiko). Need docs, then will commit that + move on to ProxyCommand which is a little more involved. Mostly unrelated: I re-checked #78 and offhand I don't think it is actually that strongly related, in the "tunneling non SSH traffic" sense. I've probably directed some folks there thinking it was about having gateway behavior itself be context-manager'd; for that, with this new feature, they can just use env.host_string = "realtarget"
run("this will connect directly to realtarget")
with settings(gateway='mygateway'):
run("this will connect to realtarget via mygateway")
run("this goes back to direct connections") EDIT: Though to make that fully workable I need to boost EDIT 2: I'm punting on that, it requires actually updating |
Totally missed @clarete's reply above, sorry! I did mention in the post that one of
|
Thank you for your answer @bitprophet and sorry for missing that part! :) |
Somewhere in this thicket of tickets (heh) I recall saying "when both ProxyCommand and env.gateway are set, ProxyCommand wins". However, after thinking about it I've come to the opposite conclusion:
So I'll set things up so |
Had a minor freakout because @clarete was a bad, bad contributor and put Python 2.6+ specific code in his pull request. Thankfully (It's ok @clarete, I forgive you, I need better contribution docs that absolve me of any blame when people forget the minimum Python version supported ;)) ProxyCommand support appears to be working now. Gonna bang on it a little more but I think we're shipping this puppy tomorrow, sometime after I vote. |
N.B. What's not included yet and will need adding in 1.6 or later, if possible:
|
Closing this because it's, like, done now and merged into master. Above items to be spun off into real tickets later on. |
Hey @bitprophet! Sorry for the 2.6 specific code! In the first place I'm not actually a big fan of subprocess! (you can see why I don't like it here. I just couldn't use it's I was just too focused on the task that I forgot this very important detail!! Thanks for the heads up! |
@clarete -- no worries, I was just poking fun at you :) Thanks again for all the work on this! |
@bitprophet: Hi.. I am trying to use fabric module to solve this tunneling: |
@puppet-py Probably a better question for the mailing list, not a comment on a defunct ticket :) I personally have not done any multi-hop tunneling like that, so not sure how well it'll work. |
Description
It should be possible to tunnel all the commands through a single entry point in the network.
Originally submitted by Anonymous () on 2009-07-27 at 05:22pm EDT
Attachments
Relations
The text was updated successfully, but these errors were encountered: