Skip to content
This repository

add ssh tunnel support to qtconsole #686

Merged
merged 6 commits into from over 2 years ago

2 participants

Min RK Fernando Perez
Min RK
Owner

title pretty much covers it.

This adds init_ssh step to qtconsole app, which will use the ssh tunneling code used elsewhere to forward connections.

So, in addition to pasting the usual --existing..., just add --ssh=foo, and you should be set.

Fernando Perez
Owner

This is excellent, the only thing I'd like to see is a short section added to the qt console docs explaining users how to call it. It will save us from answering that question more than once on-list.

IPython/frontend/qt/console/qtconsoleapp.py
... ...
@@ -25,6 +26,7 @@ import sys
25 26
 from IPython.external.qt import QtGui
26 27
 from pygments.styles import get_all_styles
27 28
 
  29
+# from IPython.external.ssh import tunnel
1
Fernando Perez Owner
fperez added a note August 15, 2011

This should probably just be deleted, since it's commented out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Min RK
Owner

@fperez I fixed the comment you mentioned with a slightly more substantial change (I fixed the circular import altogether by making external.ssh as standalone as it should be in the first place). I'm working on the doc now.

Min RK
Owner

doc updated, can probably be merged

Fernando Perez
Owner

except right now it won't merge anymore, there's a conflict on ssh/tunnel.py. Can you have a look?

added some commits August 07, 2011
Min RK add ssh tunnel support to qtconsole fcf87c2
Min RK Remove IPython dependency in external.ssh
copy parallel.util.select_random_ports into external.ssh.tunnel

This lets external.ssh be moved to another project without IPython, only changing the pexpect import.

This also resolves a circular import in the qtconsole
34eefb9
Min RK increase default ssh tunnel timeout to 60 seconds
also expose timeout to tunnel_connection function
65d9f3e
Min RK add security / ssh tunnel notes to qtconsole docs d3a4730
Min RK fix key->kbd role name for keys in qt doc fc63d37
Min RK fix check/try typo in paramiko_tunnel c001961
Min RK
Owner

Roger - enginessh touched external.ssh as well, which was already merged. I rebased that one, because I thought this had already been merged. Now it should be clean.

Fernando Perez
Owner

Looking great, thanks! Merging now.

Fernando Perez fperez merged commit ba8f067 into from August 16, 2011
Fernando Perez fperez closed this August 16, 2011
Fernando Perez fperez referenced this pull request from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 6 unique commits by 1 author.

Aug 16, 2011
Min RK add ssh tunnel support to qtconsole fcf87c2
Min RK Remove IPython dependency in external.ssh
copy parallel.util.select_random_ports into external.ssh.tunnel

This lets external.ssh be moved to another project without IPython, only changing the pexpect import.

This also resolves a circular import in the qtconsole
34eefb9
Min RK increase default ssh tunnel timeout to 60 seconds
also expose timeout to tunnel_connection function
65d9f3e
Min RK add security / ssh tunnel notes to qtconsole docs d3a4730
Min RK fix key->kbd role name for keys in qt doc fc63d37
Min RK fix check/try typo in paramiko_tunnel c001961
This page is out of date. Refresh to see the latest.
47  IPython/external/ssh/tunnel.py
@@ -16,6 +16,7 @@
16 16
 from __future__ import print_function
17 17
 
18 18
 import os,sys, atexit
  19
+import socket
19 20
 from multiprocessing import Process
20 21
 from getpass import getpass, getuser
21 22
 import warnings
@@ -34,12 +35,32 @@
34 35
 except ImportError:
35 36
     pexpect = None
36 37
 
37  
-from IPython.parallel.util import select_random_ports
38  
-
39 38
 #-----------------------------------------------------------------------------
40 39
 # Code
41 40
 #-----------------------------------------------------------------------------
42 41
 
  42
+# select_random_ports copied from IPython.parallel.util
  43
+_random_ports = set()
  44
+
  45
+def select_random_ports(n):
  46
+    """Selects and return n random ports that are available."""
  47
+    ports = []
  48
+    for i in xrange(n):
  49
+        sock = socket.socket()
  50
+        sock.bind(('', 0))
  51
+        while sock.getsockname()[1] in _random_ports:
  52
+            sock.close()
  53
+            sock = socket.socket()
  54
+            sock.bind(('', 0))
  55
+        ports.append(sock)
  56
+    for i, sock in enumerate(ports):
  57
+        port = sock.getsockname()[1]
  58
+        sock.close()
  59
+        ports[i] = port
  60
+        _random_ports.add(port)
  61
+    return ports
  62
+
  63
+
43 64
 #-----------------------------------------------------------------------------
44 65
 # Check for passwordless login
45 66
 #-----------------------------------------------------------------------------
@@ -101,7 +122,7 @@ def _try_passwordless_paramiko(server, keyfile):
101 122
         return True
102 123
 
103 124
 
104  
-def tunnel_connection(socket, addr, server, keyfile=None, password=None, paramiko=None):
  125
+def tunnel_connection(socket, addr, server, keyfile=None, password=None, paramiko=None, timeout=60):
105 126
     """Connect a socket to an address via an ssh tunnel.
106 127
     
107 128
     This is a wrapper for socket.connect(addr), when addr is not accessible
@@ -110,12 +131,12 @@ def tunnel_connection(socket, addr, server, keyfile=None, password=None, paramik
110 131
     selected local port of the tunnel.
111 132
     
112 133
     """
113  
-    new_url, tunnel = open_tunnel(addr, server, keyfile=keyfile, password=password, paramiko=paramiko)
  134
+    new_url, tunnel = open_tunnel(addr, server, keyfile=keyfile, password=password, paramiko=paramiko, timeout=timeout)
114 135
     socket.connect(new_url)
115 136
     return tunnel
116 137
 
117 138
 
118  
-def open_tunnel(addr, server, keyfile=None, password=None, paramiko=None):
  139
+def open_tunnel(addr, server, keyfile=None, password=None, paramiko=None, timeout=60):
119 140
     """Open a tunneled connection from a 0MQ url.
120 141
     
121 142
     For use inside tunnel_connection.
@@ -136,10 +157,11 @@ def open_tunnel(addr, server, keyfile=None, password=None, paramiko=None):
136 157
         tunnelf = paramiko_tunnel
137 158
     else:
138 159
         tunnelf = openssh_tunnel
139  
-    tunnel = tunnelf(lport, rport, server, remoteip=ip, keyfile=keyfile, password=password)
  160
+    
  161
+    tunnel = tunnelf(lport, rport, server, remoteip=ip, keyfile=keyfile, password=password, timeout=timeout)
140 162
     return 'tcp://127.0.0.1:%i'%lport, tunnel
141 163
 
142  
-def openssh_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=15):
  164
+def openssh_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=60):
143 165
     """Create an ssh tunnel using command-line ssh that connects port lport
144 166
     on this machine to localhost:rport on server.  The tunnel
145 167
     will automatically close when not in use, remaining open
@@ -171,7 +193,9 @@ def openssh_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, pas
171 193
         password : str; 
172 194
             Your ssh password to the ssh server. Note that if this is left None,
173 195
             you will be prompted for it if passwordless key based login is unavailable.
174  
-    
  196
+        timeout : int [default: 60]
  197
+            The time (in seconds) after which no activity will result in the tunnel
  198
+            closing.  This prevents orphaned tunnels from running forever.
175 199
     """
176 200
     if pexpect is None:
177 201
         raise ImportError("pexpect unavailable, use paramiko_tunnel")
@@ -215,7 +239,7 @@ def _split_server(server):
215 239
         port = 22
216 240
     return username, server, port
217 241
 
218  
-def paramiko_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=15):
  242
+def paramiko_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=60):
219 243
     """launch a tunner with paramiko in a subprocess. This should only be used
220 244
     when shell ssh is unavailable (e.g. Windows).
221 245
     
@@ -250,13 +274,16 @@ def paramiko_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, pa
250 274
         password : str; 
251 275
             Your ssh password to the ssh server. Note that if this is left None,
252 276
             you will be prompted for it if passwordless key based login is unavailable.
  277
+        timeout : int [default: 60]
  278
+            The time (in seconds) after which no activity will result in the tunnel
  279
+            closing.  This prevents orphaned tunnels from running forever.
253 280
     
254 281
     """
255 282
     if paramiko is None:
256 283
         raise ImportError("Paramiko not available")
257 284
     
258 285
     if password is None:
259  
-        if not _check_passwordless_paramiko(server, keyfile):
  286
+        if not _try_passwordless_paramiko(server, keyfile):
260 287
             password = getpass("%s's password: "%(server))
261 288
 
262 289
     p = Process(target=_paramiko_tunnel, 
40  IPython/frontend/qt/console/qtconsoleapp.py
@@ -20,11 +20,15 @@
20 20
 import os
21 21
 import signal
22 22
 import sys
  23
+from getpass import getpass
23 24
 
24 25
 # System library imports
25 26
 from IPython.external.qt import QtGui
26 27
 from pygments.styles import get_all_styles
27 28
 
  29
+# external imports
  30
+from IPython.external.ssh import tunnel
  31
+
28 32
 # Local imports
29 33
 from IPython.config.application import boolean_flag
30 34
 from IPython.core.application import BaseIPythonApplication
@@ -34,6 +38,7 @@
34 38
 from IPython.frontend.qt.console.rich_ipython_widget import RichIPythonWidget
35 39
 from IPython.frontend.qt.console import styles
36 40
 from IPython.frontend.qt.kernelmanager import QtKernelManager
  41
+from IPython.parallel.util import select_random_ports
37 42
 from IPython.utils.traitlets import (
38 43
     Dict, List, Unicode, Int, CaselessStrEnum, CBool, Any
39 44
 )
@@ -219,6 +224,7 @@ def closeEvent(self, event):
219 224
 
220 225
     editor = 'IPythonWidget.editor',
221 226
     paging = 'ConsoleWidget.paging',
  227
+    ssh = 'IPythonQtConsoleApp.sshserver',
222 228
 )
223 229
 aliases.update(qt_aliases)
224 230
 # also scrub aliases from the frontend
@@ -266,6 +272,12 @@ class IPythonQtConsoleApp(BaseIPythonApplication):
266 272
         Consoles on other machines will be able to connect
267 273
         to the Kernel, so be careful!"""
268 274
     )
  275
+    
  276
+    sshserver = Unicode('', config=True,
  277
+        help="""The SSH server to use to connect to the kernel.""")
  278
+    sshkey = Unicode('', config=True,
  279
+        help="""Path to the ssh key to use for logging in to the ssh server.""")
  280
+    
269 281
     hb_port = Int(0, config=True,
270 282
         help="set the heartbeat port [default: random]")
271 283
     shell_port = Int(0, config=True,
@@ -322,7 +334,32 @@ def parse_command_line(self, argv=None):
322 334
                 key = a.lstrip('-').split('=')[0]
323 335
                 if key in qt_flags:
324 336
                     self.kernel_argv.remove(a)
325  
-
  337
+    
  338
+    def init_ssh(self):
  339
+        """set up ssh tunnels, if needed."""
  340
+        if not self.sshserver and not self.sshkey:
  341
+            return
  342
+        
  343
+        if self.sshkey and not self.sshserver:
  344
+            self.sshserver = self.ip
  345
+            self.ip=LOCALHOST
  346
+        
  347
+        lports = select_random_ports(4)
  348
+        rports = self.shell_port, self.iopub_port, self.stdin_port, self.hb_port
  349
+        self.shell_port, self.iopub_port, self.stdin_port, self.hb_port = lports
  350
+        
  351
+        remote_ip = self.ip
  352
+        self.ip = LOCALHOST
  353
+        self.log.info("Forwarding connections to %s via %s"%(remote_ip, self.sshserver))
  354
+        
  355
+        if tunnel.try_passwordless_ssh(self.sshserver, self.sshkey):
  356
+            password=False
  357
+        else:
  358
+            password = getpass("SSH Password for %s: "%self.sshserver)
  359
+        
  360
+        for lp,rp in zip(lports, rports):
  361
+            tunnel.ssh_tunnel(lp, rp, self.sshserver, remote_ip, self.sshkey, password)
  362
+        
326 363
     def init_kernel_manager(self):
327 364
         # Don't let Qt or ZMQ swallow KeyboardInterupts.
328 365
         signal.signal(signal.SIGINT, signal.SIG_DFL)
@@ -420,6 +457,7 @@ def init_colors(self):
420 457
 
421 458
     def initialize(self, argv=None):
422 459
         super(IPythonQtConsoleApp, self).initialize(argv)
  460
+        self.init_ssh()
423 461
         self.init_kernel_manager()
424 462
         self.init_qt_elements()
425 463
         self.init_colors()
167  docs/source/interactive/qtconsole.txt
@@ -28,10 +28,10 @@ is not yet configurable.
28 28
 
29 29
    Since the Qt console tries hard to behave like a terminal, by default it
30 30
    immediately executes single lines of input that are complete.  If you want
31  
-   to force multiline input, hit :key:`Ctrl-Enter` at the end of the first line
32  
-   instead of :key:`Enter`, and it will open a new line for input.  At any
  31
+   to force multiline input, hit :kbd:`Ctrl-Enter` at the end of the first line
  32
+   instead of :kbd:`Enter`, and it will open a new line for input.  At any
33 33
    point in a multiline block, you can force its execution (without having to
34  
-   go to the bottom) with :key:`Shift-Enter`.
  34
+   go to the bottom) with :kbd:`Shift-Enter`.
35 35
 
36 36
 ``%loadpy``
37 37
 ===========
@@ -213,6 +213,8 @@ blocking execution. The frontend can also know, via a heartbeat mechanism, that
213 213
 the kernel has died. This means that the frontend can safely restart the
214 214
 kernel.
215 215
 
  216
+.. _multiple_consoles:
  217
+
216 218
 Multiple Consoles
217 219
 *****************
218 220
 
@@ -225,11 +227,38 @@ like::
225 227
     [IPKernelApp] --existing --shell=60690 --iopub=44045 --stdin=38323 --hb=41797
226 228
 
227 229
 Other frontends can connect to your kernel, and share in the execution. This is
228  
-great for collaboration. The `-e` flag is for 'external'. Starting other
  230
+great for collaboration.  The ``--existing`` flag means connect to a kernel
  231
+that already exists.  Starting other
229 232
 consoles with that flag will not try to start their own, but rather connect to
230 233
 yours. Ultimately, you will not have to specify each port individually, but for
231 234
 now this copy-paste method is best.
232 235
 
  236
+You can even launch a standalone kernel, and connect and disconnect Qt Consoles
  237
+from various machines.  This lets you keep the same running IPython session
  238
+on your work machine (with matplotlib plots and everything), logging in from home,
  239
+cafés, etc.::
  240
+
  241
+    $> ipython kernel
  242
+    [IPKernelApp] To connect another client to this kernel, use:
  243
+    [IPKernelApp] --existing --shell=60690 --iopub=44045 --stdin=38323 --hb=41797
  244
+
  245
+This is actually exactly the same as the subprocess launched by the qtconsole, so
  246
+all the information about connecting to a standalone kernel is identical to that
  247
+of connecting to the kernel attached to a running console.
  248
+
  249
+.. _kernel_security:
  250
+
  251
+Security
  252
+--------
  253
+
  254
+.. warning::
  255
+
  256
+    Since the ZMQ code currently has no security, listening on an
  257
+    external-facing IP is dangerous.  You are giving any computer that can see
  258
+    you on the network the ability to issue arbitrary shell commands as you on
  259
+    your machine. Read the rest of this section before listening on external ports
  260
+    or running an IPython kernel on a shared machine.
  261
+
233 262
 By default (for security reasons), the kernel only listens on localhost, so you
234 263
 can only connect multiple frontends to the kernel from your local machine. You
235 264
 can specify to listen on an external interface by specifying the ``ip``
@@ -238,14 +267,134 @@ argument::
238 267
     $> ipython qtconsole --ip=192.168.1.123
239 268
 
240 269
 If you specify the ip as 0.0.0.0, that refers to all interfaces, so any
241  
-computer that can see yours can connect to the kernel.
  270
+computer that can see yours on the network can connect to the kernel.
  271
+
  272
+Messages are not encrypted, so users with access to the ports your kernel is using will be
  273
+able to see any output of the kernel. They will also be able to issue shell commands as
  274
+you, unless you enable HMAC digests, which are **DISABLED** by default.
  275
+
  276
+The one security feature IPython does provide is protection from unauthorized
  277
+execution. IPython's messaging system can sign messages with HMAC digests using
  278
+a shared-key.  The key is never sent over the network, it is only used to generate
  279
+a unique hash for each message, based on its content.  When IPython receives a
  280
+message, it will check that the digest matches. You can use any file that only you
  281
+have access to to generate this key.  One logical choice would be to use your own 
  282
+SSH private key. Or you can generate a new random private key with::
  283
+
  284
+    # generate 1024b of random data, and store in a file only you can read:
  285
+    # (assumes IPYTHON_DIR is defined, otherwise use your IPython directory)
  286
+    $> python -c "import os; print os.urandom(128).encode('base64')" > $IPYTHON_DIR/sessionkey
  287
+    $> chmod 600 $IPYTHON_DIR/sessionkey
  288
+
  289
+To enable HMAC digests, simply specify the ``Session.keyfile`` configurable
  290
+in :file:`ipython_config.py` or at the command-line, as in::
  291
+
  292
+    # instruct IPython to sign messages with that key:
  293
+    $> ipython qtconsole --Session.keyfile=$IPYTHON_DIR/sessionkey
  294
+
  295
+You must use the same key you used to start the kernel with all frontends, or
  296
+they will be treated as an unauthorized peer (all messages will be ignored).
  297
+
  298
+.. note::
  299
+
  300
+    IPython will move to using files to store connection information, as is
  301
+    done in :mod:`IPython.parallel`, at which point HMAC signatures will be
  302
+    enabled *by default*.
  303
+
  304
+.. _ssh_tunnels:
  305
+
  306
+SSH Tunnels
  307
+-----------
  308
+
  309
+Sometimes you want to connect to machines across the internet, or just across
  310
+a LAN that either doesn't permit open ports or you don't trust the other
  311
+machines on the network.  To do this, you can use SSH tunnels.  SSH tunnels
  312
+are a way to securely forward ports on your local machine to ports on another
  313
+machine, to which you have SSH access.
  314
+
  315
+In simple cases, IPython's tools can forward ports over ssh by simply adding the
  316
+``--ssh=remote`` argument to the usual ``--existing...`` set of flags for connecting
  317
+to a running kernel.
242 318
 
243 319
 .. warning::
244 320
 
245  
-    Since the ZMQ code currently has no security, listening on an
246  
-    external-facing IP is dangerous.  You are giving any computer that can see
247  
-    you on the network the ability to issue arbitrary shell commands as you on
248  
-    your machine. Be very careful with this.
  321
+    Using SSH tunnels does *not* increase localhost security.  In fact, when
  322
+    tunneling from one machine to another *both* machines have open
  323
+    ports on localhost available for connections.
  324
+
  325
+There are two primary models for using SSH tunnels with IPython.  The first
  326
+is to have the Kernel listen only on localhost, and connect to it from
  327
+another machine on the same LAN.
  328
+
  329
+First, let's start a kernel on machine **worker**, listening only
  330
+on loopback::
  331
+
  332
+    user@worker $> ipython kernel
  333
+    [IPKernelApp] To connect another client to this kernel, use:
  334
+    [IPKernelApp] --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
  335
+
  336
+In this case, the IP that you would connect
  337
+to would still be 127.0.0.1, but you want to specify the additional ``--ssh`` argument
  338
+with the hostname of the kernel (in this example, it's 'worker')::
  339
+
  340
+    user@client $> ipython qtconsole  --ssh=worker --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
  341
+
  342
+Note again that this opens ports on the *client* machine that point to your kernel.
  343
+Be sure to use a Session key, as described above, if localhost on *either* the
  344
+client or kernel machines is untrusted.
  345
+
  346
+.. note::
  347
+
  348
+    the ssh argument is simply passed to openssh, so it can be fully specified ``user@host:port``
  349
+    but it will also respect your aliases, etc. in :file:`.ssh/config` if you have any.
  350
+
  351
+The second pattern is for connecting to a machine behind a firewall across the internet
  352
+(or otherwise wide network). This time, we have a machine **login** that you have ssh access
  353
+to, which can see **kernel**, but **client** is on another network. The important difference
  354
+now is that **client** can see **login**, but *not* **worker**. So we need to forward ports from
  355
+client to worker *via* login. This means that the kernel must be started listening
  356
+on external interfaces, so that its ports are visible to `login`::
  357
+
  358
+    user@worker $> ipython kernel --ip=0.0.0.0
  359
+    [IPKernelApp] To connect another client to this kernel, use:
  360
+    [IPKernelApp] --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
  361
+
  362
+Which we can connect to from the client with::
  363
+
  364
+    user@client $> ipython qtconsole --ssh=login --ip=192.168.1.123 --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
  365
+
  366
+Note that now the IP is the address of worker as seen from login.
  367
+
  368
+Manual SSH tunnels
  369
+------------------
  370
+
  371
+It's possible that IPython's ssh helper functions won't work for you, for various
  372
+reasons.  You can still connect to remote machines, as long as you set up the tunnels
  373
+yourself.  The basic format of forwarding a local port to a remote one is::
  374
+
  375
+    [client] $> ssh <server> <localport>:<remoteip>:<remoteport> -f -N
  376
+
  377
+This will forward local connections to **localport** on client to **remoteip:remoteport**
  378
+*via* **server**. Note that remoteip is interpreted relative to *server*, not the client.
  379
+So if you have direct ssh access to the machine to which you want to forward connections,
  380
+then the server *is* the remote machine, and remoteip should be server's IP as seen from the
  381
+server itself, i.e. 127.0.0.1.  Thus, to forward local port 12345 to remote port 54321 on
  382
+a machine you can see, do::
  383
+
  384
+    [client] $> ssh machine 12345:127.0.0.1:54321 -f -N
  385
+
  386
+But if your target is actually on a LAN at 192.168.1.123, behind another machine called **login**,
  387
+then you would do::
  388
+
  389
+    [client] $> ssh login 12345:192.168.1.16:54321 -f -N
  390
+
  391
+The ``-f -N`` on the end are flags that tell ssh to run in the background,
  392
+and don't actually run any commands beyond creating the tunnel.
  393
+
  394
+.. seealso::
  395
+
  396
+    A short discussion of ssh tunnels: http://www.revsys.com/writings/quicktips/ssh-tunnel.html
  397
+
249 398
 
250 399
 
251 400
 Stopping Kernels and Consoles
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.