Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

add ssh tunnel support to qtconsole #686

Merged
merged 6 commits into from

2 participants

@minrk
Owner

title pretty much covers it.

This adds init_ssh step to qtconsole app, which will use the ssh tunneling code used elsewhere to forward connections.

So, in addition to pasting the usual --existing..., just add --ssh=foo, and you should be set.

@fperez
Owner

This is excellent, the only thing I'd like to see is a short section added to the qt console docs explaining users how to call it. It will save us from answering that question more than once on-list.

IPython/frontend/qt/console/qtconsoleapp.py
@@ -25,6 +26,7 @@ import sys
from IPython.external.qt import QtGui
from pygments.styles import get_all_styles
+# from IPython.external.ssh import tunnel
@fperez Owner
fperez added a note

This should probably just be deleted, since it's commented out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@minrk
Owner

@fperez I fixed the comment you mentioned with a slightly more substantial change (I fixed the circular import altogether by making external.ssh as standalone as it should be in the first place). I'm working on the doc now.

@minrk
Owner

doc updated, can probably be merged

@fperez
Owner

except right now it won't merge anymore, there's a conflict on ssh/tunnel.py. Can you have a look?

minrk added some commits
@minrk minrk add ssh tunnel support to qtconsole fcf87c2
@minrk minrk Remove IPython dependency in external.ssh
copy parallel.util.select_random_ports into external.ssh.tunnel

This lets external.ssh be moved to another project without IPython, only changing the pexpect import.

This also resolves a circular import in the qtconsole
34eefb9
@minrk minrk increase default ssh tunnel timeout to 60 seconds
also expose timeout to tunnel_connection function
65d9f3e
@minrk minrk add security / ssh tunnel notes to qtconsole docs d3a4730
@minrk minrk fix key->kbd role name for keys in qt doc fc63d37
@minrk minrk fix check/try typo in paramiko_tunnel c001961
@minrk
Owner

Roger - enginessh touched external.ssh as well, which was already merged. I rebased that one, because I thought this had already been merged. Now it should be clean.

@fperez
Owner

Looking great, thanks! Merging now.

@fperez fperez merged commit ba8f067 into from
@fperez fperez referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 17, 2011
  1. @minrk
  2. @minrk

    Remove IPython dependency in external.ssh

    minrk authored
    copy parallel.util.select_random_ports into external.ssh.tunnel
    
    This lets external.ssh be moved to another project without IPython, only changing the pexpect import.
    
    This also resolves a circular import in the qtconsole
  3. @minrk

    increase default ssh tunnel timeout to 60 seconds

    minrk authored
    also expose timeout to tunnel_connection function
  4. @minrk
  5. @minrk
  6. @minrk
This page is out of date. Refresh to see the latest.
View
47 IPython/external/ssh/tunnel.py
@@ -16,6 +16,7 @@
from __future__ import print_function
import os,sys, atexit
+import socket
from multiprocessing import Process
from getpass import getpass, getuser
import warnings
@@ -34,12 +35,32 @@
except ImportError:
pexpect = None
-from IPython.parallel.util import select_random_ports
-
#-----------------------------------------------------------------------------
# Code
#-----------------------------------------------------------------------------
+# select_random_ports copied from IPython.parallel.util
+_random_ports = set()
+
+def select_random_ports(n):
+ """Selects and return n random ports that are available."""
+ ports = []
+ for i in xrange(n):
+ sock = socket.socket()
+ sock.bind(('', 0))
+ while sock.getsockname()[1] in _random_ports:
+ sock.close()
+ sock = socket.socket()
+ sock.bind(('', 0))
+ ports.append(sock)
+ for i, sock in enumerate(ports):
+ port = sock.getsockname()[1]
+ sock.close()
+ ports[i] = port
+ _random_ports.add(port)
+ return ports
+
+
#-----------------------------------------------------------------------------
# Check for passwordless login
#-----------------------------------------------------------------------------
@@ -101,7 +122,7 @@ def _try_passwordless_paramiko(server, keyfile):
return True
-def tunnel_connection(socket, addr, server, keyfile=None, password=None, paramiko=None):
+def tunnel_connection(socket, addr, server, keyfile=None, password=None, paramiko=None, timeout=60):
"""Connect a socket to an address via an ssh tunnel.
This is a wrapper for socket.connect(addr), when addr is not accessible
@@ -110,12 +131,12 @@ def tunnel_connection(socket, addr, server, keyfile=None, password=None, paramik
selected local port of the tunnel.
"""
- new_url, tunnel = open_tunnel(addr, server, keyfile=keyfile, password=password, paramiko=paramiko)
+ new_url, tunnel = open_tunnel(addr, server, keyfile=keyfile, password=password, paramiko=paramiko, timeout=timeout)
socket.connect(new_url)
return tunnel
-def open_tunnel(addr, server, keyfile=None, password=None, paramiko=None):
+def open_tunnel(addr, server, keyfile=None, password=None, paramiko=None, timeout=60):
"""Open a tunneled connection from a 0MQ url.
For use inside tunnel_connection.
@@ -136,10 +157,11 @@ def open_tunnel(addr, server, keyfile=None, password=None, paramiko=None):
tunnelf = paramiko_tunnel
else:
tunnelf = openssh_tunnel
- tunnel = tunnelf(lport, rport, server, remoteip=ip, keyfile=keyfile, password=password)
+
+ tunnel = tunnelf(lport, rport, server, remoteip=ip, keyfile=keyfile, password=password, timeout=timeout)
return 'tcp://127.0.0.1:%i'%lport, tunnel
-def openssh_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=15):
+def openssh_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=60):
"""Create an ssh tunnel using command-line ssh that connects port lport
on this machine to localhost:rport on server. The tunnel
will automatically close when not in use, remaining open
@@ -171,7 +193,9 @@ def openssh_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, pas
password : str;
Your ssh password to the ssh server. Note that if this is left None,
you will be prompted for it if passwordless key based login is unavailable.
-
+ timeout : int [default: 60]
+ The time (in seconds) after which no activity will result in the tunnel
+ closing. This prevents orphaned tunnels from running forever.
"""
if pexpect is None:
raise ImportError("pexpect unavailable, use paramiko_tunnel")
@@ -215,7 +239,7 @@ def _split_server(server):
port = 22
return username, server, port
-def paramiko_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=15):
+def paramiko_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, password=None, timeout=60):
"""launch a tunner with paramiko in a subprocess. This should only be used
when shell ssh is unavailable (e.g. Windows).
@@ -250,13 +274,16 @@ def paramiko_tunnel(lport, rport, server, remoteip='127.0.0.1', keyfile=None, pa
password : str;
Your ssh password to the ssh server. Note that if this is left None,
you will be prompted for it if passwordless key based login is unavailable.
+ timeout : int [default: 60]
+ The time (in seconds) after which no activity will result in the tunnel
+ closing. This prevents orphaned tunnels from running forever.
"""
if paramiko is None:
raise ImportError("Paramiko not available")
if password is None:
- if not _check_passwordless_paramiko(server, keyfile):
+ if not _try_passwordless_paramiko(server, keyfile):
password = getpass("%s's password: "%(server))
p = Process(target=_paramiko_tunnel,
View
40 IPython/frontend/qt/console/qtconsoleapp.py
@@ -20,11 +20,15 @@
import os
import signal
import sys
+from getpass import getpass
# System library imports
from IPython.external.qt import QtGui
from pygments.styles import get_all_styles
+# external imports
+from IPython.external.ssh import tunnel
+
# Local imports
from IPython.config.application import boolean_flag
from IPython.core.application import BaseIPythonApplication
@@ -34,6 +38,7 @@
from IPython.frontend.qt.console.rich_ipython_widget import RichIPythonWidget
from IPython.frontend.qt.console import styles
from IPython.frontend.qt.kernelmanager import QtKernelManager
+from IPython.parallel.util import select_random_ports
from IPython.utils.traitlets import (
Dict, List, Unicode, Int, CaselessStrEnum, CBool, Any
)
@@ -219,6 +224,7 @@ def closeEvent(self, event):
editor = 'IPythonWidget.editor',
paging = 'ConsoleWidget.paging',
+ ssh = 'IPythonQtConsoleApp.sshserver',
)
aliases.update(qt_aliases)
# also scrub aliases from the frontend
@@ -266,6 +272,12 @@ class IPythonQtConsoleApp(BaseIPythonApplication):
Consoles on other machines will be able to connect
to the Kernel, so be careful!"""
)
+
+ sshserver = Unicode('', config=True,
+ help="""The SSH server to use to connect to the kernel.""")
+ sshkey = Unicode('', config=True,
+ help="""Path to the ssh key to use for logging in to the ssh server.""")
+
hb_port = Int(0, config=True,
help="set the heartbeat port [default: random]")
shell_port = Int(0, config=True,
@@ -322,7 +334,32 @@ def parse_command_line(self, argv=None):
key = a.lstrip('-').split('=')[0]
if key in qt_flags:
self.kernel_argv.remove(a)
-
+
+ def init_ssh(self):
+ """set up ssh tunnels, if needed."""
+ if not self.sshserver and not self.sshkey:
+ return
+
+ if self.sshkey and not self.sshserver:
+ self.sshserver = self.ip
+ self.ip=LOCALHOST
+
+ lports = select_random_ports(4)
+ rports = self.shell_port, self.iopub_port, self.stdin_port, self.hb_port
+ self.shell_port, self.iopub_port, self.stdin_port, self.hb_port = lports
+
+ remote_ip = self.ip
+ self.ip = LOCALHOST
+ self.log.info("Forwarding connections to %s via %s"%(remote_ip, self.sshserver))
+
+ if tunnel.try_passwordless_ssh(self.sshserver, self.sshkey):
+ password=False
+ else:
+ password = getpass("SSH Password for %s: "%self.sshserver)
+
+ for lp,rp in zip(lports, rports):
+ tunnel.ssh_tunnel(lp, rp, self.sshserver, remote_ip, self.sshkey, password)
+
def init_kernel_manager(self):
# Don't let Qt or ZMQ swallow KeyboardInterupts.
signal.signal(signal.SIGINT, signal.SIG_DFL)
@@ -420,6 +457,7 @@ def init_colors(self):
def initialize(self, argv=None):
super(IPythonQtConsoleApp, self).initialize(argv)
+ self.init_ssh()
self.init_kernel_manager()
self.init_qt_elements()
self.init_colors()
View
167 docs/source/interactive/qtconsole.txt
@@ -28,10 +28,10 @@ is not yet configurable.
Since the Qt console tries hard to behave like a terminal, by default it
immediately executes single lines of input that are complete. If you want
- to force multiline input, hit :key:`Ctrl-Enter` at the end of the first line
- instead of :key:`Enter`, and it will open a new line for input. At any
+ to force multiline input, hit :kbd:`Ctrl-Enter` at the end of the first line
+ instead of :kbd:`Enter`, and it will open a new line for input. At any
point in a multiline block, you can force its execution (without having to
- go to the bottom) with :key:`Shift-Enter`.
+ go to the bottom) with :kbd:`Shift-Enter`.
``%loadpy``
===========
@@ -213,6 +213,8 @@ blocking execution. The frontend can also know, via a heartbeat mechanism, that
the kernel has died. This means that the frontend can safely restart the
kernel.
+.. _multiple_consoles:
+
Multiple Consoles
*****************
@@ -225,11 +227,38 @@ like::
[IPKernelApp] --existing --shell=60690 --iopub=44045 --stdin=38323 --hb=41797
Other frontends can connect to your kernel, and share in the execution. This is
-great for collaboration. The `-e` flag is for 'external'. Starting other
+great for collaboration. The ``--existing`` flag means connect to a kernel
+that already exists. Starting other
consoles with that flag will not try to start their own, but rather connect to
yours. Ultimately, you will not have to specify each port individually, but for
now this copy-paste method is best.
+You can even launch a standalone kernel, and connect and disconnect Qt Consoles
+from various machines. This lets you keep the same running IPython session
+on your work machine (with matplotlib plots and everything), logging in from home,
+cafés, etc.::
+
+ $> ipython kernel
+ [IPKernelApp] To connect another client to this kernel, use:
+ [IPKernelApp] --existing --shell=60690 --iopub=44045 --stdin=38323 --hb=41797
+
+This is actually exactly the same as the subprocess launched by the qtconsole, so
+all the information about connecting to a standalone kernel is identical to that
+of connecting to the kernel attached to a running console.
+
+.. _kernel_security:
+
+Security
+--------
+
+.. warning::
+
+ Since the ZMQ code currently has no security, listening on an
+ external-facing IP is dangerous. You are giving any computer that can see
+ you on the network the ability to issue arbitrary shell commands as you on
+ your machine. Read the rest of this section before listening on external ports
+ or running an IPython kernel on a shared machine.
+
By default (for security reasons), the kernel only listens on localhost, so you
can only connect multiple frontends to the kernel from your local machine. You
can specify to listen on an external interface by specifying the ``ip``
@@ -238,14 +267,134 @@ argument::
$> ipython qtconsole --ip=192.168.1.123
If you specify the ip as 0.0.0.0, that refers to all interfaces, so any
-computer that can see yours can connect to the kernel.
+computer that can see yours on the network can connect to the kernel.
+
+Messages are not encrypted, so users with access to the ports your kernel is using will be
+able to see any output of the kernel. They will also be able to issue shell commands as
+you, unless you enable HMAC digests, which are **DISABLED** by default.
+
+The one security feature IPython does provide is protection from unauthorized
+execution. IPython's messaging system can sign messages with HMAC digests using
+a shared-key. The key is never sent over the network, it is only used to generate
+a unique hash for each message, based on its content. When IPython receives a
+message, it will check that the digest matches. You can use any file that only you
+have access to to generate this key. One logical choice would be to use your own
+SSH private key. Or you can generate a new random private key with::
+
+ # generate 1024b of random data, and store in a file only you can read:
+ # (assumes IPYTHON_DIR is defined, otherwise use your IPython directory)
+ $> python -c "import os; print os.urandom(128).encode('base64')" > $IPYTHON_DIR/sessionkey
+ $> chmod 600 $IPYTHON_DIR/sessionkey
+
+To enable HMAC digests, simply specify the ``Session.keyfile`` configurable
+in :file:`ipython_config.py` or at the command-line, as in::
+
+ # instruct IPython to sign messages with that key:
+ $> ipython qtconsole --Session.keyfile=$IPYTHON_DIR/sessionkey
+
+You must use the same key you used to start the kernel with all frontends, or
+they will be treated as an unauthorized peer (all messages will be ignored).
+
+.. note::
+
+ IPython will move to using files to store connection information, as is
+ done in :mod:`IPython.parallel`, at which point HMAC signatures will be
+ enabled *by default*.
+
+.. _ssh_tunnels:
+
+SSH Tunnels
+-----------
+
+Sometimes you want to connect to machines across the internet, or just across
+a LAN that either doesn't permit open ports or you don't trust the other
+machines on the network. To do this, you can use SSH tunnels. SSH tunnels
+are a way to securely forward ports on your local machine to ports on another
+machine, to which you have SSH access.
+
+In simple cases, IPython's tools can forward ports over ssh by simply adding the
+``--ssh=remote`` argument to the usual ``--existing...`` set of flags for connecting
+to a running kernel.
.. warning::
- Since the ZMQ code currently has no security, listening on an
- external-facing IP is dangerous. You are giving any computer that can see
- you on the network the ability to issue arbitrary shell commands as you on
- your machine. Be very careful with this.
+ Using SSH tunnels does *not* increase localhost security. In fact, when
+ tunneling from one machine to another *both* machines have open
+ ports on localhost available for connections.
+
+There are two primary models for using SSH tunnels with IPython. The first
+is to have the Kernel listen only on localhost, and connect to it from
+another machine on the same LAN.
+
+First, let's start a kernel on machine **worker**, listening only
+on loopback::
+
+ user@worker $> ipython kernel
+ [IPKernelApp] To connect another client to this kernel, use:
+ [IPKernelApp] --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
+
+In this case, the IP that you would connect
+to would still be 127.0.0.1, but you want to specify the additional ``--ssh`` argument
+with the hostname of the kernel (in this example, it's 'worker')::
+
+ user@client $> ipython qtconsole --ssh=worker --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
+
+Note again that this opens ports on the *client* machine that point to your kernel.
+Be sure to use a Session key, as described above, if localhost on *either* the
+client or kernel machines is untrusted.
+
+.. note::
+
+ the ssh argument is simply passed to openssh, so it can be fully specified ``user@host:port``
+ but it will also respect your aliases, etc. in :file:`.ssh/config` if you have any.
+
+The second pattern is for connecting to a machine behind a firewall across the internet
+(or otherwise wide network). This time, we have a machine **login** that you have ssh access
+to, which can see **kernel**, but **client** is on another network. The important difference
+now is that **client** can see **login**, but *not* **worker**. So we need to forward ports from
+client to worker *via* login. This means that the kernel must be started listening
+on external interfaces, so that its ports are visible to `login`::
+
+ user@worker $> ipython kernel --ip=0.0.0.0
+ [IPKernelApp] To connect another client to this kernel, use:
+ [IPKernelApp] --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
+
+Which we can connect to from the client with::
+
+ user@client $> ipython qtconsole --ssh=login --ip=192.168.1.123 --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511
+
+Note that now the IP is the address of worker as seen from login.
+
+Manual SSH tunnels
+------------------
+
+It's possible that IPython's ssh helper functions won't work for you, for various
+reasons. You can still connect to remote machines, as long as you set up the tunnels
+yourself. The basic format of forwarding a local port to a remote one is::
+
+ [client] $> ssh <server> <localport>:<remoteip>:<remoteport> -f -N
+
+This will forward local connections to **localport** on client to **remoteip:remoteport**
+*via* **server**. Note that remoteip is interpreted relative to *server*, not the client.
+So if you have direct ssh access to the machine to which you want to forward connections,
+then the server *is* the remote machine, and remoteip should be server's IP as seen from the
+server itself, i.e. 127.0.0.1. Thus, to forward local port 12345 to remote port 54321 on
+a machine you can see, do::
+
+ [client] $> ssh machine 12345:127.0.0.1:54321 -f -N
+
+But if your target is actually on a LAN at 192.168.1.123, behind another machine called **login**,
+then you would do::
+
+ [client] $> ssh login 12345:192.168.1.16:54321 -f -N
+
+The ``-f -N`` on the end are flags that tell ssh to run in the background,
+and don't actually run any commands beyond creating the tunnel.
+
+.. seealso::
+
+ A short discussion of ssh tunnels: http://www.revsys.com/writings/quicktips/ssh-tunnel.html
+
Stopping Kernels and Consoles
Something went wrong with that request. Please try again.