Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Open a new notebook while connecting to an existing kernel (opened by qtconsole or terminal or standalone) #1220

Closed
wants to merge 4 commits into from

3 participants

Madhusudan.C.S Brian E. Granger Fernando Perez
Madhusudan.C.S

This adds a new GET handler for the /kernels/ RequestHandler which opens the new notebook based on the kernel file name specified as the value for the kernel_id argument in the URL.

On the UI side, it looks like this. The main dashboard already has a New button. Upon clicking it, a dialog opens with a text box with the label "kernel". It is not a mandatory field. One can leave it blank in which case it opens up a new notebook with new kernel and it is the same as the action that happens when we click the new button as of now. But if we choose to put the name of the kernel file, say "kernel-5151.json", it redirects to the URL /kernels/kernel-5151.json and opens a new notebook with that kernel.

This is not a perfect patch. I do not even expect this to be merged right away. It needs a lot of changes. So it is a first cut. Sending a pull request just to draw some attention for discussion.

madhusudancs added some commits
Madhusudan.C.S madhusudancs When a connection file is supplied, use it to start a new notebook us…
…ing the kernel in the file.

This allows us to start a new notebook with existing kernels. Moreover
if we supply the connection file details of a qtconsole based kernel
or a terminal based kernel or any standalone existing kernel files, we
can connect the notebook to it.
8715f65
Madhusudan.C.S madhusudancs Take connection file as optional kwarg for allowing kernels based on …
…existing kernel files.
6addae9
Madhusudan.C.S madhusudancs Implement a get view for the /kernels/... Request handler.
We do this so that when the kernel file name is supplied as part of the
URLPattern we can open a new notebook based on the kernel information
supplied in the file that already exists.
bfba037
Madhusudan.C.S madhusudancs Add a dialog on the main notebook dashboard page to ask for the kerne…
…l filename.
fe30533
Brian E. Granger
Owner

There is a huge amount of complexity lurking in this direction that we don't want to get into at this point. It was a deliberate choice to stick with the 1 notebook = 1 kernel constraint for now. To go this direction we would need to completely rethink how kernels and notebooks are related and develop a UI for linking arbitrary notebooks to arbitrary kernels. It would also require a complete reworking of the server backend. There are a number of other more pressing things on the table for the notebook right now so I don't think it makes sense to pursue this right now.

But even more important is that we are aiming to keep the notebook as simple as possible and that would introduce a huge amount of complexity that many users will find confusing. It is an additional level of abstraction that requires user to keep track the various kernel namespaces in their head. One feature we do plan on implementing that would provide some of the same user experience without the confusing aspects is having multiple "worksheets" in a single notebook. This would basically provide tabs in the notebook where all tabs point to the same kernels. This would allow users to organize their notebook into logical sections, but maintain the 1 kernel = 1 notebook constraint.

Madhusudan.C.S

Oh Ok. I honestly do not know the complexities involved since I do not have the full picture. If that is the reason why this doesn't exist, it makes sense. Just one question about the feature of multiple worksheets. Will that work across machines?

Brian E. Granger
Owner
Madhusudan.C.S

Will it be possible to open a different worksheet on a different computer?

Fernando Perez
Owner

Brian, I haven't read the code yet so I'm not sure of all the details, but my understanding from Madhu's description was not that it changed the 1-1 mapping of notebooks to kernels, but rather that it enabled notebooks to be hooked to already running kernels. That's precisely what the qt console and console both do with the --existing flag, and it is something that is missing from the notebook currently. We may want to revisit how to most cleanly achieve that, but if that was the intent of this PR, then I think it is something that we do want very much.

Now, the only thing I'm talking about is what kernel a notebook is opened against when freshly opened. For example, once a notebook server has associated a notebook file with a kernel, it should not be possible to reopen the same file with a different kernel (to avoid the confusion Brian alludes to). But at first shot, when no kernel has been created yet, it should definitely be possible to say 'open this notebook file -- or make a new one-- against this existing kernel' just like the --existing flag works today.

Brian E. Granger
Owner

You are right that it primarily addresses the usage case of an existing kernel that was started outside the notebook. But I think this still does break the 1-1 mapping. By that, I mean that the change would allow a single notebook to be hooked up to a number of different kernels. It doesn't matter that those other kernels were started outside the notebook. It becomes a 1-N mapping, although the notebook is hooked up to a single kernel at one time (not sure how a notebook could be hooked up to multiple kernels simultaneously).

Implementing this in the right way would involve changing the URL scheme in significant ways and introducing new abstractions in the server side code for linking kernels to notebooks. We are about to embark on a radical refactoring of the URL scheme and I don't want to introduce new things in a semi-thought-out way at this point. The refactoring process would give us a chance to consider how these things could be handled properly.

While I see the usage case for hooking a notebook up to existing kernels there is a significant complexity cost in going down this path that we can't ignore. It forces the user into thinking abstractly about notebooks and kernels in a manner that is quite subtle. And it does so the second the user tries to open a new notebook. For us, the abstraction of the kernel and frontend is completely natural. But we have been thinking about this for years and most python users, even relatively advanced ones, don't think in these terms. I would almost (not quite though) say that the kernel abstraction is an implementation detail that new users should never need to worry about. If we ever go down this path (I am still not convinced), we would need to design a UI that completely hides these things from new users and only exposes them to advanced users when they ask for it.

I feel strongly that the notebook needs to have a UI that inexperienced users can use with 0 documentation. It needs to be as simple as angry birds for these users. Think of a freshman in college with no programming experience for whom the statement "a=10" involves a significant cognitive load. The (fun!) challenge for us is that we also want the notebook to be a powerful almost-IDE environment for the best and brightest programmers in the world.

Brian E. Granger
Owner

This will be continued in #1274.

Brian E. Granger ellisonbg closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 2, 2012
  1. Madhusudan.C.S

    When a connection file is supplied, use it to start a new notebook us…

    madhusudancs authored
    …ing the kernel in the file.
    
    This allows us to start a new notebook with existing kernels. Moreover
    if we supply the connection file details of a qtconsole based kernel
    or a terminal based kernel or any standalone existing kernel files, we
    can connect the notebook to it.
  2. Madhusudan.C.S
  3. Madhusudan.C.S

    Implement a get view for the /kernels/... Request handler.

    madhusudancs authored
    We do this so that when the kernel file name is supplied as part of the
    URLPattern we can open a new notebook based on the kernel information
    supplied in the file that already exists.
  4. Madhusudan.C.S
This page is out of date. Refresh to see the latest.
18 IPython/frontend/html/notebook/handlers.py
View
@@ -299,7 +299,23 @@ def post(self):
class KernelHandler(AuthenticatedHandler):
- SUPPORTED_METHODS = ('DELETE')
+ @web.authenticated
+ def get(self, kernel_id):
+ nbm = self.application.notebook_manager
+ project = nbm.notebook_dir
+ notebook_id = self.get_argument('notebook', nbm.new_notebook())
+
+ km = self.application.kernel_manager
+ kernel_id = km.start_kernel(notebook_id=notebook_id,
+ connection_file=kernel_id)
+ self.render(
+ 'notebook.html', project=project,
+ notebook_id=notebook_id,
+ base_project_url=u'/', base_kernel_url=u'/',
+ kill_kernel=False,
+ read_only=False,
+ mathjax_url=self.application.ipython_app.mathjax_url,
+ )
@web.authenticated
def delete(self, kernel_id):
58 IPython/frontend/html/notebook/kernelmanager.py
View
@@ -16,6 +16,7 @@
# Imports
#-----------------------------------------------------------------------------
+import json
import os
import signal
import sys
@@ -29,6 +30,8 @@
from IPython.config.configurable import LoggingConfigurable
from IPython.zmq.ipkernel import launch_kernel
from IPython.zmq.kernelmanager import KernelManager
+from IPython.utils.path import filefind
+from IPython.utils.py3compat import str_to_bytes
from IPython.utils.traitlets import Instance, Dict, List, Unicode, Float, Integer
#-----------------------------------------------------------------------------
@@ -65,14 +68,54 @@ def __contains__(self, kernel_id):
else:
return False
+ def load_connection_file(self, connection_file):
+ """load ip/port/hmac config from JSON connection file"""
+ # this is identical to KernelApp.load_connection_file
+ # perhaps it can be centralized somewhere?
+ try:
+ fname = filefind(connection_file, [self.connection_dir])
+ except IOError:
+ self.log.debug("Connection File not found: %s", connection_file)
+ return
+ self.log.debug(u"Loading connection file %s", fname)
+ with open(fname) as f:
+ s = f.read()
+ cfg = json.loads(s)
+ if 'ip' in cfg:
+ # not overridden by config or cl_args
+ self.ip = cfg['ip']
+ for channel in ('hb', 'shell', 'iopub', 'stdin'):
+ name = channel + '_port'
+ if getattr(self, name, 0) == 0 and name in cfg:
+ # not overridden by config or cl_args
+ setattr(self, name, cfg[name])
+ if 'key' in cfg:
+ self.config.Session.key = str_to_bytes(cfg['key'])
+
def start_kernel(self, **kwargs):
"""Start a new kernel."""
- kernel_id = unicode(uuid.uuid4())
- # use base KernelManager for each Kernel
- km = KernelManager(connection_file=os.path.join(
- self.connection_dir, "kernel-%s.json" % kernel_id),
- config=self.config,
- )
+ if 'connection_file' in kwargs:
+ connection_file = kwargs.pop('connection_file')
+ self.load_connection_file(connection_file)
+
+ kernel_id = self.config.Session.key
+ # use base KernelManager for each Kernel
+ km = KernelManager(ip=self.ip,
+ shell_port=self.shell_port,
+ iopub_port=self.iopub_port,
+ stdin_port=self.stdin_port,
+ hb_port=self.hb_port,
+ connection_file=os.path.join(
+ self.connection_dir, connection_file),
+ config=self.config,
+ )
+ else:
+ kernel_id = unicode(uuid.uuid4())
+ # use base KernelManager for each Kernel
+ km = KernelManager(connection_file=os.path.join(
+ self.connection_dir, "kernel-%s.json" % kernel_id),
+ config=self.config,
+ )
km.start_kernel(**kwargs)
self._kernels[kernel_id] = km
return kernel_id
@@ -230,7 +273,7 @@ def delete_mapping_for_kernel(self, kernel_id):
if notebook_id is not None:
del self._notebook_mapping[notebook_id]
- def start_kernel(self, notebook_id=None):
+ def start_kernel(self, notebook_id=None, connection_file=None):
"""Start a kernel for a notebok an return its kernel_id.
Parameters
@@ -244,6 +287,7 @@ def start_kernel(self, notebook_id=None):
if kernel_id is None:
kwargs = dict()
kwargs['extra_arguments'] = self.kernel_argv
+ kwargs['connection_file'] = connection_file
kernel_id = super(MappingKernelManager, self).start_kernel(**kwargs)
self.set_kernel_for_notebook(notebook_id, kernel_id)
self.log.info("Kernel started: %s" % kernel_id)
2  IPython/frontend/html/notebook/notebookapp.py
View
@@ -69,7 +69,7 @@
# Module globals
#-----------------------------------------------------------------------------
-_kernel_id_regex = r"(?P<kernel_id>\w+-\w+-\w+-\w+-\w+)"
+_kernel_id_regex = r"(?P<kernel_id>\w+-\w+-\w+-\w+-\w+|\w+-\d+.json)"
_kernel_action_regex = r"(?P<action>restart|interrupt)"
_notebook_id_regex = r"(?P<notebook_id>\w+-\w+-\w+-\w+-\w+)"
20 IPython/frontend/html/notebook/static/js/projectdashboardmain.js
View
@@ -21,7 +21,25 @@ $(document).ready(function () {
$('div#content_toolbar').addClass('ui-widget ui-helper-clearfix');
$('#new_notebook').button().click(function (e) {
- window.open($('body').data('baseProjectUrl')+'new');
+ $('#dialog-form').dialog({
+ resizable: false,
+ modal: true,
+ title: "Delete notebook",
+ buttons : {
+ "New": function () {
+ var kernel = $('#kernel').val();
+ if (kernel) {
+ window.open($('body').data('baseProjectUrl')+'kernels/'+ kernel);
+ } else {
+ window.open($('body').data('baseProjectUrl')+'new');
+ }
+ $(this).dialog('close');
+ },
+ "Cancel": function () {
+ $(this).dialog('close');
+ }
+ }
+ });
});
$('div#left_panel').addClass('box-flex');
6 IPython/frontend/html/notebook/templates/projectdashboard.html
View
@@ -27,6 +27,12 @@
<span id="notebooks_buttons">
<button id="new_notebook">New Notebook</button>
+ <div id="dialog-form" title="Create new notebook" style="display: none;">
+ <form>
+ <label for="name">Kernel</label>
+ <input type="text" name="kernel" id="kernel" />
+ </form>
+ </div>
</span>
</div>
Something went wrong with that request. Please try again.