Notebook feature requests #977

Open
fperez opened this Issue Nov 6, 2011 · 193 comments

Projects

None yet
@fperez
Member
fperez commented Nov 6, 2011

This is a meta-issue to list, discuss and shake out ideas for improvements to the notebook. It can serve as a central page for users to propose ideas and discuss them; as they get more fleshed out and there's a chance of implementing one of them, we can make a separate issue (or pull request, if there's code) for it.

In our docs we already have some items noted, so that list should be used as a starting point for the discussion.

@Carreau
Member
Carreau commented Nov 6, 2011

Hi,
Because you're asking for it I thought of :

  • A post-saving hook (like for example to call a versioning system)
  • save a .ipynb~ after each execution in case of the user closing the page (but without the output, for speed)
  • Visual indication of which cell are currently busy [edit] forget it, I was using an older version at work... should be more up to date[/edit]
  • (as an option ?) if help pannel if visible, update its content when '(' is inserted. kind of a tooltip.
  • a "read-only" adress for sharing ?
@fperez
Member
fperez commented Nov 6, 2011

On Sun, Nov 6, 2011 at 2:03 AM, Bussonnier Matthias
reply@reply.github.com
wrote:

  • Visual indication of which cell are currently busy

This is already done in master, busy cells have their prompt written
as 'In []' while busy, with the '' replaced by their number when
they finish.

  • a "read-only" adress for sharing ?

Already done in master. You can start a notebook with --read-only and
it will open in a new port in read-only mode. If it's a notebook that
has password protection, then the same address can be used both for
read-write (authenticated users) and read-only (unauthenticated).

@Carreau
Member
Carreau commented Nov 10, 2011

I've just created PR #987 which kind of replace my 5th point.
and another idea. Maybe we can add the shortcut for the action at the end of the "tooltip" that apear when hovering the corresponding button. like :
Delete [Delete the selected cell (Ctrl+m d)]

@minrk
Member
minrk commented Nov 11, 2011

Adding a few of our existing plans, to be tracked here:

  • IPython project == folder == repo. Everything you would have in a project goes here - scripts, notebooks, data. It all gets tracked together, and there will be some simple interface for managing non-notebook contents, and a few VCS actions. You can use multiple projects with one server, but they know nothing of each other (GitHub is used frequently as a reference).
  • Notebooks can have multiple worksheets: Our model is that a notebook == a kernel == a namespace, but you can have multiple pages in that notebook separately working against the same kernel.
  • Notebooks should have multi-level sections, which should facilitate collapsing groups of cells, as well as a good model for a notebook in rst.
  • ReST support - ReST support will be provided at the full document level, so you get all the cross-referencing, linking, scoping, etc. power that you expect from sphinx, which you cannot have if you render text cells with rest one at a time.
@minrk
Member
minrk commented Nov 11, 2011

An idea @ivanov had the other day:

  • reproducible/copyable urls/anchors per-cell, so you can have links to a particular cell that last (at least for the life of the notebook server).

And one from @stefanv, that we had working briefly, but had to remove as the first implementation caused data loss on renames:

  • consistent urls for notebooks - right now, the urls for notebooks are UUIDs, which are regenerated when the server starts. These should be the same from one server session to the next. This is certainly doable, but one has to be careful
    about what happens when the notebook is renamed.
@minrk
Member
minrk commented Nov 11, 2011

@Carreau

  • A post-saving hook (like for example to call a versioning system)
  • save a .ipynb~ after each execution in case of the user closing the page (but without the output, for speed)

My idea for this is actually something akin to the OSX 10.7 Versions model (in idea, not implementation). The notebook should be autosaved silently, but the true 'save' action is actually a git add && commit. I know how much @fperez hates autosave, but I think that a web-based tool that doesn't do any kind of autosave can never be considered production-ready. With this model properly implemented, it would be impossible for autosave to cause data loss, because your last intentional save is always a git checkout -- foo.ipynb away (for which there should certainly be a simple UI in the web interface).

@fperez
Member
fperez commented Nov 11, 2011

 I know how much @fperez hates autosave,

The autosave I hate is the one that inferior tools like Word use:
autosave that clobbers the file you're working on. But there's a form
of it that I very much like, and that we should probably implement
right away: auto-save of a temporary copy on a relatively fast
schedule (say once every few minutes at most).

I don't like autosave on the main file I'm working on b/c I think it's
a bad idea on many fronts, but an autosaved file that's differently
named, and only used as a recovery tool in the case of an untimely
crash, is something I'd want to have now. Esp given that something
as simple as a browser crash can kill your user session...

Emacs does that and it works pretty well: auto-save files are silently
deleted when the buffer is closed, but if one is ever found upon
opening a file, it most likely means that emacs died badly, and it
offers you to auto-recover it.

@ivanov
Member
ivanov commented Nov 11, 2011

it'd be great to have a cell manipulating API - creating and moving cells from another session hooked to the same kernel. I remember @minrk was telling me that this would require some sort of locking or at least synchronization mechanism, since you can make oodles of cells in the browser and nothing gets sent over the wire until you want to execute some code - but still, this would be a powerful addition to the workflow to be able to create and manipulate the notebook from some other client.

one simpler implementation of this would be something like an 'edit' button, like wikis have, that notifies you of someone else holding a lock requesting that no one else edit the notebook for a while. Nothing would prevent you from overriding it, but at least you'd get the notice that another client is intending to manipulate the NB for a while

@fperez
Member
fperez commented Nov 11, 2011

More:

  • multi-cell selection and actions. Right now, there's no way to copy/paste/delete multiple cells.
  • cell split/join: we should have actions to split a cell at the current point as well as merging a cell with the one previous/next or a group of cells (once we have multi-cell selection implemented).
@minrk
Member
minrk commented Nov 11, 2011

I know how much @fperez hates autosave,

The autosave I hate is the one that inferior tools like Word use:
autosave that clobbers the file you're working on. But there's a form
of it that I very much like, and that we should probably implement
right away: auto-save of a temporary copy on a relatively fast
schedule (say once every few minutes at most).

I don't like autosave on the main file I'm working on b/c I think it's
a bad idea on many fronts, but an autosaved file that's differently
named, and only used as a recovery tool in the case of an untimely
crash, is something I'd want to have now. Esp given that something
as simple as a browser crash can kill your user session...

Emacs does that and it works pretty well: auto-save files are silently
deleted when the buffer is closed, but if one is ever found upon
opening a file, it most likely means that emacs died badly, and it
offers you to auto-recover it.

This is makes sense, as long as it doesn't do what emacs et al. do (Word did the same, if I recall), and put recovery files adjacent to the originals. They should go in /tmp, or possibly the cache dir proposed for outputs - they must be hidden somehow, and efficiently and very aggressively cleaned up. I have had to delete thousands of foo~ files that supposedly get cleaned up by various apps, and I wish fewer applications created them, not more.

Personally, I find in-place autosave+vcs preferable, because not only does it make it impossible for autosave to lose your data, it also makes it impossible for you to clobber your own work with imprudent saves. It does have plenty of disadvantages, of course (reliance on vcs, proliferation of commits, etc.), so I fully expect to be overruled on this point, as it may well not be the best general solution.

@takluyver
Member

I'll agree with autosaving a copy in a hidden location (I think the IPython
directory is probably the place - temporary files might be lost if your
system goes down).

I'd certainly like it to be easy to integrate with a VCS, but I don't think
it should be the default mechanism, essentially for the reasons Min
mentioned. Also, we'd either have to depened on a specific VCS (annoying if
you want to keep your project in another system), or go to the trouble of
handling different systems (probably at least hg, git and svn).

@Carreau
Member
Carreau commented Nov 11, 2011

Throwing more ideas:

Could the <tab> completer keep a list of the completion possibility and do a filtering "as you type", then be dismissed when there is no matches left or escape is pressed. All in js of course to avoid querrying the kernel ...

About the Cell copy/pasting, could it be possible to have a copy, collapse (...) button that appear on the left when hovering a rendered/executed cell, a little like editing your message when you just post it on github.

Is there a way to know if a certain notebook is opened from somewhere else, or if the kernel status has been modified since last execution. I'm not familiar with websocket, but are the messages broadcasted or send on a by-host basis? If so and we get a message that don't belong to us, we might want to prompt the user with a message saying that the kernel status may have changed ? Or just count the prompt number...

@fperez
Member
fperez commented Nov 11, 2011

Actually @minrk, it seems you do like how emacs does it, because you haven't actually had a problem with it :) Let me explain: the ~ files that bug you are not the autosave ones, those are backup files that contain a copy of the previous saved version of your current file. You can actually instruct emacs to never create those if you don't want them.

The autosave files (in the sense I described above) are by default named #filename#. The fact you're not complaining about those means that it works fine: you hardly ever see them, the only show up if you had emacs segfault in the middle of a big edit.

And I would argue that by default, those should be saved next to their normal file, because the whole point is to make them easy to find in the case of an accident, and in normal usage patterns they never are a bother. In 15 years of daily emacs use I've probably had to clean those up a handful of times.

@fperez
Member
fperez commented Nov 11, 2011

@Carreau: yes, we do know when a notebook has been opened, and I'd love to add to the dashboard a button next to any open notebook to allow me to kill its kernel. It's easy to do, we just haven't put too much work on the dashboard functionality yet because our focus has been on the notebooks themselves.

@minrk
Member
minrk commented Nov 11, 2011

@fperez - 'like' is much too strong a word for anything I feel about emacs ;), but it's fine, as long as they do not appear adjacent to real files. A quick search reveals that I do have some of those files lying around, and given how extremely rarely I use emacs, that is not a good sign. Most of the 'foo~' files on my system were created by various other apps, many of which do a bad job at cleaning up.

If emacs is good at cleaning up that's great, but it is not particularly relevant - the model of writing extra files means we have to be extremely aggressive/careful about cleaning up residue, lest we create a huge annoying mess. Putting those files aside somewhere is a big step towards controlling that mess.

I do think this more primitive model is the right solution for us in general, I would just be very sad if IPython becomes one of the far-too-many Linux apps that litter the filesystem with garbage.

@Carreau
Member
Carreau commented Nov 17, 2011

I'd like to have an option in %pycat that output the file in a cell, and not in the pager for the notebook.

@fperez
Member
fperez commented Nov 17, 2011

On Thu, Nov 17, 2011 at 8:54 AM, Bussonnier Matthias
reply@reply.github.com
wrote:

I'd like to have an option in  %pycat that output the file in a cell, and not in the pager for the notebook.

That already exists, it's called %loadpy. It even works with URLs,
not just with local file paths.

@Carreau
Member
Carreau commented Nov 17, 2011

yes, except that I don't want the cell to be executable... And if I do a "run all" it append the code again ant run it twice ...

@fperez
Member
fperez commented Nov 17, 2011

On Thu, Nov 17, 2011 at 10:42 AM, Bussonnier Matthias
reply@reply.github.com
wrote:

yes, except that I don't want the cell to be executable... And if I do a "run all" it append the code again ant run it twice ...

Ah, you want it just printed out... OK. The easiest solution would be
to add a flag to pycat so it prints to stdout unconditionally and
never use the pager. That would be a quick and easy fix.

@Carreau
Member
Carreau commented Nov 17, 2011

Yes, I should have been more precise, but I was on the go and post the message not to forgot later...

@satra
satra commented Nov 19, 2011

finally getting to this (better late that never). so here it goes?

  • i'd like to see username back. why? because if i'm sharing a read-only or read-write notebook, i would like to know who i'm sharing with. (kind of like google chat). since tornado supports openid, perhaps set a dual authentication scheme, openid and/or password. talking of google chat, integrated chat would be nice :)
  • also i'd like to be ping-ed each time somebody else has added or changed a cell. i know this is rather fragile at the moment.
  • ctrl+enter should execute the current cell and leave the cursor in the cell without highlighting
  • duplicate a notebook that you are working on and haven't saved into a new notebook
  • the "stay or leave the page" message should say "you have unsaved changes. do you want to cancel, save and quit or quit with saving"
  • notebook diff. what has changed between last saved version and now.
  • split a cell at an arbitrary point, merge cell with previous or next cell
  • save workspace optionally with the notebook
  • ctrl g+?? for git/github key bindings :) i kind of simply stick a cell and do !git ...
  • gist like revision history
  • any possibility of a key binding to toggle the help display and the sidebar
  • open help in a new synched browser tab (one can have it side by side)
  • optional layouts for notebooks (on my giant screen, i might want [sidebar, notebook, help] in columns; on my little screen, i might just want notebook)
  • open a local notebook in the remote session.
  • indicate notebook status: loca/remote, notebook-userid

... more to come

@fperez
Member
fperez commented Nov 19, 2011

I'm just going to respond to the ones that are already there or for
which we have already an answer, the rest are valid, longer-term
requests as per the intent of this page. Thanks for the ideas!

  • also i'd like to be ping-ed each time somebody else has added or changed a cell. i know this is rather fragile at the moment.

It's not fragile, it simply doesn't exist! There's zero
synchronization support right now, but it's high on our
priority/interest list.

  • ctrl+enter should execute the current cell and leave the cursor in the cell without highlighting

I can see people also wanting the current behavior, since it makes it
easy to quickly retype a different command in-place, which is how I
often find myself using the in-place mode. To be honest, I find that
half the time I'd want what you ask for, the other I want what we have
now. So there's probably not a perfect default for this one :)

  • the "stay or leave the page" message should say "you have unsaved changes. do you want to cancel, save and quit or quit with saving"

Unfortunately we can't: that dialog is generated by the browser, not
by us, and we have zero control over it. It's the same one you get in
gmail if you try to close a page mid-operation.

  • any possibility of a key binding to toggle the help display and the sidebar

C-m h already toggles the help. We don't have one for the sidebar, though.

  • open a local notebook in the remote session.

You can already do this: drag and drop a notebook file onto the list
area in the dashboard, and it will be uploaded to the remote session
for opening.

@minrk
Member
minrk commented Nov 20, 2011
  • the "stay or leave the page" message should say "you have unsaved changes. do you want to cancel, save and quit or quit with saving"

Unfortunately we can't: that dialog is generated by the browser, not by us, and we have zero control over it. It's the same one you get in gmail if you try to close a page mid-operation.

Not quite, we have control over part of the message here.

@fperez
Member
fperez commented Nov 20, 2011

On Sat, Nov 19, 2011 at 4:02 PM, Min RK
reply@reply.github.com
wrote:

Not quite, we have control over part of the message here.

Ah, OK, I didn't realize at least we could improve it a little. Thanks
for the pointer!

@fperez
Member
fperez commented Nov 20, 2011

Note that #888 is a dedicated discussion about reST support, for anyone interested. That's one of the biggest topics and deserves a separate discussion, as there are quite a few complex issues there.

@minrk
Member
minrk commented Nov 21, 2011
  • i'd like to see username back. why? because if i'm sharing a read-only or read-write notebook, i would like to know who i'm sharing with. (kind of like google chat). since tornado supports openid, perhaps set a dual authentication scheme, openid and/or password. talking of google chat, integrated chat would be nice :)

Brian and I removed the username because it didn't actually do anything, and we didn't want to present the illusion of a multi-user environment we don't actually have. Once the server has some sense of users being distinct, then it should be returned.

@fperez
Member
fperez commented Nov 23, 2011

On Mon, Nov 21, 2011 at 11:41 AM, Min RK
reply@reply.github.com
wrote:

Brian and I removed the username because it didn't actually do anything, and we didn't want to present the illusion of a multi-user environment we don't actually have.  Once the server has some sense of users being distinct, then it should be returned.

I think the right approach is to offer a 'nickname' field, pre-filled
to the real username (so people who don't care about it don't have to
type anything). By calling it 'nickname' instead of 'username' it's
clear that this isn't a user in the real sense of the word, and simply
a nickname for display/chat purposes.

It should be very clear to everyone that the notebook is fundamentally
a single-user system, and that when you have a collaborator
connected you've given them full-authority shell-level access to
your system. They can do !rm -rf ~ and they'll nuke your home
directory if they are so inclined.

@minrk
Member
minrk commented Nov 23, 2011

That makes sense, but I still think having such a field that has exactly no function is more confusing than anything, so we shouldn't put it back until we have some functionality that uses it, and we can call it a nickname if that best suits whatever functionality it is that we have.

@fperez
Member
fperez commented Nov 23, 2011

Agreed, I just wanted to hash out the idea. Definitely shouldn't put it there until there's actual usage code for it.

@satra
satra commented Nov 23, 2011

@minrk: i agree that the username is useless unless it shows up on the app as a list somewhere. but even if all it did was say who is viewing and what ability they have (read/write vs read only) that would be extremely informative. it's like gmail saying that your account is being accessed via the following IPs and browsers.

@fperez: i do want people to view what i'm doing and refresh their content without being able to make any changes. and i think that's what the readonly bit does, right? so it would be good for me to know who is in that mode.

ps. this is a feature request page, so we can dream right :)

@fperez
Member
fperez commented Nov 24, 2011

@satra, yes: that's precisely what the read-only mode offers. We figured it could be very useful in teaching/demo scenarios.

Adding IP info (with rw/ro indicator) is definitely the simplest solution, and I wouldn't mind having something that also allows for nicknames and finer control over collaborators (such as a dict of passwords for different nicknames instead of just a global one). But it must remain very clear that it's still at heart a single-user system, and that people are giving out their real shell access to others.

@fperez
Member
fperez commented Nov 24, 2011

pps. and yes, dreaming here is OK, that's what this page is for :) Out of these ideas, no matter how crazy, we'll come up with a good list of practical, implementable stuff.

@Carreau
Member
Carreau commented Nov 24, 2011

add a name="someIDwhyNotThePromptNumber" to cell , so that you can reference it by http://x.x.x.x/uuid#cell25

@Carreau
Member
Carreau commented Nov 24, 2011

Still dreaming,
can we parse/get the parsed version of the docstring from the kernel, to have link instead of : :func:'~matplotlib.use' (backquote replace by simple quote ) changed to html link which [open the online doc | open the corresponding help pager | etc. ]

Note that this should be do-able in the qtconsole as well...so that why I'm thinking of passing by the kernel
Maybe we could add an optionnal dependency to sphynx ?

@fperez
Member
fperez commented Nov 24, 2011

Yes, eventually we should definitely add better handling of docstrings with formatting, syntax highlighting, etc (if the necessary dependencies are installed, though it shouldn't be mandatory).

@Carreau
Member
Carreau commented Dec 1, 2011

Will it make sens to have a 'user written' javascript file which will be loaded by the notebook and in which you can just set parameters which are on the browser side only.
That will avoid having to go through Python to 'configure' the notebook. And allow al kind of 'user hack' (good or bad) on the notebook.

I'm thinking of it by seeing #1077 (no-matjax) and IMHO, and my little knowlege on JS, I would have only load a script which looked like

var UserConf = function(...
    [...]
    var matjax.enable=false;
    [...]
    [some other stuff like tooltip or not, collapsed sidepannel]
    []

Always (try to) load it. And check the values. And why not generating it 'on the fly' later with templates as done for the html, have a per notebook/per user file.
You can even (in the case lage noteebook loading) write your own alert('this notebook is huge, would you like to load matjax') to choose at startup what to do...

you can dream of a --js-profile=thisprofile.js on command line, or several open button on the main page which give you loading options by just selectin the js-profile

@takluyver
Member

I would suggest going with json, rather than executable javascript, at least for basic config. Executable config files can't be changed programmatically. We can always allow extensions in javascript for things like checking the size of a notebook when it's opened.

@Midnighter

I very much like a feature that SAGE has but the notebook not (yet). When executing a cell you get a nice 'Busy' sign at the top in the ipython notebook but SAGE also shows this indicator in the tab title. This means I can visit other tabs in my browser and know exactly when the operation has finished.

@fperez
Member
fperez commented Dec 1, 2011

Your wish is our (or rather, @Carreau's :) command: #1084.

@Midnighter

Thanks for the super fast reaction, awesome!

@fperez
Member
fperez commented Dec 1, 2011

@Midnighter: not to be accused of slacking off after an accolade, it's now merged and pushed :)

@Carreau
Member
Carreau commented Dec 4, 2011

About the notebook shortcut.
I often find myself wanting to move a cell Down , and then do a Ctrl-m d which delete it, even if I know that it is Ctrl-m j. As long as the Ctrl-m z or Ctrl-z is not implemented to undo cell deletion, I'll propose to ship 0.12 with dangerous shortcut removed.

@yarikoptic

"Navigation" sidebar

The simplest (for user) imho it would be the ability to navigate sections defined in the notebook. At the moment I break long notebook into sections using markup cells but lack easy ability (besides PgUp/Down and browser's "search") to navigate the notebook.

Thanks

@fperez
Member
fperez commented Dec 14, 2011

Yup, @yarikoptic, we were just having this exact conversation in IRC. The plan is to have special structural cells (rather than trying to infer structure from inline markup). So a cell can be 'level 0', 'level 1', etc, and then we can build into the UI a navigation widget that lets you move around this structure. Furthermore, we want to add tabs like worksheets have, so that it's easier to break up long notebooks into parts.

@takluyver
Member

I'll +1 having collapsible cells.

Also, I think it would be good to have a 'pylab' flag in the notebook format, triggering the kernel to enter pylab mode when you open that notebook, rather than having to start the nb server with pylab. This could be extended to general options for starting the kernel, to cover things like which profile to use.

@bjedwards

I've been seriously using the notebook for a research project now, and I have some feature requests, some possibly simple and one probably not.

  1. In the newest version of the notebook with the menus, have the keyboard shortcuts displayed next to each menu item.
  2. Be able to change the directory in the Dashboard. I've made launcher in Ubuntu to launch the notebook, but it either starts in my home directory (by default) or I have to hardcode the launcher to start somewhere else. It'd be nice to have a single launcher to fire up new kernel/notebook pairs, as opposed to having to do each at the command line. This might make things easier to make launchers on other platforms (there was a discussion on the mailing list about making one for OS X). There was a discussion about an upload dialog on the mailing list as opposed to the simple drag and drop. The drag and drop is insufficient for me because it copies the file rather than keeping it in my version controlled directory.
  3. This is likely the hard one. I'd love it if help via the '?' and tooltip would work while another cell is executing. I have become addicted to the ability to have a cell executing and being able to edit other cells. My workflow often goes:

a) setup long running data analysis code in a cell and execute it.
b) start editing the next cell to plot whatever results come out of the analysis. Having autocomplete and documentation here would make my life so much easier.

I know this would possibly require having another kernel running, and having the two kernels be aware of each other's namespaces. If this were even possible, then it would be useful to not only have a kernel for help and tooltips, but multiple kernels for execution. I think this is similar to how Mathematica works with multiple kernels, but understand that this might be way beyond what the current architecture makes possible. I guess it also might be related to issue #1274?

@fperez
Member
fperez commented Jan 16, 2012

In the newest version of the notebook with the menus, have the keyboard shortcuts displayed next to each menu item.

We'd already discussed this one, @ellisonbg looked into it and it wasn't too straightforward with the current jquery menu code, unfortunately. The menu support in jquery is still highly experimental (we're using a dev branch, not even master, of jquery), so hopefully this will just improve soon from upstream and it will then be easy to do.

Be able to change the directory in the Dashboard. I've made launcher in Ubuntu to launch the notebook, but it either starts in my home directory (by default) or I have to hardcode the launcher to start somewhere else. It'd be nice to have a single launcher to fire up new kernel/notebook pairs, as opposed to having to do each at the command line. This might make things easier to make launchers on other platforms (there was a discussion on the mailing list about making one for OS X). There was a discussion about an upload dialog on the mailing list as opposed to the simple drag and drop. The drag and drop is insufficient for me because it copies the file rather than keeping it in my version controlled directory.

Yes, this is important and high on our list. It needs to be paired with a UI element to shut down the server from the dashboard, though, for the GUI launcher not to simply leave zombie servers around.

This is likely the hard one. I'd love it if help via the '?' and tooltip would work while another cell is executing. I have become addicted to the ability to have a cell executing and being able to edit other cells. My workflow often goes:

Yup, nearly impossible, I'm afraid. It would require running multiple threads in the kernel to be able to respond to client queries while executing user code. Doing that in the heart of the kernel is more or less out of the question, due to the brittleness and complexity that it would introduce. I know exactly why you want it, b/c I've also wanted it pretty badly, but right now I don't see a robust way to provide the feature, unfortunately.

@satra
satra commented Jan 23, 2012
  • clickable thumbnails for embedded pylab images

very often i would like to work remotely on a notebook (on a flight for example and i shy away from it because many of my notebooks have large sets of images that take a long time to load. but most of the time, i'm not interested in seeing all the images. would it be possible to have a configurable option that loads images as clickable thumbnails (low resolution images) that when clicked or hovered on can pull up a high res image?

@ellisonbg
Member

This would be impossible to implement in the current design. It would
require not only a complete refactor of the frontend, but also of the
IPython message protocol and the way the server handles images. I
don't think these things are on the table for now, but we will keep
this in the back of our minds as we move forward.

On Mon, Jan 23, 2012 at 6:14 AM, Satrajit Ghosh
reply@reply.github.com
wrote:

  • clickable thumbnails for embedded pylab images

very often i would like to work remotely on a notebook (on a flight for example and i shy away from it because many of my notebooks have large sets of images that take a long time to load. but most of the time, i'm not interested in seeing all the images. would it be possible to have a configurable option that loads images as clickable thumbnails (low resolution images) that when clicked or hovered on can pull up a high res image?


Reply to this email directly or view it on GitHub:
#977 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@bjedwards

Here's a new one. I've been learning a bit of R and have been using R-studio, with their pretty rad rstudio-server, which is somewhat similar to the IPython notebook. I really like the division of the screen space between the console, editing, plots/help and namespace much like matlab and octave. I much prefer the IPython notebooks cell based editing and layout to Rstudios editor/console, but a frame dedicated to plots might make a cleaner notebook. Inline plots can make my document long and cluttered when I am not making my notebook literate but just doing some quick analysis. If they could all show up in a frame on the right side of the notebook, that would be fantastic. The namespace browser in R-studio can be particularly useful as well, and something like this in IPython would be neat.

My dream layout might be: the entire left portion of the notebook was just for code cells. The right portion would be divided into two halves. The top half would be tabbed with a one tab namespace browser and the other a file browser. The bottom half would have on tab display plots (tabbed or indexed somehow), and the other tab would display help.

I guess this might be cramped on small screens. And certainly some people would want the ability to do inline plots. I can also appreciate that this would be a massive amount of work to alter the layout so massively, so maybe it's unreasonable at the current time, but maybe something to keep in mind going forward?

A Screenshot and More Screenshots

@ellisonbg
Member

Ben,

We are very familiar with the layout of R-studio and there are
definitely some nice things about it. There is a big philosophical
difference between our UI's. The IPython is a has a very linear and
literate feel. R-studio has more of a widget-pane feel. While I
don't think we will move completely away from the linear/literate
feel, we do have plans to introduce some features that would provide
some of the benefits of the widget/pane model:

  • Allowing outputs to be side-by-side with inputs. This helps to use
    the wide aspect ratios of most monitors. We could allow inputs and
    outputs to scroll separately but would need to figure out how to
    enable users to match inputs/outputs.
  • In cell widgets. We have recently added some of the infrastructure
    to make it possible to develop widgets in cells, you could do
    something like the following:

browser = NamespaceBrowser()
display(browser)

and a namespace browser would show up in the output of that cell.
This architecture has a number of advantages:

  • Users and third party devs can develop widgets without our coordination.
  • We don't have to try to cram more and more widgets into a finite
    amount of space. The thing that I don't like about R-studio is the
    crammed look and feel of everything. Everytime someone wants/needs a
    new widget, you just have to cram more in - doesn't scale well at all.
  • We could introduce a L or R panel for extra widgets. Because of the
    clutter of this I am not convinced we want to do this. The current
    notebook UI is super clean and simple and we don't want to loose that.
    Anytime we are tempted to add clutter or complexity, I think we need
    to step back and think creatively about how we can accomplish those
    things in the simplest way possible.
  • Custom UI's. We are also trying to make our Javascript code very
    modular. We are not quite there yet, but in principle it would be
    possible for someone to build a custom notebook UI for themselves. It
    would obviously be a lot of work, but if we can provide reusable JS
    classes, it is not crazy to think about.

On Wed, Feb 8, 2012 at 12:04 PM, Ben Edwards
reply@reply.github.com
wrote:

Here's a new one. I've been learning a bit of R and have been using R-studio, with their pretty rad rstudio-server, which is somewhat similar to the IPython notebook. I really like the division of the screen space between the console, editing, plots/help and namespace much like matlab and octave. I much prefer the IPython notebooks cell based editing and layout to Rstudios editor/console, but a frame dedicated to plots might make a cleaner notebook. Inline plots can make my document long and cluttered when I am not making my notebook literate but just doing some quick analysis. If they could all show up in a frame on the right side of the notebook, that would be fantastic. The namespace browser in R-studio can be particularly useful as well, and something like this in IPython would be neat.

My dream layout might be: the entire left portion of the notebook was just for code cells. The right portion would be divided into two halves. The top half would be tabbed with a one tab namespace browser and the other a file browser. The bottom half would have on tab display plots (tabbed or indexed somehow), and the other tab would display help.

I guess this might be cramped on small screens. And certainly some people would want the ability to do inline plots. I can also appreciate that this would be a massive amount of work to alter the layout so massively, so maybe it's unreasonable at the current time, but maybe something to keep in mind going forward?

We are definitely thinking hard about these things.

Cheers,

Brian

A Screenshot and More Screenshots


Reply to this email directly or view it on GitHub:
#977 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@bjedwards

Thanks for the response. I figured you guys were aware of Rstudio, but didn't know your feelings about it's layout. I agree that things can looked cramp in the Rstudio layout.

Widgets are an intriguing idea. Especially if there is a good API for making new ones. Your example of a namespace browser is kind of unsatisfactory because it shows up in the output of a cell, which might be a few hundred lines of code away from where I want to access it again. Maybe a collapsible pane (left right top bottom), in which widgets could live might be nice. That way the user could do the cluttering in that particular pane.

I don't know if this came across in my comment, but the notebook is totally rad. I used to use emacs and a console (qt or terminal), but now I just do all of it in a notebook, and it is head and shoulders above. So thanks for that!

@ellisonbg
Member

Yes, the inline widgets don't really cover all the usage cases because they scroll off the page.

Glad you are liking the notebook!

@hukka
hukka commented Mar 23, 2012

It's not mentioned here yet, so don't know if it's already planned... but better saving would be really nice. I think that perhaps a data urified html page (a single file) would be the best choice, unfortunately browsers can't save like that even if they can read the file format. I know that there's also MHTML, maff and WebArchive, but none of those are really cross browser.

Would be also nice to have ipython do the PDF processing too, as saving from the browser usually seems to get pretty badly formatted pages.

With those things the notebook could replace latex for many things. With future interactive elements, the html version could vastly surpass it.

@hukka
hukka commented Mar 23, 2012

More on the dreaming side, some kind of sandbox would be really nice. Right now the ssl and password protects unauthorized users from accessing the notebook, but there's no way to limit the control those users have on the file system. I could create a new user just for ipython, but for quick and dirty "Hi Bob, my dear co worker, check my calculations and the results here" it would be nice if Bob couldn't also ls my whole home folder, not to mention run shell commands as my user.

@takluyver
Member

Until that happens, it should already be possible to show Bob a read-only view. Obviously that's less useful if you want Bob to be able to tweak your calculations, though.

@EnricoGiampieri

Hello to everyone!
I've been using the IPython notebook in the last weeks and i fell completely in love with it!
The whole idea of literate programming got me, and right now I am using it regularly, making commenting my code and deploying reports a lot easier..almost fun (but don't tell my boss :) )

There are just a couple of small details that, without being a problem, could really improve my user experience:

Freezing text: sometimes i just want to read the text that is formatted as markdown, without modify it. Could it be possible to add a check box or something similar that prevent the text to enter the editing mode, showing only the formatted HTML?

Commenting and uncommenting the code: some button to just comment a whole block...just a small thing, but very useful in the exploratory phase, without having to repeatedly push those #

an optional way to use the MarkDown formatting on the output of a text: nothing big, but using sympy for symbolic calculus would look very very cooler if i could see the latex formatted formulas instead of the ascii version.

Thank you very much for this wonderful piece of software, and keep the good job!

@takluyver
Member

You can already get Sympy expressions rendered in latex - run %load_ext sympyprinting

@EnricoGiampieri

Thank you, worked like a charm!

@ivanov
Member
ivanov commented Apr 6, 2012

@satra already mentioned optionally saving the workspace - it'd be nice to actually do that selectively - like adding a _sticky_ attribute to things that you want saved along with the notebook, so that anyone else opening the notebook would have access to it. This could then live in the previously mentioned "preamble" type of worksheet.

I can imagine a little dialog that pops up with the info available in %whos and allows you to place checkboxes next to the variables you'd like saved with the notebook

@wkerzendorf

First of all: What you guys have done is mind-blowing. I have just started to use the new ipython notebooks and they are awesome!

My feature requests is an interactive plotting in the html5 notebook and matplotlib. I know there's the html5 canvas for mpl and I wonder if it's possible to combine those. Is this in the works? If so it would be nice to add multitouch support for tablet users (not in the html5 canvas currently).

again, great work!

@fperez
Member
fperez commented Apr 11, 2012

On Wed, Apr 11, 2012 at 6:28 AM, Wolfgang Kerzendorf
reply@reply.github.com
wrote:

My feature requests is an interactive plotting in the html5 notebook and matplotlib. I know there's the html5 canvas for mpl and I wonder if it's possible to combine those. Is this in the works? If so it would be nice to add multitouch support for tablet users (not in the html5 canvas currently).

Yes, this is very high on our priority list. It needs some
refactoring of our JS code, which we're trying to accomplish as fast
as possible, b/c that is a necessary first step for many of the things
we're trying to do next...

@jakobgager

Hi, and thanks for the great work!

Two small issues I would really like to see:

  1. Cell folding: If you have longer scripts inside a cell it would be very convenient to fold the cell similar to syntax folding in vi
  2. sympyprinting (-> really really nice feature): possibility to render matrices (numpy arras) as well (OK, up to 2 dimensions :) ). This way nice formatted matrix algebra would be possible as well.

Thanks again!

@jorisvandenbossche

Another idea: highlighting of matching brackets. The qtconsole does this, and i find it often useful to have a better overview of what I have written.

And thanks for the wonderfull IPython! (only a bit pity that Spyder does not support it anymore)

@aheilbut
aheilbut commented May 4, 2012

Another idea: it would be nice to have a panel listing the variables in the current scope which one could click on to inspect, a à la RStudio or matlab

@EnricoGiampieri

A very simple and small upgrade could be to add, under the number of the block, a little block of text that show how long the code has been running or, if it has finished, how long did it take to complete the step.
in two words, implicit timing (of the code).

I like the idea of the variable exploring, but maybe instead of a panel (that can be cumbersome), just a special magic that return a well formatted html would be more interesting (for example you could insert a "preview" of the object representation or something like that)

@EnricoGiampieri

On a second thought, showing the line number next to the code (very small, and non intusive) would be useful if you have a function with ore than a dozen line and a syntax error pop up. The system gives you the line number, but it is worthless without any number line.

It would be useful also for the explanation of the code: "read line 6" sound a lot better than "look at that line in the middle that start with foo.foomethod"

@Carreau
Member
Carreau commented May 4, 2012

@EnricoGiampieri We decided to keep the UI in notebook as simple as possible, but it should (soon) be easy to implement your own JS/html widget for the notebook.

The magic system will also be soon reimplemented,for all magic to be standalone object easily extensible. You can try to have %who returning a object with a _repr_ and a _repr_html_. I've tried with %ls to have nice list with icons but html/css subset supported in qtconsole is really small compared to recent browser.

As for the line number in notebook, it's already a feature use the C-m l(lower casse L) shortcut. See C-m h for the whole list of shortcut.

@takluyver
Member

I wonder if we should extend the repr protocol in some way to
distinguish a full browser from the Qt console. E.g. if you want a
repr that requires proper HTML support, you could define something
like repr_rich_html, or repr_html5, which would take precedence
over repr_html for clients that supported it.

@Carreau
Member
Carreau commented May 4, 2012

Do we really want more _repr_ than frontend ?
I two html repr will be confusing.
Object with custom _repr_ representation can just have a class flag to toggle between real and minimal html representation.

@takluyver
Member

Well, we already have more _repr_ than frontends: pretty, html, latex, javascript, json, svg, png and jpeg. I agree that adding another type of HTML would be confusing, but on the other hand, I don't want the Qt console to stop people developing interesting HTML reprs that other frontends could use. Another approach might be for the Qt console to detect HTML features it doesn't support, and fall back to another repr; I don't know how hard it would be to implement that.

I'm not sold on the class toggle idea - it requires manual intervention from the user, because the kernel doesn't know about the frontend in use, and it'll be awkward if you have a notebook and a Qt console attached to the same kernel. The display protocol is designed for precisely this: letting different frontends display the same thing in different ways.

Of course, maybe the benefit of rich HTML reprs isn't worth the added complexity of handling them. The important question is: what cool stuff could we do with rich HTML reprs in the notebook that isn't possible with plain HTML?

@Carreau
Member
Carreau commented May 4, 2012

hum... I don't think video works in QTconsole, neither does audio.

You can also get rid of a big layer of complexity with using HTML5. I've tried to make a Graphically improved %ls command, (à la termkit, which seem pretty cool, but I was never able to make it works..)

termkit screenshot
and it is a pain in the ass with qtconsole html.

First because you have to embed images, second because you have a really small subset of css, which prevent you from doing things easily.

At the beggining I was thinnking of smth like <li class='fileExtension'> filename </li> then just a css multicolumn layout, and a css before: for a icon that was depending of the filetype. And i'm now using nested table/tr/tds, transforming the list of file into a matrix, to transpose it just because I still want ls result to be arranged in columns... but it works (will make a PR when magics are refactored)

qtconsole pretty le proof of concept

I guess we can also try to have the qtconsole pop up/slide a webkitview if it gets html5 contents.

So the pain to have one more repr now might be worth for later things we don't think of right now (or things other will do)...

@minrk
Member
minrk commented May 4, 2012

The QtConsole barely supports HTML - it's basically good for HTML as RTF (a/b/img/table). What we want to (and do) support in the notebook is arbitrarily complex dynamic web content.

I don't think people are really writing HTML reprs for the QtConsole, so the main thing would be for the qtconsole to somehow guess if a object fits in the tiny subset of HTML that it supports, or not. Simple checks might be:

  • any reference to http[s], for loading external content
  • frame/iframe tags
  • scripts (script tags, and maybe onclick/etc. trigger entries)

I expect those together would cover ~all of the interactive objects that it doesn't support.

I would like to see something from people who are using HTML reprs in the QtConsole, because the answer might be that the QtConsole simply shouldn't claim to support HTML, even though it does support static HTML.

@minrk
Member
minrk commented May 4, 2012

Honestly, the QtConsole should probably be a WebKit view, not a RichText widget.

@takluyver
Member

Can we interact with a Webkit view as tidily as with a RichText widget, or would a lot of the logic need to go in javascript?

@minrk
Member
minrk commented May 4, 2012

I have no idea - it may not actually be reasonable at all. But the cursor navigation and such is a bit of a mess in the RichTextWidget, partially because there's no DOM. Simple things like append X after Y are quite complicated as a result. That's why we still have issues where you can lock up the QtConsole by having a weird sequence of messages that put the cursor in the wrong place, or end up being allowed to edit areas that are not input.

@Carreau
Member
Carreau commented May 4, 2012

Webkit as no notion of cursor, so we would have to re-implement all cursors mode for selection ... etc
But then, would the qtconsole just be 'another notebook' ?

@Carreau
Member
Carreau commented May 6, 2012

Do you think a ipython notebook --shutdown-kernels [file.ipynb | kernel-id | notebook-id] could be a good idea to complete the ability to open .ipynb files from the command line ?

@ellisonbg
Member

I don't see how we could support the ability to kill individual
kernels. The problem is that the notebook server would see those as
dead kernels and try to restart them.

I do think it would be nice to have the ability to start and stop the
notebook as a daemon:

ipython notebook start
ipython notebook stop

But that would start/stop the entire notebook and all of its kernels.

On Sun, May 6, 2012 at 5:00 AM, Bussonnier Matthias
reply@reply.github.com
wrote:

Do you think a ipython notebook --shutdown-kernels [file.ipynb | kernel-id | notebook-id] could be a good idea to complete the ability to open .ipynb files from the command line ?


Reply to this email directly or view it on GitHub:
#977 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@takluyver
Member

On 7 May 2012 17:49, Brian E. Granger
reply@reply.github.com
wrote:

I don't see how we could support the ability to kill individual
kernels.  The problem is that the notebook server would see those as
dead kernels and try to restart them.

Could we not ask the notebook server to stop a particular kernel?

Thomas

@ellisonbg
Member

Yes, we could do that by simply making the appropriate http request.
That would require knowing the kernel id though.

On Mon, May 7, 2012 at 10:00 AM, Thomas Kluyver
reply@reply.github.com
wrote:

On 7 May 2012 17:49, Brian E. Granger
reply@reply.github.com
wrote:

I don't see how we could support the ability to kill individual
kernels.  The problem is that the notebook server would see those as
dead kernels and try to restart them.

Could we not ask the notebook server to stop a particular kernel?

Thomas


Reply to this email directly or view it on GitHub:
#977 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@Carreau
Member
Carreau commented May 7, 2012

We could kill individual kernel by sending a delete to [notebook-server-url]/kernel/kernel-id. That what I'm dooing in #1676.
The frontend is just complaining that the kernel has died...

@tkf
tkf commented May 7, 2012

I am writing an Emacs IPython notebook client because writing code in browser is pain for me.

Plotting in Emacs IPython Notebook

For writing a new client, I had a chance to see notebook internal so I have a few ideas/requests, not only for Emacs client but also for improving HTML client.

  1. Isn't it better to hold notebook data on server side so that JS client does not need (re)serialize notebook data to save it? I guess it has many benefits:

    1. It simplifies JS code.
    2. It makes easier to test as you can implement it in pure python.
    3. It helps implementing ability to manipulate notebook from command line like in this PR (add file_to_run=file.ipynb capability to the notebook). For example, isn't it better to clear output cells in-place using command, so that you can write a shell script to backup and commit clean IPython notebooks automatically?
    4. And of course, it helps me implementing Emacs client, and possibly other IPython user implementing another client!
  2. Loosely relating to the previous idea, it will be nice to have synchronization support for notebook clients. For example, if you open a notebook in two browser tabs (or in different computers), it would be nice to see one notebook client update notebook automatically when you save the notebook (or even whey you execute a cell) on the other client.

    It will be very nice if I can open same notebook in Emacs and browser side by side, write notebook in Emacs and see the result updated in browser.

    I think synchronization support is important when you implement autosave because you can accidentally override saved data when opening two clients.

    I see synchronization support is discussed above. What is the state of this?

  3. How long do you plan to support draft 76 protocol? As websocket.el (Emacs websocket client) does not support RFC 6455 yet, it will be nice if you support it, at least until tornado support it.

@minrk
Member
minrk commented May 7, 2012

Some points:

Isn't it better to hold notebook data on server side so that JS client does not need (re)serialize notebook data to save it?

The notebook struct will never live in-memory on the server, as this obliterates scalability for the planned multi-user notebook server.

It simplifies JS code.

Actually, it doesn't. Moving the nb struct to the server also does not simplify the javascript code, as it increases the number of server communications dramatically. Managing the json that is the file-format is a very simple part of the JS, so there is not much to simplify there. Almost all of the javascript is for handling dynamic interaction, which is not associated with managing the file format itself.

It makes easier to test as you can implement it in pure python.

In a manner of speaking, yes - though it doesn't help test the Javascript at all, which is where we desperately need tests. And as discussed above, it doesn't significantly reduce the amount of javascript code there is to test.

It helps implementing ability to manipulate notebook from command line like in this PR (add file_to_run=file.ipynb capability to the notebook). For example, isn't it better to clear output cells in-place using command, so that you can write a shell script to backup and commit clean IPython notebooks automatically?

True, though this is already quite easy. You can load the Notebook object in Python, and work with it and a KernelManager without much difficulty. There could certainly be a higher-level wrapper to this model, but it would be very light indeed.

It will be nice to have synchronization support for notebook clients.

We absolutely do have diff-based Save on the roadmap, where only changes are propagated from client to server. This will finally make decent live collaboration feasible, which is our single most-requested and highest-priority new feature.

What is the state of this?

The state of this is "planned", but nothing yet implemented.

How long do you plan to support draft 76 protocol

I imagine we will support draft76 as long as tornado does - we don't do any work to support it other than switch tornado's default from on to off in recent versions, so I don't see any reason to disable it. If tornado drops support, we likely won't do any work to restore it, so you would have to start requiring less-than-current tornado.

@tkf
tkf commented May 7, 2012

The notebook struct will never live in-memory on the server, as this obliterates scalability for the planned multi-user notebook server.

I can imagine, but still unclear. Can you elaborate it a little bit more? You already have shared kernel, right? How can having notebook in-memory become a bottleneck? I have very limited experience on server side programming so maybe I missed some point.

It simplifies JS code.

Actually, it doesn't. Moving the nb struct to the server also does not simplify the javascript code, as it increases the number of server communications dramatically.

For code cell, it does not, right? Because you need send it to server anyway. For other types of cell, you are right. I missed that point.

It will be nice to have synchronization support for notebook clients.

We absolutely do have diff-based Save on the roadmap, where only changes are propagated from client to server.

Great! I was uneasy about sending all data from clients. Diff-based approach also solves the problem.

But again, what is the difference between the in-memory approach and this diff-based approach? Once you have diff-based synchronization, you can also save notebook data by transmitting only differences, right? Then, hypothetically, you can easily hook notebook saving to execution of code cell or rendering of other cell types. I guess this is almost equivalent to in-memory approach. The only difference is that data is in-memory or always stored in the file. Of course, you may not want to hook save to execution/rendering as it increases file access (although I don't think it is a bad idea if you are using it in single user environment). But I don't see much difference on implementation, except the point you mentioned about scalability. Especially on JS side, I imagine that the complexity will be equivalent.

(Well, that means it does not decrease client side complexity anyway...)

This will finally make decent live collaboration feasible, which is our single most-requested and highest-priority new feature.

What is the state of this?

The state of this is "planned", but nothing yet implemented.

How about making an issue on the github tracker then? If it is frequently asked feature, I imagine people want some place to get notification when there are some movements.

How long do you plan to support draft 76 protocol

I imagine we will support draft76 as long as tornado does

Good news for me. Thanks!

@tkf
tkf commented May 7, 2012

I have another question about diff-based approach. Let's consider the following situation:

  1. You open a notebook A on client C1.
  2. You edit the notebook A, which makes diff D. (You don't save A yet.)
  3. You open the notebook A on client C2.

How can you synchronize C2 with C1 if you don't have the diff D in server memory? Of course, you can ask C1 to send D it to server but it will be hassle to implement that.

Or do you mean synchronization occurs only at the saving time? That means difference of notebook contents between clients may grow than always-synchronize approach. You can increase frequency of autosave, but again, it just moves the notebook structure to server memory to server file system.

@minrk
Member
minrk commented May 7, 2012

I can imagine, but still unclear. Can you elaborate it a little bit more? You already have shared kernel, right? How can having notebook in-memory become a bottleneck? I have very limited experience on server side programming so maybe I missed some point.

Current server state:

  • set of connections to kernels, and mapping of notebook-ids to kernel connections.

That is all. The size of notebooks has no lasting effect on the server, and only affects the cost of save/load operations (network transit to server, and save to filesystem or database). Further, this is readily sharded across multiple server instances once numbers are large enough.

If the server has to track which notebooks are open/closed in memory, this is a great deal more state, logic, and load in the server. For instance, if you have notebooks with many large figures, it becomes easy to run the server out of memory just by opening them, if they are all kept in memory.

But again, what is the difference between the in-memory approach and this diff-based approach?

There are two things for what 'in-memory' means:

  1. The server is the object that creates the notebook from the message protocol.
  2. The server has access to the notebook, and is the source for retrieving its current state, and possible updates.

case 1. will likely not happen for scaling and complexity reasons detailed above, given what tiny benefit it would provide.
case 2. differs only as a server-side implementation detail, whether the server persists the full notebook in a database, the filesystem, or memory. The important difference is that all notebook operations are initiated by Clients, none on the Server. A change to the notebook will never happen until a client asks it to.

Once you have diff-based synchronization, you can also save notebook data by transmitting only differences, right?

Sorry I wasn't clear. We will implement diff-based save first, and once we had this language established, sync should be feasible.

How about making an issue on the github tracker then?

There is already an issue for the diff-based save: #1583. I suppose the collaboration could be tracked separately as well, but it obviously won't happen until #1583 is finished. We haven't quite figured out how best to decide when feature requests should leave this meta-issue, because there are so many that we want to avoid drowning in individual feature requests.

@minrk
Member
minrk commented May 7, 2012

I have another question about diff-based approach.

I think we will have to make a lot more progress exploring implementations before we can have a fruitful discussion of what this kind of sync will look like.

@tkf
tkf commented May 7, 2012

Thank you very much for the explanation.

  1. The server is the object that creates the notebook from the message protocol.
  2. The server has access to the notebook, and is the source for retrieving its current state, and possible updates.

So this case 2 is the diff-based you were talking about? This is what I have in mind in the first place also (although I mean to store notebook data literally in memory, not on file or DB). I didn't mean to send 'insert_cell_below' from client and got the new cell from server (this is the case 1, right?). Sorry that I was not clear.

We will implement diff-based save first, and once we had this language established, sync should be feasible.

So notebook server will save to the (temporally?) file before any sync is done, right?

We haven't quite figured out how best to decide when feature requests should leave this meta-issue, because there are so many that we want to avoid drowning in individual feature requests.

Well, I think top request serves the right to leave this meta-issue! And, if the state is "planned" as you said, it's not a request anymore, no? But I guess it's organizing issue so you should know it better than me.

Anyway, thanks for the pointer to the diff-based save issue! I will watch that issue.

@tkf
tkf commented May 8, 2012

I think we will have to make a lot more progress exploring implementations before we can have a fruitful discussion of what this kind of sync will look like.

Actually, if case 2 is the diff-based approach you are talking about, my question is solved because the server is "the source for retrieving its current state".

@ellisonbg
Member

A few more points (@minrk has made many of these already):

  • On the server side notebooks no nothing about kernels and kernels no
    nothing about notebooks.
  • In the future, notebooks may be stored on servers that are
    completely different from those running kernels.
  • This is a deliberate architecture decision that is important for
    scaling purposes.
  • We do need to figure out a way to allow more efficient saves. The
    top candidates are to 1) just save diffs or 2) don't send back the
    large images and other output - cache it on the server side.
  • Allowing cell level synching of two notebook instances is very
    desirable but it is going to be very tough to implement. When we get
    to this point, it is not clear how we will store notebooks on disk. I
    have a feeling a flat file of JSON might not be the best approach. We
    might need something that is more structured.

On Mon, May 7, 2012 at 5:53 PM, Takafumi Arakaki
reply@reply.github.com
wrote:

I think we will have to make a lot more progress exploring implementations before we can have a fruitful discussion of what this kind of sync will look like.

Actually, if case 2 is the diff-based approach you are talking about, my question is solved because the server is "the source for retrieving its current state".


Reply to this email directly or view it on GitHub:
#977 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@tkf
tkf commented May 16, 2012

Hi again,

Is it possible to provide meta IPython information from the IPython notebook server using JSON API? For example, send JSON containing IPython version, Kernel protocol version (if any), Notebook version, or something more when client access "/info".

The primary reason why I want this feature is that I am developing a notebook client, so I wnat know if the protocol is supported. But I guess it will be useful if you are going to support notebook in qtconsole (qtconsole as notebook client) and if this client connects to server via HTTP (maybe not, as you can directly connect to the kernel). Or, you can use that information to make "About" page (well, that's really tiny point I guess...)

I can write a PR if it is OK feature but there is no one willing to implement it.

Thanks.

@ivanov
Member
ivanov commented Aug 22, 2012

from an attendee of the UC Berkeley python bootcamp August 2012:
have the cell background change to indicate that the code inside it has changed after execution, if it hasn't been reexecuted

@crbates
crbates commented Aug 23, 2012

Two things:

  1. The list of notebook to-do's in the first post in this issue now 404's just FYI
  2. There was some discussion in January on the mailing list about a "presentation view" and or adding support for embedding cells/notebooks in html5. I know @ellisonbg was working on the JS backend earlier this year to help enable this sort of thing but what still needs to be done?
@Carreau
Member
Carreau commented Aug 23, 2012
  1. There was some discussion in January on the mailing list about a "presentation view" and or adding support for embedding cells/notebooks in html5. I know @ellisonbg was working on the JS backend earlier this year to help enable this sort of thing but what still needs to be done?

@ellisonbg is really busy until mid september.
you can have a look at
http://nbviewer.ipython.org/

it uses https://github.com/ipython/nbconvert
Which need work to be integrated into IPython so that you could directly export to html.

by presentation do you mean "static view" or "slideshow mode" ?

@pleabargain

I scanned and searched for some mention of a hot key for editing the cells in the notebooks... I for one would love the ability to use a hot key combo to create a text cell or a code cell without having to mouse up. For creating tutorials in notebook, which is what I'm doing, a hot key option for cell types would be AWESOME!

@Carreau
Member
Carreau commented Nov 26, 2012

Short from my phone.
Ctrl+m, h. Or help menu -> shortcut list

Ctrl+m, b -> create cell below

Ctrl+m, m/y convert to markdown/python cell.
Le 26 nov. 2012 20:35, "pleabargain" notifications@github.com a écrit :

I scanned and searched for some mention of a hot key for editing the cells
in the notebooks... I for one would love the ability to use a hot key combo
to create a text cell or a code cell without having to mouse up. For
creating tutorials in notebook, which is what I'm doing, a hot key option
for cell types would be AWESOME!


Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-10730175.

@ivanov
Member
ivanov commented Nov 26, 2012

@pleabargain you mean like Ctrl-m a and Ctrl-m b to create a new cell above or below where you are, and then use Ctrl-m m to switch it to markdown? Ctrl-m t to switch to raw, Ctrl-m y to switch back to code cell.

Ctrl-m h will show you the rest of the shortcuts.

@ghost
ghost commented Dec 1, 2012

Hi , I didn't go through all the above in detail so these requests may have been discussed.

  1. I was wondering if there is any plan to implement a simple find and replace under the edit menu in the current notebook. I can see how short cut keys would conflict with the browser, but an option under a menu would still be useful.

  2. What about having the ability for a collapsible folder structure under the IPython Dashboard. This would allow you to create a .ipynb under a folder in your notebook home on the file system.

Thanks

@ghost
ghost commented Dec 15, 2012

Hi everyone, I'd like to make a suggestion:

How difficult/realistic would an ipython notebook "Export to PDF" option be?
I like to do homework assignments in the notebook, but when I save the page as pdf (via google chrome's printing option), the page breaks often break up code snippets, go through plots etc.

@Carreau
Member
Carreau commented Dec 15, 2012

Hi @rapidfyre007 ,

What you asked is nbconvert, and is on its way.
You can test if you wish.

(Firefox works better to split on the right placed when dooing "print")

@LachlanGunn

I am hoping to begin working on a spreadsheet-like interface for matrix entry. One of the irritating things about Python, Matlab, and such is that quick data entry feels slower because a big chunk of key presses are syntax rather than data. The result is that I end up using Excel and when things escalate ending up with an OFFSET(OFFSET(OFFSET(-laden monstrosity.

Is this something that others would be interested in as well?

@jasongrout
Member

Definitely. In fact, it would be really cool if it was a widget that was directly embedded in a codemirror cell. So I could imagine something like:

a=[spreadsheet widget]

in a code cell. Press a button or key, and that javascript widget flips back and forth between the list of lists syntax and the widget.

Then anytime I have a list of lists, I can highlight the list of lists and click your spreadsheet widget icon and I'll get the list of lists as a spreadsheet form, right there in the code cell.

As another example, Mathematica has an array input that is activated by doing ctrl-comma to add a column, and ctrl-enter to add a row. So if you want to enter a small array of 3x3, then you do: C-, C-, C-, C-enter C-enter C-enter and then you have a 3x3 grid of little empty square placeholders. Then the normal arrow controls are used to navigate and enter data. See also the documentation: http://reference.wolfram.com/mathematica/howto/InputAMatrix.html

@LachlanGunn

That makes some sense. The only difficulty then would be in parsing the code to find the array. I was originally thinking of using a dedicated cell with keymaps to make CodeMirror behave more like a spreadsheet on the text, but your way is better.

@Carreau
Member
Carreau commented Jan 2, 2013

Just bind

you don't need to find the array, you can just ask the user to select it, and replace the selection.

The second is the only one that have to be merged into IPython itself, the rest can be done fully through custom.js and async loading.

@ellisonbg
Member

We are very much interested in this and we have plans to start working on
our architecture for interactive widgets like this very soon. A
spreadsheet widget has been discussed specifically. But our vision is that
these things would not be in the input area, but rather the output area.
But that is a very minor point.

This may seem to be a bit confusing, because of how we refer to "input" and
"output". Our vision of the output area has grown far beyond output and is
more of a generic and dynamic area for media, code, widgets, etc. The
reason we want to keep the input area to pure input is that the notebook
format's output area has the ability to handle all of the media, code and
widgets, but the input area is reserved purely for text based program code.

Cheers,

Brian

On Wed, Jan 2, 2013 at 2:57 AM, Lachlan Gunn notifications@github.comwrote:

I am hoping to begin working on a spreadsheet-like interface for matrix
entry. One of the irritating things about Python, Matlab, and such is that
quick data entry feels slower because a big chunk of key presses are syntax
rather than data. The result is that I end up using Excel and when things
escalate ending up with an OFFSET(OFFSET(OFFSET(-laden monstrosity.

Is this something that others would be interested in as well?


Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-11804842.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@astrofrog

I'd like to be able to not have to start a notebook for each directory I have notebook files in. It would be better (IMHO) if there was a single server instance, with a file browser built-in to the notebook, rather than having to browse to the correct directory and starting up a notebook server there.

@Carreau
Member
Carreau commented Jan 12, 2013

I'd like to be able to not have to start a notebook for each directory I have notebook files in. It would be better (IMHO) if there was a single server instance, with a file browser built-in to the notebook, rather than having to browse to the correct directory and starting up a notebook server there.

It is the plan in the end, it just have to be coded.

@ellisonbg
Member

Yes, early work on this has already begin. It will probably take a few
months as we really want to do this right and there are many ways of doing
it wrong.

Cheers,

Brian

On Sat, Jan 12, 2013 at 12:27 PM, Bussonnier Matthias <
notifications@github.com> wrote:

I'd like to be able to not have to start a notebook for each directory I
have notebook files in. It would be better (IMHO) if there was a single
server instance, with a file browser built-in to the notebook, rather than
having to browse to the correct directory and starting up a notebook server
there.

It is the plan in the end, it just have to be coded.


Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-12183840.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@dwwkelly

I'd like to be able to reference python variables that exist in a python cell in a markdown cell. I haven't found any similar requests on github but I found this request for it.

@ellisonbg
Member

Here is a pull request implementing thisL

#2592

It would be great if you could try it out as we are still finishing the
design and would love feedback.

Cheers,

Brian

On Tue, Jan 15, 2013 at 10:53 AM, Devin Kelly notifications@github.comwrote:

I'd like to be able to reference python variables that exist in a python
cell in a markdown cell. I haven't found any similar requests on github but
I found this requesthttp://mail.scipy.org/pipermail/ipython-user/2011-October/008643.htmlfor it.


Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-12282814.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@certik
certik commented Jan 25, 2013

Allow to upload files

Frequently I calculate some results in Fortran and save them to text files. Then I want to plot it or analyze it in the notebook. So I have to maintain & archive both the notebook and the data files. (My data files actually happen to be really small, for example today I have two data files of 35kB each.)

Would it make sense to somehow embed the files with the notebook?

That way the notebook would be truly self-contained. Another usage would be when the notebook is hosted online (or on another computer) and you want to upload the data files through web browser so that you can access it.

Currently the only way is to copy & paste the contents of the file into a cell (as a string for example).

@Carreau
Member
Carreau commented Jan 25, 2013

I would suggest you use the %load magic to put the file in the next cell, and then use the %file magic
to seamlessly write it on disk when the cell is executed.
It can then be easily shared.

The problem with "embeding data" is that the kernel is unaware of the fact that it speaks to a notebook or not. So how do you send the data to the kernel ? Moreover kernel and webserver can be on 2 different machines, so in many casses embeding the data does not make sens.

It will be as painfull to embed the data in the ipynb file each time you regenerate it than to archive both together.

Json is also really bad at embedding binary data (33% overhead of base64...). Tar is made to attach file together, and I don't think we should reinvent the wheel. Still notebook can have metadata and nothing prevent you from attaching data in those field, but I don't think it will be supported in the core.

I think there is also a discussion like that elsewhere, and the idea of ipynb 'folder' did emerge. but in the end, it will be the same when you will be able to browse disk : 1 project, 1 folder with data and ipynb file in the same folder. then you can share this folder.

It is viable as nbviewer for example allow you to browse multi-file gist and to link from your notebook to siblings files that shoudl show up magically.

@certik
certik commented Jan 26, 2013

Nice, I wasn't aware of %load and %file (resp. %%file, as %file didn't work for me). That works great. Except I am a little worried of accidentally modifying the cell with the file. Also, it occupies quite a lot of space, depending on the number of lines in the file.

But it does the job. Thanks!

@ellisonbg
Member

Ondrej,

A good while back we decided that the proper way of including other assets
with notebook document was to bundle everything into a git repo. Otherwise
the notebook files themselves would gradually morph into something like a
repo in JSON, gzippped dir bastard of a file format. Plus git repos are
designed for this purpose precisely. But we do have plans on making it
possible to edit file on the file system using the notebook.

Cheers,

Brian

On Fri, Jan 25, 2013 at 4:08 PM, Ondřej Čertík notifications@github.comwrote:

Nice, I wasn't aware of %load and %file (resp. %%file, as %file didn't
work for me). That works great. Except I am a little worried of
accidentally modifying the cell with the file. Also, it occupies quite a
lot of space, depending on the number of lines in the file.

But it does the job. Thanks!


Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-12727016.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@certik
certik commented Jan 26, 2013

Brian, good decision. So in this case, I think it might be a good idea to allow:

a) upload files over the browser

b) click a button and upload the notebook + other files into a gist.

But for my local usage, I will continue simply using git, as that what I already use.

@amir-f
amir-f commented Jul 16, 2013

Run selected text in an IPython notebook cell

I'd appreciate the ability to run a selected part of a cell quickly. Similar to the way pressing F9 in Matlab works. Often times I realize I want to add another statement to a cell but don't want to run the whole cell again but instead execute that line only.

@Carreau
Member
Carreau commented Jul 17, 2013

@amir-f, Doable in your custom.js in 10 lines,
This is not the correct name for the event to listen for, but I don't remember the correct one.

Not tested, will not produce output anywhere. You should execute on the behlf of the selected cell if you want it to show the output, but you get the principle.

$([IPython.events]).on('notebook_loaded.Notebook', function(){
  IPython.toolbar.add_buttons_group([{
       'label'   : 'run selected text',
       'icon'    : '--icon name to chose in font-awesome--', 
       'callback': function(){
          var selection = IPython.notebook.get_selected_cell().code_mirror.getSelection();
          IPython.notebook.kernel.execute(selection)
          }
    }]);
});
@niallrobinson

I'd love a simple keyboard short cut to insert a time stamp into my notes so I can see when I said what.

Also, I know this might be more a markdown thing, but I'd like some basic form of checklist, like in GitHub flavoured markdown. It would also be nice to be able to make simple tables without having to resort to putting them in a separate html cell, like with this markdown
https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#wiki-tables

@aflaxman

@niallrobinson I've been timestamping my notebooks with !date, maybe that will work for you, too.

I would like to paste images into markdown cells, a la github comments, the digital equivalent of pasting a xerox into my (physical) notebook. It seems like jquery is ready to do this: http://strd6.com/2011/09/html5-javascript-pasting-image-data-in-chrome/

(Edit: A javascript code snippet to do this was posted to the IPython-Dev mailing list 2/7/2014, and you can see it work here)

@niallrobinson

Great thanks - those are two good suggestions.

@germank
germank commented Sep 23, 2013

Hi!
I was wondering what happened with the multi-level sections idea for the notebook. It would be such a useful addition! When notebooks grow large, they start getting out of control for me. Is that feature in the near plans?
Thanks for the great work!

@takluyver
Member

@germank We know that handling large notebooks gets pretty awkward, and we definitely plan to improve that experience. I don't know quite what or when we're going to do, though.

@niallrobinson

I second @germank - that's the feature at the top of my list. Nice to know its on the way at some point

@epifanio

A sidebar on the left with cells preview scrollable that will scroll the full notebook to the point... Something like in any pdf viewer (see preview on osx) but instead of pages can be used a svg-box with a customizable height... Something Like the overview map in most webmap tools. (see openlayers overview map)
... Hope it make sense, my 2cents

@smackesey

Input history search as is standard in any terminal REPL. I know you can use the browser functionality to search, but this will only capture commands that still have a visible cell, not the whole history of executed commands, and will not allow you to execute it again easily in your current cell.

@JohnGriffiths

_Move cell to end_.

All my notebooks have a 'graveyard' section at the bottom (mine actually has two tiers because I'm a hoarder and feel it's important to separate 'things I dont want but don't want to delete right now' from 'things I don't know if I might want later or not'). In any case some kind of shortcut for shunting cells to the end would be nice.

Small thing I know, but sometimes it can take more than half a minute to do this with copy + delete cell + end + paste + home + scroll back to starting point. Not hard to see how that can add up to half an hour or more in a day.

Personally this leads to something of a conflict between being tidy and being productive. My personality inevitably makes me lean towards the latter, so I often (always?!) end up with notebooks that are otherwise scrappier than they would be during development. And then I spend time going through tidying things up. And I've got stung once or twice removing something I needed.

Maybe this will resonate with many. Maybe not. Cheers anyhow!

@StefanKarpinski StefanKarpinski referenced this issue in JuliaLang/julia Oct 19, 2013
Closed

IJulia notebooks for documentation? #4579

@elyase
elyase commented Oct 27, 2013

It would be nice to have some sort of quick variable inspector. What I have in mind would be something like a tooltip/popup, (similar to the one which displays the function signature when tabbing inside a function) which activates when the mouse goes over the variable and some modifier key(for example Alt(Windows)/ ⌥(Option) on Mac) is being pressed. As soon as the mouse leaves the area over the var it goes away. Screenshot:

variable_inspect

@Carreau
Member
Carreau commented Oct 27, 2013

It would be dangerous as it might have side effect like inspecting next() would advance an iterator.
though you can select and shift-tab, it would pop-up a tooltip on selection.

@ellisonbg
Member

I agree with @Carreau that it could be dangerous at times, but overall I really like this idea. It would not be difficult to implement on top of some of the changes in this PR: #4352

@Carreau
Member
Carreau commented Oct 27, 2013

No, it should be done as refactor of Object-info-request, I think.

@ellisonbg
Member

I agree with you in terms of how it should talk to the kernel, but I was
more thinking of reusing the logic I have in that PR for triggering
functions if the user 1) selects something and 2) presses a key (alt) and
3) selection matches some regexp. That is the perfect user interface for
triggering the underlying objet into request and displaying the result.

On Sun, Oct 27, 2013 at 9:52 AM, Matthias Bussonnier <
notifications@github.com> wrote:

No, it should be done as refactor of Object-info-request, I think.


Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-27172945
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@Carreau
Member
Carreau commented Oct 27, 2013
  1. selection matches some regexp

This might be where it gets problematic. As basicaly you would like to inspect on almost any selection.
Whereas the input widget would actually have a behavior that depend on the regex.

But I hear you, I'm just wondering is this would not go a step too far to actually do a request to the kernel.

@guidov
guidov commented Oct 28, 2013

Now that we have multi-directory support in the master branch (awesome!!) is there a way to implement dashboard navigation (at the /tree level) of all the subdirectories with links (i.e. collapsible folders with point and click) instead of having to type in the tree subdirectories in the url bar?

@minrk
Member
minrk commented Oct 28, 2013

Yes, working on it. Just wanted to get the backend functionality into master first, so it stopped blocking other work.

@thevladsoft

I know this have been asked before, but it could be very useful to be able to comment/unconment a line with a keyboard shorcut or button. In other words, add or remove a # in the beginning of the line.

Thanks.

@minrk
Member
minrk commented Oct 28, 2013

@theviadsoft - this is provided by CodeMirror, via ctrl-/ or cmd-/

@thevladsoft

Well, it wasn't working for me. Tried in tree different browsers. Finally changed it in codecell.js to another key. So I will change my request to:

An easy way to define shorcuts or simply info in the on-line docs about how to do it. (It took me a while to find how :) )

@stared
stared commented Nov 4, 2013

I second @fperez @cmrivers and @Carreau:

multi-cell selection and actions. Right now, there's no way to copy/paste/delete multiple cells.

As of now, it is the key feature I am missing in IPython notebook (and why sometimes I miss Mathematica).

@astrofrog

The idea of a slideshow mode (i.e. interactive, not based on reveal.js) was mentioned several times in this thread - is this something that is planned?

@Carreau
Member
Carreau commented Dec 10, 2013

It is implemented as an extension you can find in here damian avilla also have a prototype of this working with reveal.

@damianavila

The interactive prototype working with reveal.js is here: https://github.com/ipython-contrib/live_reveal
I will probably change some things in the next weeks, but you can use it and blame me right now... 😉

@kkmann
kkmann commented Jan 14, 2014

Well, at the moment the single most annoying issue to me is, that I cannot use the IPython notebook as a frontend to Pweave. How great would it be to have the full power of Latex at your hands for layouting while you run the code chunks truely interactively. I know the workflow from R using Sweave, but no environment has the power of interactive literate programming with publication quality pdf output. So using the %%latex cellmagic comes pretty close, but it would be a dream comming true to have the same control over the output of code chunks as in Sweave (like manually embedding graphs, surpressing code, etc...)

@RinaldoB

Navigate Cells with keyboard

I really would like to navigate from one cell to the next (witthout executing) it with one key press.
Ctrl-<down arrow> kind of works, but Ctrl-<up arrow> does not. Just discoved that Alt-<up arrow> works for navigating up (but not for navigating down).
Could we make it more consitant and document it in the keyboard shortcut section? The Page Up and Page Down keys would be good candidates IMHO.
Edit Ctrl-m n already exists for navigating down

@ellisonbg
Member

In IPython 2.0 you will be able to define your own keyboard shortcuts for
things like this.

On Wed, Feb 19, 2014 at 2:39 AM, RinaldoB notifications@github.com wrote:

Navigate Cells with keyboard

I really would like to navigate from one cell to the next (witthout
executing) it with one key press.
Ctrl- kind of works, but Ctrl- does not. Just
discoved that Alt- works for navigating up (but not for
navigating down).
Could we make it more consitant and document it in the keyboard shortcut
section? The Page Up and Page Down keys would be good candidates IMHO.

Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-35485579
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@takluyver
Member

For that matter, in IPython 2.0 you can go into command mode and navigate between cells with just up/down arrows (or j/k if you prefer).

@EoinTravers

A minor idea, but it would be nice to be able to set variables as True or False using checkboxes.
In my data analysis, I often want to selectively include and exclude snippets of code.

# What to exclude
exclude_subjects = True
exclude_incorrect = True
exclude_moderate_condition = True

Later on:

if exclude_subject:
    do xxx

Checkboxes would be a snazzy way of toggling these options!

@jimmyc42
jimmyc42 commented Apr 6, 2014

Sorting cells?

I have been using the ipython notebook as an actual Markdown notebook, just something to take notes at meetings, etc. Not all meetings are independent, though. I was thinking about a way we could sort cells by date (the way I take them) but also a way to sort them based on some metadata that I can attach to the cell. Then I can organize the notebook relative to some metadata content for quick reference.

This also made me think of the way I code, usually having some kind of parent cell where I read in some data, and then daughter cells where data manipulation, plotting, or prototyping is done. I guess this falls into the idea of code blocks (A collection of related cells). I think that some version of the sorting method described above could be a kind of solution for unwieldy or long notebooks, or at least a way of coming back into a notebook and retasking back to where you left off.
If there is a better way, please let me know.
Thanks.

@RinaldoB

Revision control Integration

My notebooks are inside a git repository for revision control. Recently, thanks to a post on stackoverflow I set up a git filter to strip the output from the notebooks before committing to the repo, enabling an efficient revision control.

To further improve the workflow, I suggest basic integration into the notebook interface itself. In it simplest version, after an initial setup, CTRL+S would trigger a commit command of the output-stripped notebook into a repository (for example: git add <file>; git commit), optionally allowing to enter a commit message (e.g. with CRTL+SHIFT+S)

@arogozhnikov

Tab suggestions to work while kernel is busy

this probably can be implemented by having separate thread for suggestions,
computations sometimes take too much (more than 10 minutes), it it hard to write code without any suggestions.

Thanks for IPython!

@acorbe
acorbe commented May 16, 2014

Allowing single notebook to be split among multiple pages

It would be nice to be allowed to split a single notebook in different sub-pages.

Logic:
At the moment I am using a notebook as a "final front-end" for a python application for data analysis running on a server.

As more and more development is carried out, the notebook (as the "back-end" code) is growing being populated with pictures and code.

At the present time, the notebook got very large and not very handy to work with anymore.

Splitting it in different pages would be handy.

@rossant
rossant commented May 16, 2014

@acorbe would it be sufficient to have collapsable headers? (like in Mathematica)

@acorbe
acorbe commented May 16, 2014

@rossant , It would be an improvement, I guess.
Is this feature present already? (I didn't know, sorry)

Imagine having O(200) cells with codes and picts already "thematically grouped" by titles and so on (my present situation).

It would be cool to be able fold/unfold all the cells in a "single chapter" at a time, as well as to navigate through different "chapters" in a click.

what do you think?
thanks

@rossant
rossant commented May 16, 2014

It looks like it has been implemented as an extension (see #805).

@JohnGriffiths

My two penneth:

Magic combo =

  • toc extension
  • hierarchical collapse extension
  • show/hide all cells extension
  • show/hide single cells extension

On 16 May 2014 13:57, "Cyrille Rossant" notifications@github.com wrote:

It looks like it has been implemented as an extension (see #805#805
).


Reply to this email directly or view it on GitHubhttps://github.com/ipython/ipython/issues/977#issuecomment-43327757
.

@niallrobinson

If I start a notebook up and I want to pick up where I left off, I need to go through and find all the cells I need to rerun to get back to where I was. Would it be possible for the user to create macros to run certain cells in a certain order?

btw I'm using the code folding extension and its no exaggeration to say its completely changed the way I use notebooks. I'd love to see something equivalent supported properly in NBViewer etc.

@niallrobinson

Just one more thought - to do with code/section folding and @acorbe s suggestion of splitting one notebook over different pages...it might be worth considering some kind of system of name spaces. As I've used code folding, and my notebooks have been able to get much longer, I'm finding the occasional accidental conflict with objects from way up in my notebook. Maybe different pages could be different name spaces.

@robertmarley

2 cents: Toggle Input Cells

1. Cell -> Current Input -> Toggle. 2. Cell -> All Input -> Toggle.

You already have this going for output cells. This would be very very useful for illustrations and explanation of many many concepts (teaching essentially).

@dsblank
dsblank commented Jul 19, 2014

Agreed that for education, this would be very useful. And for other uses too. However, due to restrictions, such behavior will probably not be able to be added to IPython.

But, we are working on suitable functionality that would encompass this behavior. Basically, we are thinking of In/Out Tabs for a particular cell.

If you would like further details, feel free to email me directly at doug.blank@gmail.com.

@juhasch juhasch referenced this issue in ipython-contrib/jupyter_contrib_nbextensions Aug 1, 2014
Closed

Can one copy/paste an entire section? #85

@minrk minrk removed the type-enhancement label Jan 14, 2015
@varnion
varnion commented Feb 3, 2015

pandocs-citeproc support

I'd be really nice if we could use citations in markdown cells like those supported by pandocs-citeproc:
http://johnmacfarlane.net/pandoc/README.html#citations

for instance, with this:

    ... in a process called *Muller's Ratchet* [@Muller1964; @Haigh1978].
    .. as suggested by Drake [-@Drake1991].
    My model is based on that by @Hadany2003 who showed...

we 'd get this:
... in a process called Muller's Ratchet (Muller 1964; Haigh 1978).
.. as suggested by Drake (1991).
My model is based on that by Hadany and Beker (2003) who showed...

(I picked up those examples on the blog of Yoav Ram)

Another nice alternative would be proper latex citation syntax, but i think those were already requested.

@takluyver
Member

Citations are possible with syntax like this <span data-cite='key'></span>. See my cite2c extension for a way to insert and manage them automatically. For a nicer syntax, we're waiting for CommonMark to standardise a generic inline extension syntax for Markdown. We've talked to John Macfarlane, the author of pandoc, and he's heavily involved in CommonMark, so we're optimisitic about that.

@varnion
varnion commented Feb 3, 2015

whoa, that was fast. thanks! this syntax is a bit ugly, but your extension will save me a LOT of time.

Now I'll be looking foward to see the progress of CommonMark =D

@bencharb

Recover trashed cells after holding down the "d" key. I've lost many hours of work from accidentally deleting many cells because I've leaned on the "d" key too long, usually when I fall asleep at the keys. The feature could work like this: after leaning on the "d" for about one second, pause the deleting with a modal dialog, "Hit 'y' to continue deleting everything."

@takluyver
Member

I wonder if we can distinguish between the user pressing and holding a key versus pressing it repeatedly. Deleting a cell should ideally require pressing d twice, not just holding it down until it automatically repeats. I would imagine that the answer is 'not in any reliable way across different browsers', though.

Multiple undelete is already tracked as an issue: #4924

@jasongrout
Member

We probably can by detecting keydown and keyup events. At least, I'm pretty sure key repeats don't trigger multiple down/up events.

@takluyver
Member
@minrk
Member
minrk commented Feb 14, 2015

For reference: repeats appear to result in consecutive keydown events without keyup events in between (at least on Chrome + OS X).

If our keyboard shortcuts used keypress instead of keydown, the vast majority of the compatibility and weirdness of browsers and keyboards would go away. There's one caveat, though: we still must use keydown for non-text shortcuts (mainly esc).

@bencharb

Instead of watching keystrokes, could you put a listener on the delete event itself that splashes a nag dialog when it sees repeated deletes?

@fbrundu
fbrundu commented Mar 17, 2015

I think it would be nice on big desktop screens to have a two pages layout of one notebook, what do you think?

@ellisonbg
Member

Yep :)

On Tue, Mar 17, 2015 at 11:18 AM, Francesco notifications@github.com
wrote:

I think it would be nice on big desktop screens to have a two pages layout
of one notebook, what do you think?


Reply to this email directly or view it on GitHub
#977 (comment).

Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger@calpoly.edu and ellisonbg@gmail.com

@brianbreitsch

Hello! I'm not sure if this is already available, but is there a way we can have a javascript hook when the notebook text editor gets loaded. Right now I think custom.js only let's you hook into when notebooks get loaded, but I could be wrong.

I'm trying to do this so I can add a codemirror keymap option to the edit dropdown.

@brianbreitsch

It looks like we just need to add events.trigger('app_initialized.EditorApp'); to main.js under edit.
Question: should the event be 'app_initialized.EditorApp' or something else?

@l00mi
l00mi commented Apr 29, 2015

Is there already a Feature Request for Block comment/uncomment?

@dsblank
dsblank commented Apr 29, 2015

Block comment/uncomment is handled by the editor (CodeMirror). As that is language-dependent, that could be tricky. But control+/ does comment out the block one line at a time, and is very robust... even if the user's selection is a bit off. In any event, CodeMirror would be the place to make the request.

@l00mi
l00mi commented Apr 29, 2015

Yes, just discovered that.. unfortunately it seems only to work on US Keyboard layout..
Could it make sense to uncomment all if having multiple lines selected and hit # ? A bit like indentation works?

@tmearnest

Tabbed notebooks

I was thinking that it would be really nice to have separate "pages" or "tabs" associated with a notebook. These would be essentially subnotebooks that are all connected to the same kernel. The main idea is that it would be easy to separate concerns within the document, e.g. one page for imports, initializations, data loading, another page for defining common functions, then a page for the main calculations. Or if there are multiple parts to the calculation that are logically separable, these could go to different tabs as well. I've been finding myself working with large notebooks with tons of output and it gets cluttered really fast. This kind of organization would fix that problem.

@takluyver
Member

We actually had a plan to have worksheets in notebooks - it was even reflected in earlier versions of the file format. But we didn't do it, and we decided that we probably won't, although we're open to someone doing it as an extension.

We know that working with long notebooks is awkward, and we do have some plans to address this, like collapsible sections or tables of content.

@MInner
MInner commented May 20, 2015

Hello everyone! I have no idea of how it could be implemented, but it feels like a somewhat critical, at least I face this problem very often. If something occasionally outputs some huge chunk of text into it's output then chrome (and I guess the same thing applies to other browsers too) becomes super-unresponsive (consumes entire CPU and stuck) trying to render all that text in a cell output. The most unpleasant part is that if you close this unresponsive tab and try to reopen the notebook, it would very likely tell you that it is corrupted or too huge to be successfully opened :(

Probably, if you could add some internal super-simple profiling thing that could detect (e.g. too much time between two sequential timeout callback calls) those sucks, stop pushing output to notebook cell and suggest user to clear cell (or continue)?

I personally suffer from that effect a lot :( That would be great if you could solve it somehow.

@niallrobinson

I've suffered from this too. It would be lovely to stop this happening. Especially as the only way to recover is to hand edit the notebook!

@InonS
InonS commented Jun 29, 2015

Hi all!
After making what I consider a reasonable effort to find a duplicate, I've decided to chip in with a major request. I know that Jupyter is taking over some of this stuff, but I find it incredibly detrimental to my productivity that I can't refactor code in an IPython notebook. To do so I either have to edit the ipynb file as text, or to export the whole thing to python code, which defeats the purpose of IPython in the first place. I've even looked at some of the python IDEs (PyDev, PyCharm, Spyder) but I think that IPython is only used there as an interactive console, and the entire concept of the notebook is lost.
Firstly, I'd request a "rename" refactor. Later on, maybe some object-oriented stuff like generation of accessor & mutator methods. All of this stuff seems pretty straightforward for the most part, and I'd hate to think of the IPyhton project loosing steam just because productivity is handicapped because of it.
Many thanks to all the developers who make this project happen!

@rossant
rossant commented Jun 29, 2015

@InonS ipymd might be a partial solution: you can replace the ipynb format by a Markdown document, which makes refactoring easier in a good text editor

@InonS
InonS commented Jun 30, 2015

@rossant Thanks! I took a gander, and confirm that the notebookfile file format probably could and should be more portable, e.g. Markdown.
Still, anything that forces me to do my refactoring in an external text editor is a setback, as far as I'm concerned. A user must be able to do all the development in one environment, this includes both running interactively and refactoring.
Thank you for your feedback!

@alvinwt
alvinwt commented Jul 16, 2015

Is there a way to run my scripts in the notebooks that persist after closing the browser? I expect that for simulations that run for days, it would be a good feature to have a screen like feature to detach the notebook and let the kernel run in the background without having the browser open. I use both the Python and IRKernel for my work which requires running the jobs on servers from my laptop, so leaving the browser on is not an option. This is a feature that would be useful to many people for sure.

@varnion
varnion commented Jul 16, 2015

@alvinwt this would be SO AWESOME. the only way I know is using the ipython nbconvert from ipynb to ipynb, but this is a non interactive solution.

@InonS
InonS commented Jul 17, 2015

@alvinwt, for "running the jobs on servers" non-interactively (i.e. batch processing), would you consider ipcluster?
@varnion, I think I don't understand how your proposal addresses the OP's concern. Could you please explain?

@alvinwt
alvinwt commented Jul 20, 2015

@InonS I would try that out if my script is in Python and I have been struggling with understanding the documentation. Currently I work a lot in the IRKernel and it would be nice if the feature is not limited to Python kernels.

@orome
orome commented Aug 19, 2015

I'd like to be able to rearrange multiple cells at at a time by dragging them. It'd be nice if you could even do this between notebooks. Sometimes I find that code for data analysis would make more sense in a different notebook, but it sucks to copy/paste it all.

This seems to me like an essential feature that I was hoping would appear eventually. It never takes much time working in a notebook before I realize that things need to be rearranged, and then I hit a dead end and need to switch to a text editor. The omission of this feature alone (at least just multiple selection and rearranging) shapes my entire workflow: start in a notebook and quickly (after a few hours) get so constrained by the inability to move or delete groups of cells that I switch to a conventional text editor.

@act65
act65 commented Feb 26, 2016

Hey, new to this sort of thing (git etc...). I just wanted to add something to the wishlist that you guys seem to keep.

Can we have a live/interactive version of the markdown preview - like Mou does

screen shot 2016-02-27 at 9 30 52 am

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment