Modal UI #3605

ellisonbg · 2013-07-11T00:27:53Z

The goal of this PR is to fix the various focus related problem in the notebook user experience. The issue for this is #3441. The source of the problem is that we used to call this.code_mirror.focus when a cell was selected. This caused all of the jumpy behavior. However, when this call was removed, the entire user experience fell apart. This PR is an attempt to restore a solid user experience while also removing the problematic focus calls.

The result is that the notebook is now a dual mode editor with a command mode and edit mode. Command mode is entered using ESC or Ctrl-m. Thus our old keyboard shortcuts still work, but multiple of them can be given after you type Ctrl-m. Edit mode is entered for the current cell by pressing Enter. This is not ready for merge, but we are thinking about trying to merge this for 1.0. But, it will need lots of testing. The code is relatively simple, but the new interaction model is quite subtle and there are numerous tweaks we could make.

Another side effect is that multiple markdown, rawtext and heading cells can now be unrendered at the same time.

Overall I think this is a massive improvement of the user experience. But, this needs tons of interactive hands on user experience testing.

Remaining issues:

Test various notebook and cell actions (cut, copy, paste, merge, split, etc.).
Keyboard shortcuts for command mode need to be turned off when the notebook area is not in focus. This is currently preventing dialogs and other things that require keyboard input from working.
Test and debug tooltips.
Test to make sure that the pager can be closed with ESC.
Test and debug tab completion.
Possibly refactor the keyboard shortcut code to make it all configurable.

Questions:

Do we like how Shift-Enter, Ctrl-Enter and Alt-Enter behave?
Should we always enter edit mode when a cell is changed type?
Do people like the green border to indicate edit mode?
Do we want to keep ESC for entering command mode?

haard · 2013-07-11T10:24:01Z

Won't some of those changes make e.g. Emacs bindings even harder to implement by locking in a vim-ish model in the implementation? Would C-c C-c be forced to become C-m C-c C-c?

I'd like to see a default keypress handling that could be overloaded with either vim-style dual mode or emacs style bindings, or whatever is cool next year.

Carreau · 2013-07-11T10:30:21Z

IPython/html/static/notebook/js/cell.js

@@ -38,6 +38,8 @@ var IPython = (function (IPython) {
        this.placeholder = options.placeholder || '';
        this.read_only = options.cm_config.readOnly;
        this.selected = false;
+		this.rendered = false;


indentation.

Carreau · 2013-07-11T11:51:59Z

Had a quick read-through, not enough to understand everything so I would like to have time to look at it more before judging.

I still don't like the if-else block with event number to choose between action, IMHO, we shoudl be able to do that using a dict with value beeing function.

It feel strange to me that switching to command-mode is handeled on a per-cell basis, but as I wrote earlier I want to understand more.

I feel more-and more that this is too big for 1.0, and that we should really discuss dual-mode editting, and give it more than 10-15 day tests.

Carreau · 2013-07-11T11:53:37Z

IPython/html/static/notebook/js/cell.js

     * @method unselect
+     * @return is the action being taken


I don't understand this statement.

you actually have "@return is the action being taken" in lots of places.

it's like an event handler, return true if an action is taken, false if nothing happend

Carreau · 2013-07-11T12:19:45Z

And now some usage feedback.

would bind pgup pgdown to +10 cell - 10cell to move fast.

Carreau · 2013-07-11T12:24:37Z

Completion is broken. Should not unselect the cell on focus out as completer take focus.

Carreau · 2013-07-11T12:27:48Z

Focusing out a cell make it jump to top of screen. Eg I'm editting a cell in the middle of my screen, I reach menu.
It jumps.

Carreau · 2013-07-11T12:35:30Z

The different behavior when clicking in cell vs cell-editor (focus and scrolling behavior) is weird and annoying. Fell too jumpy. Impression that I want to do sometime twice (select a cell) and got 2 different behavior.

Carreau · 2013-07-11T12:42:59Z

shift enter do not select next cell for execution.

Carreau · 2013-07-11T12:54:47Z

I'll revert to master, is is too hard to edit and tweek a notebook with this branch.
It feel right when using only keyboard at beginning, but as soon as you use the mouse to reach stuff and modify code it becomes annoying.

I also have a lot of shortcut in my muscle memory ('C-m,a' , type import), and a few time this screwed up, interrupting the kernel (i) and making my cell a md cell (m) with port in it... sight.

What about Esc beeing the only way to switch to command mode for good, C-m switching only for 1 action, drop back to edit current selected cell ?

jakobgager · 2013-07-11T13:19:14Z

If I just move out of a heading cell its not get rendered as heading cells. It needs to be executed.

Seems like Shift+Enter and Ctrl+Enter are exchanged.

minrk · 2013-07-11T17:55:28Z

If I just move out of a heading cell its not get rendered as heading cells. It needs to be executed.

I think that's by design

minrk · 2013-07-11T17:57:06Z

One comment: I think the cell highlight should be visibly different for edit / command mode

ivanov · 2013-07-11T18:17:26Z

j and k keys MOVE the cells, and there is no way to move the cursor. I suggest making j and k move the focus around, and maybe shift-j and shift-k to move cells around

ptone · 2013-07-11T18:21:50Z

Some quick usability comments from playing around:

the 'h' key for help needs to be displayed somewhere in the UI - maybe only appearing during command mode.

the help screen should have a link that opens a more complete page documenting usage.

I agree with @ivanov about the J,K - as a vim user, it moves where my cursor is, not the line I'm on

maybe keyboard shotcuts could appear with hover-help on toolbar buttons

some more pronounced visual feedback of what mode is active

Should there be a way to disable dual mode completely for beginners?

damianavila · 2013-07-12T00:13:40Z

One weird thing: if I convert as cell to markdown, and later again to a code cell, it seems to lost the grey border of the command mode (we enter the phantom mode...booo... like vim... hehe ;-) @ivanov) . I mean, you can execute it, but if you want to see the border again you have to alt+enter or ctrl+enter.

ellisonbg · 2013-07-12T00:25:43Z

Can you give the exact keystrokes you used?

Sent from my iPhone

On Jul 11, 2013, at 5:13 PM, Damián Avila notifications@github.com wrote:

One weird thing: if I convert as cell to markdown, and later again to a code cell, it seems to lost the grey border of the command mode (we enter the phantom mode...booo... like vim... hehe ;-) @ivanov) . I mean, you can execute it, but if you want to see the border again you have to alt+enter or ctrl+enter.

—
Reply to this email directly or view it on GitHub.

damianavila · 2013-07-12T01:50:38Z

Can you give the exact keystrokes you used?

Open a new notebook
print "hi" or whatever you want...
Ctrl-m m
Ctrl-m y
Phantom mode ;-)

takluyver · 2014-01-10T00:49:56Z

I might be inclined to do that for meta-s as well, since I don't even know where my meta key is ;-). I'm happy for only one shortcut to be documented for each option.

ellisonbg · 2014-01-10T00:50:02Z

takluyver · 2014-01-10T00:54:06Z

👍 to that

ivanov · 2014-01-10T01:16:37Z

perhaps instead of h, help keyboard shortcuts key should be ? (shift + /) like in gmail?

ivanov · 2014-01-10T01:34:58Z

IPython/html/static/notebook/js/keyboardmanager.js

+            }
+        },
+        'shift+o' : {
+            help    : 'toggle output',


should read 'toggle output scrollbar' or something like that to distinguish it from 'o' above

takluyver · 2014-01-10T01:50:32Z

Split leaves the cursor in the second part of the cell, but we only have a shortcut for join below, not join above, so if you use the shortcuts to split and immediately join, you don't get back to where you started. That's probably not a big problem when you're not testing shortcuts, but it might be an awkward asymmetry to remember. I know we considered whether split should leave the cursor above or below the split - this is a point in favour of above.

ivanov · 2014-01-10T03:59:35Z

IPython/html/static/notebook/js/keyboardmanager.js

+            }
+        },
+        'up' : {
+            help    : 'select previous cell',


"select previous cell (only if on top line, otherwise, go up a line" or something like that

Too long, I think. I'd just leave the help strings out, so they're not listed in the shortcuts help - it should be obvious enough to users how moving the cursor works.

minrk · 2014-01-10T19:47:15Z

I think @ivanov's suggestion of matching Gmail's '?' shortcut for help makes the most sense now that we don't have a modifier key. But this brings up an issue: while we are using KeyDown, '?' is 'shift+/'; but on French keyboards, it is 'shift+,'. If we use the KeyPress event instead, then it's clear, and the shortcut is just '?' (no shift necessary).

However

Chrome doesn't fire KeyPress for non-printing characters (esc, arrows, etc.), so we can't use KeyPress exclusively. It seems to me that the most logical approach is:

KeyPress for regular key shortcuts
KeyDown for non-printing keys

But this is obviously more complex than just using KeyDown. Still, I think 'A' as a shortcut makes more sense than 'shift+a', and it would mean that we are not (as) sensitive to locale differences.

It's also possible that we should look into that in a separate PR, rather than here.

Carreau · 2014-01-10T20:29:41Z

Keyboard shortcut in IPython is already a mess on non-english keyboard, it is always like that in webapp anyway.

IMHO, but probably in a subsequent PR, one should be able to express in the shortcut string wether the shortcut is bounded to a key, or a char. smth like ctrl+alt+char:6 (will actually be ctrl+alt+shift+§ on fr_FR) by vs ctr+meta+key:+ omitting the key:|char: woudl have a default behavior.

Eg, french again, restart kernel should be bound to char:. (make sens for consistency with qtconsole), but headers-cells need to be bound to key:1-6 (as number require shift on fr_FR).

ellisonbg · 2014-01-10T20:52:00Z

OK, I have fixed all outstanding review comments:

Minor changes to shortcuts.
Use UI rather than UX to describe this in the example notebook.

Modal UI - a whole new world of fun....its like vim, but not!

sjobeek · 2014-01-13T09:35:59Z

I have some bug reports / feedback on the CodeMirror vim-mode integration with the modal UI. Let me know if I should split this into a separate issue.

Each code cell individually remembers the vim mode (normal or insert) that it was in when it looses focus, creating inconsistency when entering edit mode to edit a cell. Sometimes you start in command mode, sometimes straight to insert mode. Exiting a cell with esc > esc works fine, however executing a code cell (e.g. shift-enter) which was in insert-mode leaves it in insert mode. My suggestion is to have cells always enter vim normal-mode when the notebook activates a cell for "edit" mode
Some weird issues inserting text into a heading cell in vim mode. The normal-mode vim cursor doesn't render properly, and there is some strange insertion / over-writing behavior going on when attempting to modify existing text in insertion mode in a heading cell.
When editing a cell, pressing "up" while on the top line of a cell in while in vim insert mode, the focus will jump to the cell above (and probably force you to vim normal mode), while no cell jumping occurs in vim normal mode. This can strand you in an adjacent cell in an unpredictable vim-mode state. Fixing issue 1. might be enough to make this acceptable as 'normal' behavior, if it was at least predictable.
The clip-board appears to be independent of the vim buffers, meaning you can't "yank" text from vim, then ctrl-v paste it into another window. This wouldn't be too big of a problem (you could just paste vim-style), however other open windows do not seem to share the same buffers. This makes it really annoying trying to copy/paste around different windows or text editors. Not sure what is possible here (synced clipboard / vim yank buffer would be best), but the current state of yank/copy/paste with anything more than a single notebook open is pretty messy.

Other than those issues it seems to be working quite well, though! This interface is a HUGE improvement over the previous one. There's definitely some "vimception" going on with the three levels of command mode -> edit/normal > insert, but it actually works pretty well together.

ellisonbg · 2014-01-13T18:12:24Z

Thanks for the feedback. Can you open a Github issue and assign it to
ivanov for this stuff?

On Mon, Jan 13, 2014 at 1:36 AM, sjobeek notifications@github.com wrote:

I have some bug reports / feedback on the CodeMirror vim-mode integration
with the modal UI. Let me know if I should split this into a separate issue.

Each code cell individually remembers the vim mode (normal or insert),
creating inconsistency when entering edit mode to edit a cell. Sometimes
you start in command mode, sometimes straight to insert mode. Exiting a
cell with esc > esc works fine, however executing a code cell (e.g.
shift-enter) which was in insert-mode leaves it in insert mode. My
suggestion would to have cells always enter vim normal-mode when the
notebook activates a cell for "edit" mode
2.

Some weird issues inserting text into a heading cell in vim mode. The
normal-mode vim cursor doesn't render properly, and there is some strange
insertion / over-writing behavior going on when attempting to modify
existing text in insertion mode in a heading cell.
3.

When editing a cell, pressing "up" while on the top line of a cell in
while in vim insert mode, the focus will jump to the cell above (and
probably force you to vim normal mode), while no cell jumping occurs in vim
normal mode. This can strand you in an adjacent cell in an unpredictable
vim-mode state.

Other than those issues it seems to be working quite well, though! This
interface is a HUGE improvement over the previous one.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/3605#issuecomment-32154932
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

sjobeek · 2014-01-14T00:32:35Z

I've opened the issue (#4798), but unfortunately I'm not able to assign it to ivanov.

I'll add any additional feedback to that issue.

takluyver · 2014-01-14T00:37:53Z

Done the assignment to @ivanov

ivanov · 2014-01-14T00:39:32Z

looks like someone else did this for you, thanks, i'll take a
look

Modal UI - a whole new world of fun....its like vim, but not!

Carreau mentioned this pull request Jul 11, 2013

Notebook support for Emacs/vim key bindings #2319

Open

Carreau reviewed Jul 11, 2013
View reviewed changes

Carreau mentioned this pull request Jul 11, 2013

Added shorcuts to split cell, merge cell above and merge cell below. #3571

Merged

Adding sorting and better layout to the KB shortcuts.

2259a24

ivanov reviewed Jan 10, 2014
View reviewed changes

ellisonbg added 3 commits January 10, 2014 09:32

Fixing design of quickhelp.

3be8af8

Adding basic examples notebook about modal editing.

6573c62

Fixing links and text of User Experience notebook.

ab7fc46

ellisonbg added 2 commits January 10, 2014 12:38

Minor changes to KB handling.

66589e1

Work on shortcuts and examples notebook.

a6562f1

ellisonbg added a commit that referenced this pull request Jan 11, 2014

Merge pull request #3605 from ellisonbg/newux

5923544

Modal UI - a whole new world of fun....its like vim, but not!

ellisonbg merged commit 5923544 into ipython:master Jan 11, 2014

francisco-dlp mentioned this pull request Jan 14, 2014

Notebook modal UI: "merge cell below" key binding, shift+=, does not work with some keyboard layouts #4790

Closed

ellisonbg deleted the newux branch January 28, 2014 17:49

This was referenced Apr 29, 2014

2.0 Keyboard Shortcuts #5494

Closed

Modal UI: render Markdown and heading cells when leaving edit mode #5749

Closed

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#3605 from ellisonbg/newux

3e61b6a

Modal UI - a whole new world of fun....its like vim, but not!

spyder-bot mentioned this pull request Feb 17, 2015

Implement matlab-like cell features spyder-ide/spyder#852

Closed

Modal UI #3605

Modal UI #3605

Conversation

ellisonbg commented Jul 11, 2013

haard commented Jul 11, 2013

Carreau Jul 11, 2013

Choose a reason for hiding this comment

Carreau commented Jul 11, 2013

Carreau Jul 11, 2013

Choose a reason for hiding this comment

Carreau Jul 11, 2013

Choose a reason for hiding this comment

minrk Jan 9, 2014

Choose a reason for hiding this comment

Carreau commented Jul 11, 2013

Carreau commented Jul 11, 2013

Carreau commented Jul 11, 2013

Carreau commented Jul 11, 2013

Carreau commented Jul 11, 2013

Carreau commented Jul 11, 2013

jakobgager commented Jul 11, 2013

minrk commented Jul 11, 2013

minrk commented Jul 11, 2013

ivanov commented Jul 11, 2013

ptone commented Jul 11, 2013

damianavila commented Jul 12, 2013

ellisonbg commented Jul 12, 2013

damianavila commented Jul 12, 2013

takluyver commented Jan 10, 2014

ellisonbg commented Jan 10, 2014

takluyver commented Jan 10, 2014

ivanov commented Jan 10, 2014

ivanov Jan 10, 2014

Choose a reason for hiding this comment

takluyver commented Jan 10, 2014

ivanov Jan 10, 2014

Choose a reason for hiding this comment

takluyver Jan 10, 2014

Choose a reason for hiding this comment

minrk commented Jan 10, 2014

Carreau commented Jan 10, 2014

ellisonbg commented Jan 10, 2014

sjobeek commented Jan 13, 2014

ellisonbg commented Jan 13, 2014

sjobeek commented Jan 14, 2014

takluyver commented Jan 14, 2014

ivanov commented Jan 14, 2014