Feedback on beforeinput (spec and the current implementation in Chrome Canary)

[Reinmar]

Hi all!

Unfortunately, neither I nor @fredck will be able to join you in Lisboa, so I wanted to give you some feedback on the current shape of the spec and the implementation of beforeinput which is available in the latest Chrome Canary (remember about enabling chrome://flags/#enable-experimental-web-platform-feature).

In order to better understand how it suits us, I decided to make a proof of concept of CKEditor 5's typing feature using the beforeinput event.

(At the end of this post I wrote a few words about how CKEditor 5 works and about the state of the project. This information may help to understand the demo that I created.)

Conclusions on beforeinput

I've been writing down conclusions while working on the prototype. I tried to structure them as much as possible, but they are still a bit random. Sorry for that :).

Useful samples:

• Simple test page which logs the event and the current selection: https://jsfiddle.net/gqnq4h5r/8/
• Like above, but logs the target ranges: https://jsfiddle.net/gqnq4h5r/17/
• Preventing the default action: https://jsfiddle.net/gqnq4h5r/10/

Note: When I've been logging range.startContainer to the console, it seemed that beforeinput is fired when the DOM has already been modified. However, it was only a problem with Chrome's console which apparently reads the values with a slight delay.

What works great

• The event works great for normal typing, deleting, etc. The target ranges available in the delete operations allowed me to quickly implement things like deleting a whole word. I wasn't able to test deleting the whole line cause I couldn't find a keystroke for that in MacOS :|.
• The event is well integrated with MacOS's IME balloon (for typing variations of a letter) and normal IME.
• Pressing space multiple times in a row generates beforeinput events with a data==space. The JS engine must then work on converting them to a proper mix of   and normal spaces (e.g. to display them at block boundaries). Pressing Alt+Space generates beforeinput with a data== . This is great.
• Integration with Spanish writing accents (Spanish-ISO keyboard) seems to be fine.
• Preventing default actions – I can prevent everything except IME operations (and spell checker, but as I wrote below, I assume that support for spell checker isn't yet implemented). This is also super cool.

And many more things which simply didn't catch my attention cause they worked as expected.

Problems

These are the things which I identified as possible problems, either with the spec or the current state of implementation in Chrome.

• Spell checker triggers only the input event. The evt.data is empty and evt.inputType equals insertFromPaste. This, of course, makes it impossible to handle it in JS, but I guess that support for spell checker is simply not yet implemented.

• Dragging text fires only the input event. At the same time, cutting and pasting trigger their respective beforeinput events. I can see that according to the spec, a deleteByDrag + insertFromDrop should be fired. So I assume that this isn't yet implemented.

• I assume that deleteComposedCharacterForward/Backward aren't implemented yet, cause I couldn't trigger those events.

• getTargetRanges() returns an empty array when typing a letter into a non-collapsed selection. I can take the affected range from the selection, but I found a spec a bit unclear, cause it says that getTargetRanges() returns an "array of StaticRanges affected by this event". Also, when doing a simple composition of an "á" character (on the Spanish-ISO keyboard) I must use the selection to understand which existing character (you type the accent first) should be replaced by the composed character (when you add "a"). I'd expect to use getTargetRanges() for that instead. It would be more consistent with the delete* events.

• Undo/redo have their respective input types. In general, firing events for undo/redo is great, cause it could allow us to integrate with the native controls (context menu, the "Edit" menu or shaking on iOS, etc.), but unfortunately the event isn't fired when we're at the end of the undo stack (e.g. there's noting to undo according to the native undo manager). I know there was some work on exposing the native undo manager, but I don't know how it looks now. Every JS editor has its own undo manager anyway, so if those events will work like they do now, they will be pretty much useless for us.

  Proposal – the events should be always fired, even if we're at the end of the undo stack.

  I've seen July 29th F2F Agenda Item - History handling (undo/redo) #136 but it doesn't clarify anything.

IME

IME deserves its own section :D.

As for IME, I know there were a lot of discussions how to handle it. For me, it looks pretty good how it works now. We get the beforeinput events, based on which we can update the model (editor's internal data model) and broadcast the changes to the other collaborating clients. We can also post-fix some details after composition has ended. The only really tricky thing is how to show the changes from other users for a user who's currently composing.

To understand that, I've checked how stable IME is when composition takes place (in one of the text nodes):

• Touching a different text node is not a problem: https://jsfiddle.net/gqnq4h5r/11/ (start composing text in "Apple" and the composition won't be disturbed).
• Touching the same text node (by using methods like CharacterData.replaceContent()) doesn't break the composition too: https://jsfiddle.net/gqnq4h5r/12/ (you can place caret inside "App*" and use IME there). This is super cool :).
• Moving the text node between elements: https://jsfiddle.net/gqnq4h5r/13/ – this doesn't work at all because when the text node is moved to a different element, the selection is lost.

If browsers could handle the last case (preserve the selection and ensure continuing composition) and be consistent with the other cases, then I'd consider this case solved.

Summing up:

• beforeinput lets us update the internal data model,
• we can fix things after composition has ended, so the fact that we cannot block it (prevent the default action) is fine,
• we only miss the option to modify the DOM around the composition (like moving the text node in which it is anchored between elements).

PS. I've spotted one inconsistency. With a Spanish-ISO keyboard, when the default action is prevented, I can type "´" which starts the composition, but when I press "a", composition ends and nothing else happens (normally, the expected result would be the "á" character). This works different when entering Hiragana characters, because the whole composition is committed, despite blocking beforeinput.

Surprises

I wasn't able to follow all the discussions and I've been reading the spec while testing the behaviour on Chrome, so there were couple of things which surprised me.

• Open https://jsfiddle.net/gqnq4h5r/8/ and cut the word "Apple". The selection contains only this word on beforeinput, but Chrome removes also the space after it. This may be surprising, but should be fine as it's simply how the native implementation acts when cutting a word (JS editors can mimic it). But I wonder if e.g. target range should not reflect this (you can check on https://jsfiddle.net/gqnq4h5r/17/ that there are no target ranges).

  BTW, An interesting thing happened when I pasted this word back in the middle of another word. It inserted  Apple .

• When pressing the Enter key in the middle of a heading element, a beforeinput event with type==insertParagraph is fired. That's a bit unfortunate name, since often such input will be splitting different kind of blocks, inserting new list items or even outdenting lists. Therefore, in CKEditor, we decided to call this action "enter" as none other name suited it.

• I've been surprised to see clipboard events duplicated in the spec. The ones given by the Clipboard API (copy, cut, paste, drag*) seems to be sufficient.

• In general, I think that the meaning of "target ranges" must be explained clearly in all the cases where they will be provided. In other words – what's the difference between selection and target ranges.

Prototype of the CKEditor 5 typing feature

Demo: http://ckeditor.github.io/ckeditor5-design/poc-typing-beforeinput/

About CKEditor 5

(Note: I don't know if this will be useful for anyone except us, but have to write the summary down anyway, so here it goes...)

Before I start, just a quick note about CKEditor 5. It's under development and hence the demo I'll show you later is not fully functional and is buggy.

The CKEditor 5 engine features a custom data model with support for operational transformations (needed for collaboration). In order to change the DOM, you must create an operation on the model, which is then converted to a virtual DOM (called "the view") and rendered to the real DOM (only if needed).

This data flow allowed us to handle user input in two ways:

• If we can intercept some action (e.g. pressing Enter, Backspace, pasting, etc.), we prevent the default action and apply the operations on the model. The change gets rendered immediately.
• If we cannot intercept some action (typing, IME, spell checker, native autocompleting, etc.) we use DOM mutation observers. Based on heuristics we try to discover different type of input and convert them to operations on the model.

As I wrote above, we do not change the real DOM if we don't have to. This means that using this architecture we can handle e.g. IME by not rendering for a while (or, what's the current but imperfect approach, by assuming that operations on the model will generate exactly output as what user did in the DOM).

Unfortunately, all this is super tricky. This is how the delete feature looks. And this is how the input (typing) feature looks. The latter is obviously a mess comparing to Delete handling. (Note: both features are incomplete – e.g. we don't support yet deleting whole words).

And, now let's compare this with the PoC using the beforeinput event:

• https://github.com/ckeditor/ckeditor5-typing/blob/t/46/src/inputobserver.js – transforms beforeinput with type insertText into our custom input event.
  Note the issue with lack of target ranges. I use the selection (instead of the missing target ranges) later on here to replace the modified piece (e.g. during various compositions).
• https://github.com/ckeditor/ckeditor5-typing/blob/45b47a3cc68e36932592ebcc151b12ea79f149d5/src/input.js – that's the input feature now. It boils down to listening on the input, deleting what's in the selection and inserting new text. The ugly heuristic is gone!
• https://github.com/ckeditor/ckeditor5-typing/blob/t/46/src/deleteobserver.js and https://github.com/ckeditor/ckeditor5-typing/blob/t/46/src/delete.js – and this is the delete feature now. More complicated than before, but handles all kinds of delete (word, line, etc.). Most of the code converts native event into our custom event.

Summary

It's been really cool to see the beforeinput event in action and I can already tell that it'll simplify a lot of things for us. There are some missing pieces though (e.g. support for spell checker) and we still haven't found an ultimate solution for IME (but I feel that we're close).

Thanks for the Chrome team for implementing the event!

[johanneswilm]

Thanks a lot! This is all a lot of important information. I think IME and spellcheck we just figured out (IME in Japanese and on Android have some extra requirements that don't apply to all other IMEs). Some of the other issues seem to be bugs in theChrome implementation. Firing undo even though the browser's history stack may be something we still need to clarigy in the spec.
Again, thanks a lot!

[chong-z]

Thanks for the amazing demo! That really help a lot!

Just a quick response, yes most of the issues are bugs/TODOs:

• getTargetRanges() is intentional but might require more discussion
• IME is close (but I'm not sure about moving elements though)
• Enter key requires more discussion
• For clipboard yes you can just use Clipboard API, but there is a little bit behavior difference, and I assume we want 'beforeinput' to catch all DOM modifications (preventing all 'beforeinput' would be similar to cE=caret)
• undo/redo is useless as I can see currently, and we have to find a different solution

I'll fire separate issues for them, and probably add more descriptions to the spec.