caret-based selection movement

[johanneswilm]

The selection Api so far doesn't talk about changing the selection when moving the caret.

This will only apply when cE=typing and cE=true are specified, so it may have to go into another document.

If it's not specified anywhere, and browsers choose to implement it different, we won't be sure so we will need to continue to interrupt each key stroke event to check whether the caret may be at the border of an element before letting it do its default action. Then it may be easier to use a hidden textarea for input instead of cE=typing.

Two issues come to mind:

1. Caret at the end of an inline element

   <p>Hello <i>Austri|a</i></p> ( | = caret)
   

   If the user hits the right-key and then enters a "!", where should that end?

   <p>Hello <i>Austria</i>!|</p> or  <p>Hello <i>Austria!|</i></p> ?
   

   And what is the intermediate step? To the user, these two situations will look the same:

   <p>Hello <i>Austria</i>|</p> and <p>Hello <i>Austria|</i></p>
   

   so it seems it would be reasonable that they will behave the same as well.

2. It should be ensured that the movement of the caret is the same in all browsers.

   There are some well-known issues with that now, mostly related to the caret "jumping" another character when moving across inline void elements, foreign elements (SVGs) or elements that are specified as being noneditable or the caret not being able to go certain places (such as before one of the before mentioned elements when these are at the start of a block element). In all these cases, I believe it should be specified that these elements should be treated like single characters in relation to caret movement.

[teleclimber]

And what is the intermediate step? To the user, these two situations will look the same:

<p>Hello <i>Austria</i>|</p> and <p>Hello <i>Austria|</i></p>

so it seems it would be reasonable that they will behave the same as well.

I am not sure each of these should behave the same. In the first, the subsequently typed text should not be italic, in the second it should be italic (because it is inside the <i>.)

In practice they won't look the same either because the editor UI usually provides feedback on the style about to be used by showing the style buttons as pressed (the [i] button would be "pushed/on" in the second case.)

If the user hits the right-key and then enters a "!", where should that end?

<p>Hello <i>Austria</i>!|</p> or  <p>Hello <i>Austria!|</i></p> ?

I think the default behavior should mimic existing paradigms for editing UIs, so I played with a couple of existing "standard" rich text editors to see how they behaved (Mac's textEdit and Google "Docs".) The result I have so far is as follows. (assume we have entered 'Hello Austria!', and the country name is italic)

• place caret to the left of 'A' in Austria, and type. result: text typed is not italic.
• place caret to the right of 'a' in Austria (end of word). result: text typed is italic.

With this we can say that the default position of the caret when the user places it at the beginning of an element like <i> is that it is outside that element. When placed at the end of the element, it is inside the element. See the two pipes in the markup below:

<p>Hello |<i>Austria|</i></p>

This "default" position for caret should be in effect when user presses arrow keys to move caret around and when the user clicks the mouse to place the cursor at the edge of an element.

By the way it turns out this is already the default behavior on the browsers I tested (Safari, Chrome, FF, IE11).

[johanneswilm]

You are right, the [I]-button would probably be pushed. More importantly though, the user cannot switch between the two places of the caret. The browser will decide where it goes. Defining the default position as you do seems to make the issue go away.

However: notice that while this works for i-elements, it's quite different for links, where the default position seem to be:

<p>Hello |<a href="http://www.austria.com">Austria</a>|</p> 

This difference in behavior should be spec'ed and it should be made clear when which of the two applies.

[teleclimber]

I just fired up my old MS Word 2000 and found that they handle links the same way you found in browsers (outside by default).

Google docs handles links in the same manner as <i>: outside~inside.

I can't think of a reason why there should be two different behaviors? Good user interfaces tend to be consistent. My inclination is to specify that the default cursor position is the same regardless of the element.

[johanneswilm]

Really? That's weird. I tested Google Docs and Gmail using Chrome 39 on Linux, and in both cases it exited the link when typing at the end of it. Same applies to LibreOffice 4.3.3.2.

[johanneswilm]

I think the behavior should be both consistent across browsers and "reasonable" in the sense that it would correspond to a normal setup for an editor. If one of those two conditions is not given, all editors will constantly have to check where the caret is in relation to its surroundings before executing a character insertion.

As for why links are treated differently: Possibly because the makers of browsers thought it easier to exit <i> elements than <a> elements with most editor UIs?

Take the below test. In Chrome the behavior at the end of the three lines is:

1. <i> outside-inside. However, the user can exit the <i> by hitting CTRL+I.
2. <a> outside-outside.
3. <span class="custom"> outside-inside.

In Firefox the behavior is however;

1. <i> outside-inside.
2. <a> outside-inside.
3. <span class="custom"> outside-inside.

Source:

<html>
<head>
    <style>
        .custom {
            color: green;
        }
    </style>
</head>
<body>
<div contenteditable=true>
<p>Hello <i>Austria</i></p>

<p>Hello <a href="http://www.austria.at">Austria</a></p>

<p>Hello <span class="custom" data-custom="something important">Austria</span></p>
</div>
</body>
</html>

[teleclimber]

Actually I take that back. In Google Docs it's outside ~ outside as you've noticed. However, when the cursor is at the end of the link, the U button is depressed and the text-color button indicates the color is blue giving the impression that it's outside-inside. However, once you start typing that all goes away and you're inserting unstyled text.

I agree that it's probably because it's easier to cancel a '' than an '' that the behavior is different. I really don't love that there are two behaviors, but it seems that if most existing rich editors do it this way it would be sensible to specify:

• outside ~ inside for all inline elements
• except for '' elements where it's outside ~ outside .

Are there other elements/situations to consider?

One caveat is that this should only apply to inline elements. For block elements we probably need to think about this some more.

Another aspect to look into which is very important is the relationship between the data provided by the getSelection API versus where text actually ends up going upon typing. To me, for the sake of developer experience there is no question it should 1-to-1: text gets inserted wherever the selection API says the caret is. For that to work the selection API has to be able to convey that info accurately under all circumstances. I'm not 100% sure it can do that yet. Need to look into it.

[johanneswilm]

Are there other elements/situations to consider?

Not that I can think of. But it doesn't seem very HTML5-like that something is hardcoded to only work on <a> that is unavailable to any span-class combination the site developer may think of themselves. Just thinking allowed: Maybe "outside ~ inside" vs. "outside ~ outside" could be controlled via CSS?

[teleclimber]

I agree. It seems arbitrary. And if it can't be controlled on a per-element basis it's even worse. CSS could maybe be used to help with this.

Maybe we should explore other areas of this issue (caret movement around block elements, void elements) as that might make the right solution come to light.

[johanneswilm]

Ok, true, maybe that leads us somewhere. Check out this example: https://bug873883.bugzilla.mozilla.org/attachment.cgi?id=751510 . All of the elements there should be handled as single characters: SVGs, void elements such as canvas elements, a noneditable element. The possible exception:
<div contenteditable=true>Hello |<span contenteditable=false>Aus<span contenteditable=true>tr</span>ia</span>!</div>

What should happen when hitting the right arrow?

<div contenteditable=true>Hello <span contenteditable=false>Aus<span contenteditable=true>tr</span>ia</span>|!</div>

or

<div contenteditable=true>Hello <span contenteditable=false>Aus<span contenteditable=true>|tr</span>ia</span>!</div>

?

[teleclimber]

I agree it seems we want to give users the ability to insert text immediately on either side of anything that is in their content but they can't edit.

What if we specified the general rule that any visible but non-editable part of the document should be treated in its entirety as a single character as far as caret movement is concerned?

A visible but non-editable part of the document could be: an SVG, IMAGE, VIDEO, whatever. Also, it implies the visible characters (and any other dom element) inside a cE=false.

Given your first example the result would be this:

<div contenteditable=true>Hello <span contenteditable=false>Aus<span contenteditable=true>|tr</span>ia</span>!</div>

Also, by defining behavior in terms of "visible content" we handle some odd edge cases in a manner that I think is best for the user:

• invisible elements (display:none) are "transparent" to the caret
• cE=false elements or SVGs that are actually empty are also "transparent" to the caret.

Do you think these make sense as default behaviors?

[johanneswilm]

With "Also, it implies the visible characters (and any other dom element) inside a cE=false." you mean that the caret jumps inside of any cE=true should these exist within a cE=false? Your example result makes me think that.

Could you give an example in which a cE=false should be treated "transparently"? With "transparent" you mean that the caret jumps across it together with either the character before or after it, right? But which one of those is ti going to be? And how do you insert text just before AND just after such an invisible element?

[johanneswilm]

I wonder whether it would be easier to stay consistent if empty SVGs and empty cE=false as well as invisible elements were treated the same as single characters as it's hard to know when they are really empty. But I am not a browser developer. Maybe it's easy for them to distinguish.

Imagine for example this case (using webkit css version of cE):

<html>
<head>
<style>
.editor {
-webkit-user-modify: read-write;
}
.interrupter {
-webkit-user-modify: read-only;
display: inline-block;
width: 10px;
height: 10px;
background-color: blue;
}
</style>
</head>
<body>
<div class="editor">
Some text <span class="interrupter"></span>.
</div>.
</body>
</html>

[teleclimber]

Yes, I was going on the assumption that we would want the caret to jump inside a cE=true that is nested inside a cE=false should there be such a situation. It seems if there is an editable area it should be reachable via arrow keys? I'm open on this though.

Here's an example of what I meant with "cE=false transparent":

abc|<span contentEditable=false></span>def  
abc<span contentEditable=false></span>d|ef  //after right-arrow

Whether the caret jumps the cE=false with the character before or after it is a matter of picking one and spec'ing that.

And how do you insert text just before AND just after such an invisible element?

If the element is empty or is not visible (display:none) then the user has no way of knowing which side of the element the caret is on anyways. It will seem like pressing the arrow key has no effect (that caret won't physically move between the before and after position).

I wonder whether it would be easier to stay consistent if empty SVGs and empty cE=false as well as invisible elements were treated the same as single characters as it's hard to know when they are really empty. But I am not a browser developer. Maybe it's easy for them to distinguish.

I agree that UAs will have a hard time really knowing whether an SVG or other element is really empty. Also, empty elements can still have visible borders, and other things that give them a visible presence. So I think you're right it's better to be consistent and treat SVGs the same whether they are empty or not.

However, I'm still a bit torn on elements that are display:none. Invisible elements are completely removed from the document layout, and have no possible visible manifestation. The caret would just be "stuck" for a moment with no obvious reason why. Further, the UAs already track that property closely for their layouts, so there is little reason to think this would cause them a problem.

[johanneswilm]

Hey, I agree on your first point -- one should jump in to the cE=true when moving carets. If we were talking delete/backspace into it, I would however disagree and think it's poorly handled in Webkit/Blink currently, but luckily we will not have to worry about that any more if we don't handle backspace/delete.

On your last point: I agree it would be confusing if the caret doesn't move because there is a display:none element there. But it would also be problematic if it would be completely impossible to move the caret either directly before or after it (at least programmaticly). Maybe we should start out by looking at possible use cases for this? The one example I can think of are when using a system to "track changes" but currently hiding them: http://nytimes.github.io/ice/demo/ (click on "Show/Hide changes"). In that case it would indeed make sense to do as you suggest. However, I would probably generally think this is quite advanced usage and possibly should be handled by cE=typing at all.

[teleclimber]

But it would also be problematic if it would be completely impossible to move the caret either directly before or after it (at least programmatically).

The way I see it we are just trying to sort out default behaviors (what the UA does if the user's Intent is allowed to continue without intervention by the editor). I think that editor dev would be able to programmatically set the caret on either side of the display:none element if necessary.

Edit: there is also the possibility (although a bit more far fetched) that this behavior would be customziable via CSS: "caret-outside: both | left | right;" However I am not sure we should tackle this in V1. It's bound to be full of edge cases...

Maybe we should start out by looking at possible use cases for this?

Sure! May I turn this around on you? What use case is there where an invisible display:none element should be treated as one character?

[johanneswilm]

I can't think of a good example. :) Ok, let's at least at least make sure to define the behavior exactly it. Should we say that it should handle the hidden element together with the character that follows it? Or are there good reasons not to go with that?

[teleclimber]

I think browsers currently tend to move the caret to the end of text nodes then to offset 1 of the next text node. Seeing as it's kind of arbitrary anyways let's specify this?

abc|<span style="display:none">invisible</span>def
abc<span style="display:none">invisible</span>d|ef   //after right arrow

[BenjamP]

Caret movement ideas from Olivier http://lists.w3.org/Archives/Public/public-editing-tf/2014Dec/0031.html):
Entry via click:

1. User clicks
2. UA resolves document position. It may be that the user clicked on an image, in which case it's not editable. If clicking in an editable area there may be some ambiguities where there are 1:visible:n:document positions. The spec clarifies the proper document position to use, and the UA acts accordingly.
3. if position is editable, set the selection and go to 4 below

Entry via arrow keys:

1. User presses arrow key (note that for arrow keys to effect caret movement, the caret must already be visible, ie: the selection must already be editable)
2. A new editable position is calculated. There could be ambiguities regarding the correct position. The spec should clarify proper caret movement.
3. By definition the selection is editable, set selection, go to 4.

Entry via editor's setSelection:

1. editor calls setSelection
2. if editable go to 4. UA should NOT resolve ambiguities.

Entry via external factors: If the editable position is removed from DOM or made non-editable, what do we do?

1. current editable selection made invalid
2. UA needs to unset selection or set it to something new //TODO
3. editor should have a way to know this happened. Does selection api include selection events?

Once an editable selection is set:
4. selection is editable, so show a caret at the right place
5. getSelection returns the selection
6. typing inserts text at position indicated by the selection.

[rniwa]

Spec'ing exactly how caret is moved upon arrow keys is hard because different browsers behave differently with respect to bidirectional text. Some would move the caret logically forward & backward whereas some would move the caret visually left or right.

However, we can probably specify what kind of DOM positions should be used (e.g. inner most element versus outermost) when UAs DO move the caret in response to user actions.

[BenjamP]

Here is what I'm hearing so far:

1. The location of the caret should be deterministic
2. For inline elements, it should be what we're calling "outside ~ inside", meaning it should have left-gravity, except for special cases.
3. BiDi behavior for arrow keys should not be spec'd because it's platform dependent.
4. We want to treat non-editable areas (ce=false, img, canvas, svg) as single characters.
5. We want to treat hidden elements as if they're not there at all. There is some concern about how to allow placing the caret on either side of such elements.

I agree with most of this. I think 3 doesn't really matter because we want to spec the default behavior of Responsive Input Events, not keyboard events. We should have a good way to indicate the platform agnostic meaning of the relevant beforeSelectionChange events, which is an open question I suppose.

I don't think 2 should have special cases. They can be handled in frameworks if they really matter.

[BenjamP]

Here is a possible algorithm for move caret. It relies on carets only being allowed in visible text nodes. Note that empty text nodes are fine, they just can't be invisible.

1. For directional caret movement (arrow key, etc), determine the next position in the given direction in the current text node if it exists. If it doesn't exist, use the closest position in the closest text node in the given direction. Call this position "proposedPosition".
2. For non-directional caret movement (click, etc), determine the closest position in the closest visible text node to the location of the click with left-gravity, on the closest line to the location of the click. Call this position "proposedPosition".
3. Fire beforeSelectionChange, indicating prosposedPosition and the source event (click, keyboard, etc) in the beforeSelectionChange event. If script does not like the proposedLocation, it can create a text node and place the selection where it wants, and call preventDefault.
4. If preventDefault is not called, move the caret to proposedLocation.

For <p>Hello <i>Austri|a</i></p>, this means moving right would still be italic. Moving left past the "A" would still be italic. Moving left again to the 'o' and moving right from there would not be italic. Clicking left of the 'A' would not be italic and clicking right of the 'a' would be italic (left-gravity).

For abc|<span style="display:none">invisible</span>def, moving right would be after the d, moving left from there would be at abc<span style="display:none">invisible</span>|def

[NorbertLindenberg]

This discussion so far seems to assume that the selection is empty before and after any change. Some things change if we consider non-empty selections and the ability to extend and shrink selections:
– While empty selections shouldn't be allowed in non-editable content, non-empty selections have to be allowed everywhere so that users can copy content or look up word definitions.
– When extending selections, the new selection might include both editable and non-editable content. Modifying operations on such a selection wouldn't be allowed, but copying or other read-only operations should be possible.
– Shrinking a non-empty selection in non-editable content to the point where it becomes empty should make it go away entirely.

[NorbertLindenberg]

Re "left-gravity": Please use "left" and "right" only if you're sure they're the right terms, either because you know the current writing direction or because the present use is independent of the writing direction. I think in this case you mean logical "before" or "after".

[NorbertLindenberg]

Re "determine the next position in the given direction in the current text node if it exists":
– How closely can and should "next position" be specified? Is there any agreement between current implementations what the increment, say, in Indic text (स्वागतम्) or in a sequence of regional indicators (🇩🇪🇺🇸🇯🇵) should be?
– This should mention that the "direction" is not necessarily the one that's printed on the arrow key being pressed, but may depend on writing direction and writing mode in a platform-dependent way.

[rniwa]

Right, let's not conflate left/right and next/previous. Left and right are visual directions while next and previous are logical directions. Left and right arrow keys may, for example, move caret between lines in vertical writing mode.

Since binding a particular key press to a semantic event is precisely what Responsive Input Events are for, we shouldn't talk in terms of key press here. We should only talk in terms of semantic events such as moving forward between characters and lines.

Like I mentioned previously, the challenge here is that some caret movements such as of a word boundary depends on implementation-dependent heuristics so we can't precisely define how it works. So we should focus on how normalization works after caret move instead of trying to define what moving to left/right or forwards/backwards do.

[johanneswilm]

@rniwa : Right. So if I understand you right, you are saying that the way Firefox handles caret movement, where 1 right key + 1 left key does not result in the caret being where it initially was necessarily, is not something that could be spec'ed?
How about point two in the post? Can it be spec'ed that non-editable elements are treated like a single character when moving across them (using arrow keys or otherwise)?
As for the moving between words (on Windows/Linux with CTRL + arrow keys, on Mac with Option-key instead of CTRL) -- why can this not be spec'ed? I guess it's not so important to define whether or not commas and periods should be included in words, but what's the issue?

[rniwa]

Right, we can spec that non-editable elements are treated like a single character.

The problem with word boundary movements is that the treatment of whitespace is different amongst browsers. e.g. trailing or leading whitespace may or may not be considered as a part of a word. Also, in Chinese and Japanese, word boundaries can only be determined by heuristics. See http://www.quora.com/What-are-some-Japanese-tokenizers-or-tokenization-strategies

[johanneswilm]

Ok, but this difference in how browsers behave is exactly what we want to stop by spec'ing things, right? The idea is to get behavior which editor developers (and end users) can rely on. If the caret moves an extra space in certain browsers while not in others, that can be the cause of a major headache for editor developers, who may find it easier to just override the word boundary movement mechanism altogether in order to ensure consistent behavior.

As for Japanese, Chinese, etc.: Could we spec word boundary movements for all text using the Roman or Greek alphabets?

[rniwa]

The problem is that the difference in how whitespaces are treated come from the difference in the underlying platform convention: Mac and Windows.

And I don't feel compelled to spec word boundary movements just for a subset of languages.

[johanneswilm]

Well, but you do need to define word boundary movement for example for Japanese then, right? That seems like a much smaller subset of languages than all languages using Greek/Latin alphabets. Word boundary movements may not be the most important thing to spec in the world, but that there are languages that have specific rules does not seem like a good reason not to do so.

[rniwa]

No, the point is that we can't define word boundary movement for Japanese because word boundaries can only be guessed via heuristics such as dictionary-based lookups in Japanese. And most of European languages use a space as a word delimiter so we can't define the behavior there either due to the treatment of whitespace being platform-dependent.

[johanneswilm]

Yes, so the spec for Japanese would include a description a la "space based word boundary movement does not apply. word boundary movement does either not take place at all or follows a dictionary based pattern. If we were talking about a special way the caret has to move around the Danish/Norwegian æ-character, which doesn't apply to any other language, I would agree that maybe it would be overkill to spec that. But for anything related to 8 of the top 10 languages on the internet (Roman/Russian/Greek alphabet based) it seems like this should be something we should be concerned with.

Now the issue with different behavior in Mac OS X and Windows may be more complicated and we should find out what to do about that.

[rniwa]

I don't think we can spec. word boundary movement. Also, various browsers support other granularities of caret movements. For example, Safari supports moving the caret by sentences and paragraphs. And I don't think we want to be in the business of defining what a sentence or a paragraph of text means. In general, any attempt to define how selection gets updated in response to a user action won't be fruitful.

The thing we can and should spec here is the normalization of Range browsers would apply after the user had changed the selection.

[johanneswilm]

Why would we not specify hat a sentence and what a paragraph are, if these are important for caret movement? Just because we define it, doesn't mean that all browsers are obliged to implement sentence/paragraph movement as a feature. It's just that once a specific browser decides to implement one of these features, they do so in a predictable manner.

The current situation, in which each browsers do a lot of things differently and even differ in their behavior from platform to platform and between browser versions is the main issue we are tying to deal with, as I understand it.

Currently many editors need to interrupt caret movements in Javascript and after a number of checks decide whether or not to let the default action go through. This is quite a lot of work just or right/left arrows. If additionally each browser does its own thing, which is unspec'ed (and likely not well documented), it becomes exponentially more difficult to get it right.

That being said, I agree that the normalization of the caret placement seems to be the most important thing here.

[rniwa]

I don't think a Web developer should have any expectation as to how a caret moves in response to a given key down/press. It would be entirely platform dependent. Ben's Responsive Input Events should solve the use case you're talking about by providing a new selection/range before the selection is actually changed. A Web site/app can inspect and adjust this new range before it is actually set.

[kojiishi]

I agree with @rniwa for not to define caret movements in the spec, especially word/sentence boundaries. Instead, beforeSelectionChange event in Ben's proposal could carry the original intent that caused the selection change. If a framework would like to ignore platform-standard behavior but its own standard behavior, it can trap such events and adjust selections as needed. I suppose framework is easier if the original intent is known, no?

Are we also going to define:

1. What are the conditions the "proposed positions" has to meet?
2. What would browsers do if APIs set selections to non-editable, non-selectable, atomic (e.g., img) etc.?

[NorbertLindenberg]

That there are differences between browsers or operating systems doesn't necessarily preclude a spec. For space handling, we could specify two alternatives, and say that a browser can choose which one to use, but that it has to stick with it at least for the lifetime of the document, and that it cannot invent something different. For word boundaries, we can say that the determination of the word itself is implementation dependent, but then specify the behavior relying on the word boundaries.

You can see this in the specification for NumberFormat.prototype.format in the ECMAScript Internationalization API: The spec is very precise as to how to map a number to a string of digits, but then leaves several aspects of the localization of this string implementation dependent, such as which digits to use for which locale, or which decimal and grouping separators to use or where to insert the grouping separators. The idea is that the numeric value should be recognizable in any localized format.
http://www.ecma-international.org/ecma-402/1.0/#sec-11.3.2

The question though is what level if precision would help application developers.

[johanneswilm]

This is no longer relevant with cE=intentions.

[rniwa]

Reopening the issue since this is still an issue outside of contenteditable=events.