specify how compositionend works if the caret has been moved to a different element

[johanneswilm]

If one moves the caret/selection to a different element when compoisitionstart is triggered, browsers different on compositionend:

1. Chrome/Safari trigger the compositionend event on the element where composition takes place.
2. Firefox triggers the compositionend event on the element where the caret was at the time when the compositionstart event was triggered.

Additionally, browsers behave differently if the caret is moved to a different element when compositionend is triggered:

1. Chrome/Safari paste the composed characters in the place where the caret is moved to.
2. Firefox moves the caret to a different element and inserts no characters there by default. If the JS inserts the characters by means of execCommand/insertHTML, the endComposition event is triggered a second time.

The Chrome/Safari behavior is desired for the use case where one wants to have IME character insertion into the document stream be atomic -- either by doing composition somewhere else [1] or by using a Shadow DOM during composition [2].

[1] http://jsbin.com/wimuqowimu/15/edit?html,js,console,output (works in Chrome/FF)
[2] http://jsbin.com/pacisugiwu/1/edit?html,js,output (only works in Chrome)

Edge behavior is close to FF behavior and Safari close to Chrome.

See also debate about this here: https://lists.w3.org/Archives/Public/www-dom/2015JulSep/0044.html

[kojiishi]

What use cases do you have in your mind? From the normal use cases, Chrome/Safari behavior is considered as a bug. Is it happening only when set from JS?

Consider re-conversion, that turns existing text into composition. If the composition is inserted into DOM, and if the selection was moved, the original text must be restored. In that case, moving composition text into new caret is undesirable.

[johanneswilm]

@kojiishi

What use cases do you have in your mind?

At a low level basically what you had proposed to do inside the browser with IME using the Shadow DOM to do IME input without changing the DOM. Only that @ojanvafai was strongly against that idea, so we found a solution whereby it should instead be possible to do things that way in JS code only [1].

At a higher level, the idea of having IME character commits come in "atomic commits" into the DOM. This would be helpful for among other things:

• Collaborative editing environments
• Creating a clientside undo history

From the normal use cases, Chrome/Safari behavior is considered as a bug

What do you mean by "From the normal use cases"?

The part about the Chrome/Safari behavior that I cannot see anything about in the spec is that it pastes the created characters into the editing host where the caret is placed when the compositionend event is triggered. The rest of it is according to spec, as far as I can tell.

Firefox, on the other hand, does not follow the spec in several ways, AFAICT. First of all, it should not trigger the event twice. Secondly, the spec reads "Event.target: focused element processing the composition". That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started. Unless JS moves the caret somewhere else, there will be no difference between where the caret was before composition started and during composition started, which is likely why they didn't think of this case when they wrote the code.

Is it happening only when set from JS?

I am not sure what you mean. How would you set the caret anywhere else when compositionstart is triggered without using JS?

Consider re-conversion, that turns existing text into composition.

This is something you only do in Android and that is not used anywhere else, right?

If the composition is inserted into DOM, and if the selection was moved, the original text must be restored. In that case, moving composition text into new caret is undesirable.

I don't think I understand what you are trying to say here.

In general, I don't think that this re-conversion should be a problem for the JS solution [1] possibly with some minor modifications. Could you outline what the process is exactly for re-conversion, or do you have a link?

[1] http://jsbin.com/pacisugiwu/1/edit?html,js,output

[masayuki-nakano]

Secondly, the spec reads "Event.target: focused element processing the composition".

It should be a bug of UI Events.

That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started.

In these days, modern IMEs queries content information before compositionstart. Therefore, moving focus at compositionstart is too late for native IMEs.

Firefox guarantees that a set of composition events must be fired on an element. It should be better for web developers because compositionend events may not be fired if focus is lost from listening event target unexpectedly.

[masayuki-nakano]

Firefox moves the caret to a different element and inserts no characters there by default. If the JS inserts the characters by means of execCommand/insertHTML, the endComposition event is triggered a second time.

I think that if the focused element was <input> or <textarea>, the composition string is committed on the element at moving focus?

IIRC, if it's a contenteditable element, it doesn't work fine due to our editor's bug (and it's very difficult to fix in current design...)

[johanneswilm]

Secondly, the spec reads "Event.target: focused element processing the composition".

It should be a bug of UI Events.

What do you mean? Do you mean that the UI Events spec should be changed? Or that Firefox has a bug?

That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started.

In these days, modern IMEs queries content information before compositionstart. Therefore, moving focus at compositionstart is too late for native IMEs.

What are modern IMEs for you and what are native IMEs? I tried this on Chrome on Mac OS X with one of the Mac OS X built-in IMEs, and the Chrome example still works, which means that it's no problem to move the focus somewhere else during compositionstart for Chrome on Mac OS X. Also Safari is able to move the focus during compositionstart.

If there are UAs which start composition before compositionstart and therefore cannot move the selection somewhere else, I would say that those UAs have a bug and should be fixed.

Firefox moves the caret to a different element and inserts no characters there by default. If the JS inserts the characters by means of execCommand/insertHTML, the endComposition event is triggered a second time.
I think that if the focused element was or <textarea>, the composition string is committed on the element at moving focus?

IIRC, if it's a contenteditable element, it doesn't work fine due to our editor's bug (and it's very difficult to fix in current design...)

Could you guys look at the two pieces of example code I put together? :)

And what is "our editor's bug"?

and <textarea> do not matter much to us right now. In the Editing taskforce we are trying to fix editing in the browser, and having this work correctly is a key component.

[masayuki-nakano]

What do you mean? Do you mean that the UI Events spec should be changed?

yes. It doesn't make sense compositionend (and/or compositionupdate) is fired on different element if the composition is committed after the element blurred. compositionstart should be fired on the focused element, but other composition events should be fired on the element which has the composition.

That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started.

In these days, modern IMEs queries content information before compositionstart. Therefore, moving focus at compositionstart is too late for native IMEs.

What are modern IMEs for you and what are native IMEs?

Most Japanese IMEs are using surrounding text for prioritize the items in its candidate list at converting a word. For example, "saku" can be converted to "咲く" (bloom) or "割く" (tear). Then, the previous word is important for guessing what the user expected. If the previous word is "花が" (A flower)、"咲く" must be better. If the previous word is "布を" (cloth), "割く" must be better. So, this modern IME may query previous content of the insertion point. There is another scenario, user may commit composition with wrong candidate item. Then, user can use "Kakutei-Undo" (undo the previous commit). When user type Ctrl + Backspace, IME sets selection which covers the last committed string and restart the composition. Therefore, for Japanese IMEs, the preparation before composition start is very important.

Similarly, as far as I know, Thai language's IME queries surrounding text and replace it with new composition string if it's necessary since Thai characters are combined one or more characters.

Moving focus at compositionstart kills such features completely. So, such web application isn't useful.

I tried this on Chrome on Mac OS X with one of the Mac OS X built-in IMEs, and the Chrome example still works, which means that it's no problem to move the focus somewhere else during compositionstart for Chrome on Mac OS X. Also Safari is able to move the focus during compositionstart.

I think that some features which I mentioned above must not work in such cases.

And what is "our editor's bug"?

I meant that it is a bug of Firefox's editor.

<input> and <textarea> do not matter much to us right now. In the Editing taskforce we are trying to fix editing in the browser, and having this work correctly is a key component.

contenteditable case is very complicated. If a document has two or more contenteditable elements, focus may be switched from one of them to another one. In such case, how to commit composition string at blur is difficult issue since all contenteditable elements share an editor instance in a document.

[ojanvafai]

@masayuki-nakano the use-case is to be able to move the focus into a shadow DOM inside the contentEditable. It would go in the same location in the DOM, so the IME would have the same text content to work with. It gives the web author a way of ensuring that the DOM changes from a composition only go into the DOM at compositionend as an atomic commit.

Before this suggestion, the contentEditable spec required the UA to do this shadow DOM behavior. I objected to that and pointed out that authors can already do this in script.

Even if you ignore shadow DOM, you could imagine putting it in a span inside the regular DOM, but in the same location. So the IME prediction would work the same, but the author would have full control over the DOM.

Does that use-case sound legitimate to you? I believe the IME would function exactly the same in all these cases. It's true that if you don't know what you're doing, you can mess this up, but that's true today too. You can just as easily mess with the DOM in other ways that will break IME prediction.

Regarding compositionend happening on the same element compositionstart happens on, why is that important? From a conceptual level, there are already plenty of instances where moving focus will cause the start/end events of a certain type to not fire on the same element. For example, changing focus during keydown causes keyup to fire on a different element. http://jsbin.com/kuvupalile/edit?html,console,output

There's a legitimate use-case for the behavior @johanneswilm is asking for.

[masayuki-nakano]

the use-case is to be able to move the focus into a shadow DOM inside the contentEditable. It would go in the same location in the DOM, so the IME would have the same text content to work with. It gives the web author a way of ensuring that the DOM changes from a composition only go into the DOM at compositionend as an atomic commit.

Before this suggestion, the contentEditable spec required the UA to do this shadow DOM behavior. I objected to that and pointed out that authors can already do this in script.

Even if you ignore shadow DOM, you could imagine putting it in a span inside the regular DOM, but in the same location. So the IME prediction would work the same, but the author would have full control over the DOM.

Although, I don't understand what "the use-case" you said indicates.

Thank you for your explain. I'm not familiar with "How Firefox treats shadow DOM", though, The last example is interesting. IIRC, starting Firefox 43, it won't commit/cancel composition even if selection is changed but the offset in the text isn't actually changed. So, the last example must work even on Firefox 43 for Windows or Linux. (IIRC, Firefox for Mac doesn't have this problem)

FYI: the testcase http://jsbin.com/wimuqowimu/15/edit?html,js,console,output won't work fine on Firefox 41 for Windows, unfortunately. Firefox 41 starts to support TSF on Windows. TSF is very rich framework, but very sensitive. For example, TSF doesn't allow to happen selection change/text change in focused editor at composition start. However, Firefox caches the previous contents of focused editor and lies TSF when it tries to query the content. If TSF-aware application returns unexpected content, TSF or some TIP (IME for TSF) may stop working :-( Anyway, I'll try to fix this bug latter...

Does that use-case sound legitimate to you? I believe the IME would function exactly the same in all these cases. It's true that if you don't know what you're doing, you can mess this up, but that's true today too. You can just as easily mess with the DOM in other ways that will break IME prediction.

If you said about http://jsbin.com/wimuqowimu/15/edit?html,js,console,output , this wants more hack for Firefox developers... I'm not sure if this can be fixed easy.

Regarding compositionend happening on the same element compositionstart happens on, why is that important? From a conceptual level, there are already plenty of instances where moving focus will cause the start/end events of a certain type to not fire on the same element.

The main reason of that is, some web apps may work stateful with compositionstart/compositionend. For example, if web apps need to handle input only when the focused editor doesn't have composition, they need to make a flag to store if there is composition with compositionstart and compositionend events. If such web apps fails to listen one of them, their input event handler won't work as expected. For making such web apps stateless, I suggested isComposing attribute to InputEvent, but the browsers except Firefox have not supported it yet.

Additionally, I believe that compositionstart, compositionupdate and compositionend are events of existing composition. Therefore, I believe that it makes sense the events fired on an element which includes the composition.

If a browser doesn't commit composition even when the element loses focus but the composition is committed by other reason (e.g., destroying the element), why should composition events be fired on focused element? If they are always fired on an element which has composition, such scenario can be fixed simply.

For example, changing focus during keydown causes keyup to fire on a different element.

Yeah, right. But I feel that it makes sense that keyboard events are always fired on focused element because a focused element is a receiver element of keyboard event at the time. The difference of composition events are, the events are fired for notifying web apps of the composition state changes. Therefore, an element which has composition string should receive it.

[johanneswilm]

FYI: the testcase http://jsbin.com/wimuqowimu/15/edit?html,js,console,output won't work fine on Firefox 41 for Windows, unfortunately. Firefox 41 starts to support TSF on Windows. TSF is very rich framework, but very sensitive. For example, TSF doesn't allow to happen selection change/text change in focused editor at composition start. However, Firefox caches the previous contents of focused editor and lies TSF when it tries to query the content. If TSF-aware application returns unexpected content, TSF or some TIP (IME for TSF) may stop working :-(

So it is currently working, but you are breaking it in the next version of Firefox? That does not sound like a step in the right direction. The spec says nothing about modifying the selection being forbidden as part of compositionstart, so that sounds as Firefox is planning to break with the spec?

Anyway, I'll try to fix this bug latter...

Ok, so you agree that this is a bug and you will fix it?

If you said about http://jsbin.com/wimuqowimu/15/edit?html,js,console,output , this wants more hack for Firefox developers... I'm not sure if this can be fixed easy.

Not sure what you are saying here. This is the same example that you will be breaking in Firefox 41 on Windows, unless you fix it, that you mention above.

[masayuki-nakano]

The spec says nothing about modifying the selection being forbidden as part of compositionstart, so that sounds as Firefox is planning to break with the spec?

Not planning, but if spec would be changed to so, Firefox implementation become much simpler... Sigh.

Ok, so you agree that this is a bug and you will fix it?

Yeah, if I can. Full support of TSF is very big project for web browsers because requirements (limitations) of TSF may not conform to some web standards. Therefore, Firefox including a lot of "fake" between TSF and DOM tree.

FYI: Chrome supports TSF with transitory context, that means to support partially, additionally it's not used in desktop mode in default settings. IE11 also supports TSF, but I'm not sure if it's using transitory context.

[kojiishi]

I still don't see the connection of what you're trying to do and the high level goals.

Fixing collaborative editing environments on browsers was the motivation of the Shadow DOM discussion, so I agree with the goal. But client-side Shadow DOM library will not help the goal as far as I understand. I don't know about the other goal; a clientside undo history, so I can't comment on it.

@ojanvafai knows Blink far much better than I do, so if he's against, I'm happy to follow. There should be side effects by making IME transparent or opt-in, I have to admit that I don't know all of them yet, and he should know them in depth.

So, if all other vendors are happy, and JS developers are happy, it just means that collaborative editing on browsers must support IME explicitly, or they'll be broken, as current all offerings are. It's not a disaster.

Back to the original topic, I'm still not convinced that defining the behavior of moving selections during compositions is a good thing. IME is still evolving to make better UX, and platforms do it differently. Android relies on it heavily, and OS X is going to change the UX in the next version. I'd like to avoid defining it so that we can allow platforms and IME to evolve further.

It may be possible to define it without limiting platforms and IME evolutions, but I suspect it's not an easy task. If it's really desired, I'd like to understand the use case better, and how defining it can help the use case.

[johanneswilm]

I still don't see the connection of what you're trying to do and the high level goals.

What part is unclear to you? Have you looked at the two examples linked above? They contain instructions on how they are to function and you can see the use case from them.

Fixing collaborative editing environments on browsers was the motivation of the Shadow DOM discussion, so I agree with the goal. But client-side Shadow DOM library will not help the goal as far as I understand.

Can you elaborate a bit about that? Currently this is working quite well in Chrome Desktop at least on Mac OS X and Linux (on Android the compositionend event seems to be triggered immediately after compositionstart takes places). So I don't understand what you are trying to say with " client-side Shadow DOM library will not help the goal ". It's using the browser's native Shadow DOM implementation (no polyfill or clientside library). Shadow DOM isn't yet available everywhere, so when it isn't available, one can use the second example code.

Back to the original topic, I'm still not convinced that defining the behavior of moving selections during compositions is a good thing. IME is still evolving to make better UX, and platforms do it differently. Android relies on it heavily, and OS X is going to change the UX in the next version. I'd like to avoid defining it so that we can allow platforms and IME to evolve further.

It is a pity that you weren't at the editing taskforce F2F meeting. The current spec allows for what we need. Now you are talking about changing the spec to disallow it? That does not sound like a good way forward, and if the goal was to get first-class IME support, this seems to instead just make it rather impossible to give IME in many webbased editors.

Do you have any alternative suggestion as to how to obtain the same result?

[johanneswilm]

btw, if you want to change the compositionstart event to be triggered at a time when composition has started already to the point where it is not possible to change the selection, maybe you could introduce a beforeCompositionstart event that is triggered before anything else takes place so that the caret can still be moved and which can be canceled to prevent the composition from taking place at all?

In order not to break anything, it would probably be better to just introduce it later.

Another thought may be to fix TSF itself. If the TSF IME works considerably different than other IMEs, and web browsers are one of the most common Programs people use all day, maybe Windows 10's TWSF should be fixed instead?

[kojiishi]

Can you explain what issues there are in the current collaborative editing on browsers, and how your example can help solving those issues?

The idea Ben and I were discussing was an early stage of the discussion to make IME inputs completely transparent, similar to what IME level 2 API on Windows does. If we discard the idea, there are several things apps must do to handle IME. Simple apps may need no-to-few, but collaborative editing might need several. The Shadow DOM-like building block was one part of making IME transparent. If it's not transparent, it looks to me that whether composition occurs within Shadow DOM or not isn't a big issue.

If you have clear understanding of what the issues are, and seeing connections between your code and those issues, that's ok. I'm sorry if I'm wrong, but I wonder we (including me) don't have clear understanding on them, no?

If you do, and moving compositions along with selection is a required feature to solve the issues, I'm ok to have that. Me not having an understanding isn't a blocking factor.

But please keep in mind that the behavior you described has been considered as a very bad IME UX for a long period of time. Also it's likely to break re-conversion as mentioned above. We probably have to make sure it's allowed only when JS really needs it, but the behavior never be exposed otherwise.

[johanneswilm]

@kojiishi: could you and @ojanvafai have a talk about what is and should be possible with Chrome? So far it seems he dismissed your idea of the transparent handling and you dismiss his idea of doing it through a JS controlled ShadowDOM. You guys know best what your code can be made to do without eating lots of resources. The F2F decided to go with this model which ojanvafai suggested, on the condition that IME through ShadowDOM would be possible in JS space. Ojanvafai briefly tested it during the meeting.