Make "Input Events Level 1" declare "beforeinput cancelable" of "insertCompositionText" as "Undefined"

[masayuki-nakano]

Level 1 spec declares "insertCompositionText" is not cancelable.
https://rawgit.com/w3c/input-events/v1/index.html#interface-InputEvent-Attributes

However, Level 1 does not have cancelable "insertFromComposition". And Chrome supports Level 1, and Firefox will follow it soon. This means that web apps cannot cancel inserting text only from composition so that I'd like to suggest that only when IME commits composition, Level 1 browsers should make beforeinput whose inputType is "insertCompositionText" cancelable. For doing this, it should be defined as "Undefined".

(Or, adding "insertFromComposition" into Level 1 might be more reasonable.)

[johanneswilm]

Hey @masayuki-nakano ,
yes, the way it should be written, all browsers compliant with level 2 should also automatically always be compliant with level 1. Therefore, it should always be possible to have a "insertFromComposition" inputType and still be compliant with level 1. I think that the way we wrote it, we thought the note "Other specifications may expand on this definition." would cover that case where for example Webkit could take an inputType from level 2 and still be compliant.

If the advice we received on that is incorrect and we need to add it to the level 1 spec as well, I think we need to add "insertFromComposition" to level 1 as well.

@frivoal @marcoscaceres - any comment?

[johanneswilm]

This means that web apps cannot cancel inserting text only from composition

@masayuki-nakano The changes the Chrome team did to level 1 was to make it possible to make all the text insertion events not be cancelable (only formatting changes are cancelable). The idea behind this was, AFAIAA, that given that it was not possible to have composition be cancelable on Chrome on Android, one wanted to make all the text insertion not be cancelable, so that JS editor projects would be forced to write code to undo the last change and then redo it, rather than write editors that only work on desktop computers in Latin script languages.

What you seem to describe is something similar to level 2 - which is what we actually wanted but couldn't do due to this bug on Android.

Given that they made all text insertion on-cancelable in Chrome, also from keyboards, I would be surprised if they would now want to make it cancelable anyway. And if you implement it differently in Firefox rather than use either level 1 or 2, then we are back at square one where JS editor devs need to write browser specific code.

[johanneswilm]

Is there a PR by someone from Mozila on adding this to Input Events, Level 1 somewhere and could we get someone to present the proposal to add this to the spec before it is being moved to CR either at TPAC or one of the Editing Taskforce monthly meetings some time soon?

@annevk @saschanaz @masayuki-nakano @smaug----

[masayuki-nakano]

This means that web apps cannot cancel inserting text only from composition

@masayuki-nakano The changes the Chrome team did to level 1 was to make it possible to make all the text insertion events not be cancelable (only formatting changes are cancelable).

No, on Chrome, I can cancel beforeinput event for insertText, insertReplacementText, deleteContentBackword, deleteContentForward, etc. And it actually prevents built-in editor of Blink to do anything.

The idea behind this was, AFAIAA, that given that it was not possible to have composition be cancelable on Chrome on Android, one wanted to make all the text insertion not be cancelable, so that JS editor projects would be forced to write code to undo the last change and then redo it, rather than write editors that only work on desktop computers in Latin script languages.

Well, I didn't think about Android's IME limitation. However, I'm suggesting about the case only when the composition ends. I.e., in most cases, it won't hit any IME limitation of any platforms, I guess.

Given that they made all text insertion on-cancelable in Chrome, also from keyboards, I would be surprised if they would now want to make it cancelable anyway. And if you implement it differently in Firefox rather than use either level 1 or 2, then we are back at square one where JS editor devs need to write browser specific code.

Yes, therefore, I'd like to here second opinions of other vendors.

[johanneswilm]

@masayuki-nakano The changes the Chrome team did to level 1 was to make it possible to make all the text insertion events not be cancelable (only formatting changes are cancelable).

No, on Chrome, I can cancel beforeinput event for insertText, insertReplacementText, deleteContentBackword, deleteContentForward, etc. And it actually prevents built-in editor of Blink to do anything.

Aha. In that case it seems like Chromium has moved it's implementation closer to level 2. In all of the cases where the Blink implementation matches that of Webkit/level 2, I think we should adjust level 1 to be the same as level 2. The only point with level 1 was that Blink could not implement all of level 2 for technical/internal reasons and so if those reasons are gone, we should make the delta between the two as small as possible by adjusting level 1.

@gked Any idea on who working on this in the Blink codebase who could verify that Chromium has now implemented level 2 for the various input types and that they are cancelable on all platforms?

The idea behind this was, AFAIAA, that given that it was not possible to have composition be cancelable on Chrome on Android, one wanted to make all the text insertion not be cancelable, so that JS editor projects would be forced to write code to undo the last change and then redo it, rather than write editors that only work on desktop computers in Latin script languages.

Well, I didn't think about Android's IME limitation. However, I'm suggesting about the case only when the composition ends. I.e., in most cases, it won't hit any IME limitation of any platforms, I guess.

If I remember correctly, the issue was with partially committed IME strings - what happens if there is a long string in the IME and only part of that is committed to the DOM.

If you can make it work like Webkit/level 2 by using "insertFromComposition", then it would mean that JS editor authors would only have to consider two cases instead of three different cases.

[masayuki-nakano]

React developer said that they also want to handle committing composition with beforeinput events.

I believe that this is important issue and Blink should change the behavior for making web apps possible to handle composition easier. (Then, Firefox will ship beforeinput with the new behavior.)

@gked @garykac ping for this.

[masayuki-nakano]

Ah, if insertFromComposition will be added to Level 1, it might be better to add deleteCompositionText too. Then, WebKit's behavior and the others become almost compatible.

[johanneswilm]

Ping @alexkeng and @snianu. See also w3c/editing#275 . Could we get an answer to whether we can make level 1 get most of what is specified in level 2 thus far?

[snianu]

Any idea on who working on this in the Blink codebase who could verify that Chromium has now implemented level 2 for the various input types and that they are cancelable on all platforms?

@bokand @mustaqahmed @NavidZ Adding few folks from Chrome team who would know more about these changes.

[snianu]

Just to follow-up on this issue. @gked found the change in Chromium that made the beforeInput events cancellable. Here is the thread about this issue: w3c/editing#275.

@masayuki-nakano @johanneswilm Re deleteCompositionText: According to the spec we should fire deleteCompositionText whenever IME is replacing a range during an active composition, but on Safari I see this event getting fired only when a composition is committed and not when the composition state is in-progress and the text is being replaced when a candidate is selected from the candidate window(FF, Chrome & Edge do not fire this event at all). The spec language is pretty clear and we should always fire deleteCompositionText when a text is replacing (that is being composed or committed) an existing range in the DOM. Thoughts?

[johanneswilm]

@snianu In the table it says about deleteCompositionText that it occurs: "delete the current composition string before committing a finalized string to the DOM". The way you describe Safari's behavior sounds to me like it complies with this correctly. The step 5 in the event order you link to also sounds correct - this is only to happen when the IME is about to finish and the change is to be committed permanently to the DOM or if, due to buffer issues, a part of the composition text is committed permanently to the DOM while another part continues in the composition.

Maybe some of the wording should be clarified? But I think Safari is doing the right thing if it does what you describe. The whole point of first removing it (with deleteCompositionText) and then re-adding it (with insertFromComposition) is that it allows JS editors to isolate the DOM changes made by the IME.

[johanneswilm]

FF, Chrome & Edge do not fire this event at all

That is because FF, Chrome & Edge have only implemented level 1 of the spec (and apparently recently Chrome/Edge recently also parts of level 2, except those parts that relate to IME - the deleteCompositionText and insertFromComposition events).

[masayuki-nakano]

Oh, Firefox implemented Leve2 only for the inputType values behind a pref, but I misunderstood the meaning of deleteCompositionText. I thought that it's set when composition is committed with empty string.
https://searchfox.org/mozilla-central/rev/03224522336f60a1a61a87e1fcd4feb0a0315a9b/editor/libeditor/TextEditor.cpp#826-833
https://searchfox.org/mozilla-central/rev/03224522336f60a1a61a87e1fcd4feb0a0315a9b/editor/libeditor/EditAction.h#507-511

Of course, I could change the behavior as Safari does. However, it causes input event fired twice for deleteCompositionText and insertFromComposition. This may break some web apps because some web apps may be sensitive to input events around compositionend. So, I have a concern about changing input event behavior because some web apps may be sensitive to input events around compositionend.

[masayuki-nakano]

And I feel odd for current definition of deleteCompositionText. That says

delete the current composition string before commiting a finalized string to the DOM

This means that only while dispatching beforeinput and input event, the composition string should be removed from the DOM temporarily. Safari, actually does so, but from point of view of Gecko's implementation, it's redundant DOM change since in most cases, same string will be inserted to the DOM again. Why is this required? Why it's not enough canceling insertFromComposition just to remove the commit string?

[masayuki-nakano]

Ah, Safari fires input event for deleteCompositionText, but the spec draft does not define it should be fired. (Perhaps, only when it's not followed by insertFromComposition, it should be fired though.)

[masayuki-nakano]

And neither beforeinput nor input event for deleteByComposition is fired on Safari. Perhaps, this shouldn't be ported to Level 1.

[johanneswilm]

Of course, I could change the behavior as Safari does.

@masayuki-nakano If you fire the deleteCompositionText event today, but you do so incorrectly, it would be my strong preference for you to fix Firefox as Safari has implemented this correctly and we are kind of missing the point with this if every browser implements this with an entirely different meaning. We have been in contact with a lot of JS editors and the Safari behavior is what was required of this. It's probably the biggest point of level 2 at this stage, given that level 1 and level 2 will soon be almost the same.

but from point of view of Gecko's implementation, it's redundant DOM change since in most cases, same string will be inserted to the DOM again.

Because this way the JavaScript editor can contain the DOM changes caused by IME and has the chance to influence them. For example, in a given editor, the JavaScript may not allow certain changes, or put each word or each letter inside a dom node.

We have been discussing this for many years in the Editing Taskforce and tried to find other options, but the approach that we wrote in level 2, that Safari has implemented, is the only viable way we have found. Yes, there is one dom update that is not always needed, but that is very little in comparison to the amount of dom updates that are occurring inside a JavaScript-based editor, and there is a natural speed limit anyway in that the user cannot type endlessly fast.

[johanneswilm]

Ah, Safari fires input event for deleteCompositionText, but the spec draft does not define it should be fired. (Perhaps, only when it's not followed by insertFromComposition, it should be fired though.)

The beforeinput and input events should always come in pairs where the input event is fired after the DOM change if the beforeinput event was not cancelled, so to me it sounds like the Safari behavior is correct and like we need to add a step between 5.2 and 5.3. to add the inputevent for "deleteCompositionText".

[johanneswilm]

And neither beforeinput nor input event for deleteByComposition is fired on Safari.

We should hear with Safari why this is happening and if there are reasons for why they did not implement this part.

Perhaps, this shouldn't be ported to Level 1.

The latest from the Editing Taskforce is that we have decided to investigate whether we can merge level 1 and level 2. This may be in the form of us dropping level 1, and marking the deleteCompositionText/insertFromComposition part as optional in level 2. That way, all implementations should comply to the spec AFAWK.

[masayuki-nakano]

Of course, I could change the behavior as Safari does.

@masayuki-nakano If you fire the deleteCompositionText event today, but you do so incorrectly, it would be my strong preference for you to fix Firefox as Safari has implemented this correctly and we are kind of missing the point with this if every browser implements this with an entirely different meaning.

No worry, we'd like to ship beforeinput with the behavior which is agreed by both WebKit and Blink developers.

We have been in contact with a lot of JS editors and the Safari behavior is what was required of this. It's probably the biggest point of level 2 at this stage, given that level 1 and level 2 will soon be almost the same.

If so, why WebKit still behaves differently even from the Level 2 draft? I bet nobody has checked deeply around composition.

but from point of view of Gecko's implementation, it's redundant DOM change since in most cases, same string will be inserted to the DOM again.

Because this way the JavaScript editor can contain the DOM changes caused by IME and has the chance to influence them. For example, in a given editor, the JavaScript may not allow certain changes, or put each word or each letter inside a dom node.

deleteCompositionText is not cancelable, and it does not have any information for the commit string. So, isn't insertFromComposition is what the developers want to do in your examples? If so, I don't understand why it's necessary.

We have been discussing this for many years in the Editing Taskforce and tried to find other options, but the approach that we wrote in level 2, that Safari has implemented, is the only viable way we have found. Yes, there is one dom update that is not always needed, but that is very little in comparison to the amount of dom updates that are occurring inside a JavaScript-based editor, and there is a natural speed limit anyway in that the user cannot type endlessly fast.

Unfortunately, there is no way to test composition with TestDriver for WPT, but even excepting the reason, there is too few tests for beforeinput and getTargetRanges. So, I don't believe that WebKit works perfectly as what you assume. (I always think that why do I --the last implementer of beforeinput event-- need to write every test as "tentative" for every case?)

Ah, Safari fires input event for deleteCompositionText, but the spec draft does not define it should be fired. (Perhaps, only when it's not followed by insertFromComposition, it should be fired though.)

The beforeinput and input events should always come in pairs where the input event is fired after the DOM change if the beforeinput event was not cancelled, so to me it sounds like the Safari behavior is correct and like we need to add a step between 5.2 and 5.3. to add the inputevent for "deleteCompositionText".

For the simple design, I understand that beforeinput and input events should be fired as a set. However, nobody must not want to break backward compatibility with dispatching input events newly. So, at least in this case, I don't understand the merit of firing new "input" event for deleteCompositionText if it's followed by input event for insertFromComposition.

And this issue is same as #86 . Newly dispatching input event may break existing web apps which do not check inputType. (If it had new event name rather than extending existing input event, it'd have no problem from the point of view of backward compatibility.)

[johanneswilm]

We have been in contact with a lot of JS editors and the Safari behavior is what was required of this. It's probably the biggest point of level 2 at this stage, given that level 1 and level 2 will soon be almost the same.

If so, why WebKit still behaves differently even from the Level 2 draft? I bet nobody has checked deeply around composition.

It's a combination of several things. One is that the dom diffing libraries have become quite good. Another is that some of the worst contenteditable glitches have been taken care of. A third is that still not all browsers have implemented the beforeinput event (notably Firefox), which means it is not an approach that works universally anyway. And then probably the extra input event just didn't bother anyone enough. But I agree that we need to get this in order going forward. Once all browsers have this by default, we should expect more users as well.

deleteCompositionText is not cancelable, and it does not have any information for the commit string. So, isn't insertFromComposition is what the developers want to do in your examples? If so, I don't understand why it's necessary.

The idea is that there is an before/input event for each editor dom change and that the DOM doesn't just change without there being such an event. Someone might just listen to all the before/inputevents and take a snapshot of the DOM at each stage and then keep a history as to how things changed. deleteCompositionText resets the DOM to how it was before the composition started, so that insertFromComposition is "containable".

Unfortunately, there is no way to test composition with TestDriver for WPT, but even excepting the reason, there is too few tests for beforeinput and getTargetRanges. So, I don't believe that WebKit works perfectly as what you assume. (I always think that why do I --the last implementer of beforeinput event-- need to write every test as "tentative" for every case?)

I have found bugs in both Chrome and Webkit in the past on input events. I'm not assuming they are bugfree as of today. I have tried to write tests for everything I could test using the testdriver. As you say, it is somewhat restricted. I would very much welcome more tests as long as they are really testing the things that are in level 1 or 2 of the spec. Help/collaboration would be greatly appreciated.

Newly dispatching input event may break existing web apps which do not check inputType. (If it had new event name rather than extending existing input event, it'd have no problem from the point of view of backward compatibility.)

I understand the concern and I agree that it would possibly have been better to choose another event name for this, but given the amount of bugs editors have had to deal with over the years, which have been constantly shifting and changing, I don't really think that is a great concern. Any JS editor needs to be able that any of the browsers change their behavior from version to version without any announcements for the past decade or so. We did not get a great uproar of users complaining about Safari introducing this input event either.

[masayuki-nakano]

@johanneswilm: Sorry for the delay to reply.

And then probably the extra input event just didn't bother anyone enough.

I don't think so. In my experience, web developers usually add workaround without filing issues to web browsers not spec. And this is also same as browser users. Actually, we've got a web-compat issue report which is caused by a bug of Blink that is not dispatching input event in a specific case. Therefore, I'm really afraid to fire new input event especially around compositionend event.

But I agree that we need to get this in order going forward. Once all browsers have this by default, we should expect more users as well.

Thanks. My point here is, when porting a part of Level 2 to Level 1, any things which may change current behavior should be ported to Level 1 because if it's fine, both Mozilla and Google implement Level 2 as-is. Supporting Level 1 is caused some breaking the web risk.

deleteCompositionText is not cancelable, and it does not have any information for the commit string. So, isn't insertFromComposition is what the developers want to do in your examples? If so, I don't understand why it's necessary.

The idea is that there is an before/input event for each editor dom change and that the DOM doesn't just change without there being such an event. Someone might just listen to all the before/inputevents and take a snapshot of the DOM at each stage and then keep a history as to how things changed. deleteCompositionText resets the DOM to how it was before the composition started, so that insertFromComposition is "containable".

Well, then, it becomes off topic though, deleted text by composition should be restored temporarily? Safari does not behave so.

Anyway, my main proposal here is,

• Port deleteByComposition, insertCompositionText and deleteCompositionText inputType values from Level 2 to Level 1.
• May not dispatch input event for deleteByComposition for backward compatibility.
• May not dispatch input event for deleteCompositionText for backward compatibility if it's followed by insertCompositionText.
• Define beforeinput event for insertCompositionText is "cancelable" or "undefined" for web apps which want to handle composition only with beforeinput event.

[snianu]

Actually, we've got a web-compat issue report which is caused by a bug of Blink that is not dispatching input event in a specific case. Therefore, I'm really afraid to fire new input event especially around compositionend event.

I think we should fix Chromium's behavior so all browsers have a standard behavior. I'll see if I can fix that in Chromium, but this should be a bug on Chrome and shouldn't be considered a standard behavior as that also impacts other composition events like you mentioned. From what I understand, @johanneswilm (please correct me if I'm wrong here) point is that web devs need events before and after DOM mutation during composition in order to make deterministic decisions about DOM changes. If we don't fire input event for deleteCompositionText after firing beforeInput event, then it's harder for web devs to derive what has changed in the DOM during a composition by just listening to input & beforeinput events.

But there is also concern about input event not getting fired when the in-progress composition has ended because the input control lost focus (e.g. user clicked somewhere outside the document). This is tricky to detect in Chromium as this sometimes leads to a cancel composition event from the IME with a text that is different than the partially composed text (On Windows MS Pinyin IME does that) or sometimes it just cancels the event which removes the composition marker and doesn't insert/remove any text from the DOM. I'm not really sure if we can standardize it because the IMEs have different behavior in this case.

Re: deleteByComposition, I don't see that event getting fired in Safari at all. Maybe they haven't implemented it, but I still don't understand the purpose of this event when we already have insertCompositionText that gets fired when the current composed string is replaced (during reconversion or during candidate window selection).

Also not related to beforeInput events exactly, but I see Safari's event ordering is different than Chrome & FF. Keydown and Keyup events are fired before the compositionstart event which is not what is expected as the key hasn't been released yet while the composition is in-progress so it is a bit confusing. e.g. During English typing, we don't fire keyup event until the user has released the key.

[johanneswilm]

deleteByComposition is meant to be used when recomposing existing text. So imagine on your phone, you click on a word and that word is then loaded into the IME and you exchange it with another. This event should then appear on some platforms (mobile) and likely not at all with some desktop IMEs that don't support recomposition of text.

The beforeInput of deleteByComposition is cancelable whereas that of insertCompositionText is not. The idea was that the JS editor might want to handle IME entirely separate from the context around it in order to not confuse other people who write in the document at the same time. This can be done by canceling the beforeinput event of deleteByComposition and move the caret to a different element. The composition should now take place in this new element. After the composition is done, the JS editor can move the newly composed text back to where the selection was originally.

[masayuki-nakano]

@snianu

Actually, we've got a web-compat issue report which is caused by a bug of Blink that is not dispatching input event in a specific case. Therefore, I'm really afraid to fire new input event especially around compositionend event.

I think we should fix Chromium's behavior so all browsers have a standard behavior. I'll see if I can fix that in Chromium, but this should be a bug on Chrome and shouldn't be considered a standard behavior as that also impacts other composition events like you mentioned.

Yes, of course, the Chromium's bug should be fixed. But I tried to say that different input event behavior may be delicate for some web apps especially around composition since even browsers cannot control native IME perfectly. (And thank you very much to work on the Chromium's bug!)

From what I understand, @johanneswilm (please correct me if I'm wrong here) point is that web devs need events before and after DOM mutation during composition in order to make deterministic decisions about DOM changes. If we don't fire input event for deleteCompositionText after firing beforeInput event, then it's harder for web devs to derive what has changed in the DOM during a composition by just listening to input & beforeinput events.

Right. But if it's followed by another beforeinput event, it's available to use for that. And if not followed by a beforeinput event, input event should be dispatched (except when canceled, general speaking).

But there is also concern about input event not getting fired when the in-progress composition has ended because the input control lost focus (e.g. user clicked somewhere outside the document). This is tricky to detect in Chromium as this sometimes leads to a cancel composition event from the IME with a text that is different than the partially composed text (On Windows MS Pinyin IME does that) or sometimes it just cancels the event which removes the composition marker and doesn't insert/remove any text from the DOM. I'm not really sure if we can standardize it because the IMEs have different behavior in this case.

In Gecko, a composition creates a manager object internally, and stores the event target for the following composition events until compositionend.

I think that UI Events should define event target of composition events as so.

Re: deleteByComposition, I don't see that event getting fired in Safari at all. Maybe they haven't implemented it, but I still don't understand the purpose of this event when we already have insertCompositionText that gets fired when the current composed string is replaced (during reconversion or during candidate window selection).

I think that its input event is not required if followed by beforeinput event of insertCompositionText.

Also not related to beforeInput events exactly, but I see Safari's event ordering is different than Chrome & FF. Keydown and Keyup events are fired before the compositionstart event which is not what is expected as the key hasn't been released yet while the composition is in-progress so it is a bit confusing. e.g. During English typing, we don't fire keyup event until the user has released the key.

Yeah, Safari's behavior is wired and invalid (without reasonable reasons). So, anyway, their contributors need to rewrite around it.

@johanneswilm

deleteByComposition is meant to be used when recomposing existing text. So imagine on your phone, you click on a word and that word is then loaded into the IME and you exchange it with another. This event should then appear on some platforms (mobile) and likely not at all with some desktop IMEs that don't support recomposition of text.

FYI: From point of view of native apps like browsers, it cannot distinguish that whether the new composition is for recomposition for selected text or new composition replacing selected text.

The beforeInput of deleteByComposition is cancelable whereas that of insertCompositionText is not. The idea was that the JS editor might want to handle IME entirely separate from the context around it in order to not confuse other people who write in the document at the same time. This can be done by canceling the beforeinput event of deleteByComposition and move the caret to a different element. The composition should now take place in this new element. After the composition is done, the JS editor can move the newly composed text back to where the selection was originally.

I understand the usage for the beforeinput of deleteByComposition, but I don't understand for input of it. If it does not break anything, it's okay though. In my experience, dispatching existing event newly is risky change. So, there should be a big benefit.

[snianu]

Right. But if it's followed by another beforeinput event, it's available to use for that. And if not followed by a beforeinput event, input event should be dispatched (except when canceled, general speaking).

The input event order says that the input event is fired after the DOM is updated. How would you know if the DOM was updated or not if we just fire beforeinput event for deleteCompositionText ? Firing beforeinput events back to back for deleteCompositionText & insertCompositionText doesn't mean that the DOM was updated in-between those two events by the IME(although technically we do, but its not very deterministic if there is a chance to drop the DOM updates during these events). If the web devs are taking a snapshot of the DOM during beforeinput and after input to determine what exactly has changed, then it would be hard for them to derive this information just based on beforeinput events.

FYI: From point of view of native apps like browsers, it cannot distinguish that whether the new composition is for recomposition for selected text or new composition replacing selected text.

This is true and I agree that it would be hard for us to differentiate those two cases.

[snianu]

When we recompose a text, Safari fires deleteByComposition first and then starts a composition. The event order is as follows:
deleteByComposition -> compositionstart -> compositionupdate -> insertCompositionText -> deleteCompositionText -> compositionend -> compositionstart -> compositionupdate -> insertCompositionText -> deleteCompositionText ->insertFromComposition -> compositionend
On Windows IMEs, there is no special event from text input services to indicate that the IME is starting a reconversion. It comes as a regular composition but with a range to replace existing text, so I'm not really sure if we can support this sequence.