bug(linux): Backspacing and reordering is not working well in Firefox, Terminal

[mcdurdin]

Keyman does not appear to be working correctly in Firefox, Terminal: reordering, e.g. with Khmer Angkor, xEjmr is not​ producing ខ្មែរ (normally typed xjmEr).

[glasseyes]

I can't replicate the problem in Firefox. What are you doing in Firefox that exhibits the problem

I've confirmed looking at it that the same underlying text going to Terminal and text editor (gedit/pluma). The text editor uses pango and reorders and shapes.
The terminal appears to display each individual unicode character separately. The same lack of reordering occurs if you copy the text from somewhere else and paste it into the terminal. So this looks like a limitation in any virtual terminal. I've confirmed that the same thing happens in git bash on windows.

[mcdurdin]

For the khmer_angkor keyboard, I'll give baseline results with gedit, then Terminal and Firefox. It's important to understand that when I type xEjm, the khmer_angkor keyboard reorders the output at the time the m key is pressed, to give the equivalent of xjmE. The r is extraneous (but forms part of the complete word)...

• Ubuntu 18.04
• Firefox 63.0.3 64-bit

Baseline, gedit.

This is what is expected (and what I consistently see in gedit).

Correct order key	Correct order result	vs	Alternate keying order	Alternate order result
x	U+1781		x	U+1781
xj	U+1781 U+17D2		xE	U+1781 U+17C2
xjm	U+1781 U+17D2 U+1798		xEj	U+1781 U+17C2 U+17D2
xjmE	U+1781 U+17D2 U+1798 U+17C2		xEjm	U+1781 U+17D2 U+1798 U+17C2
xjmEr	U+1781 U+17D2 U+1798 U+17C2 U+179A		xEjmr	U+1781 U+17D2 U+1798 U+17C2 U+179A

Terminal

Correct order key	Correct order result	vs	Alternate keying order	Alternate order result
x	U+1781		x	U+1781
xj	U+1781 U+17D2		xE	U+1781 U+17C2
xjm	U+1781 U+17D2 U+1798		xEj	U+1781 U+17C2 U+17D2
xjmE	U+1781 U+17D2 U+1798 U+17C2		xEjm	U+17D2 U+1798 U+17C2 *
xjmEr	U+1781 U+17D2 U+1798 U+17C2 U+179A		xEjmr	U+17D2 U+1798 U+17C2 U+179A *

Firefox

On Firefox, the replacement sometimes works -- this looks like a race condition on backspacing. I can observe failure on my VM 4 times out of 5.

Correct order key	Correct order result	vs	Alternate keying order	Alternate order result
x	U+1781		x	U+1781
xj	U+1781 U+17D2		xE	U+1781 U+17C2
xjm	U+1781 U+17D2 U+1798		xEj	U+1781 U+17C2 U+17D2
xjmE	U+1781 U+17D2 U+1798 U+17C2		xEjm	U+1781 U+17C2 U+17D2 U+17D2 *
xjmEr	U+1781 U+17D2 U+1798 U+17C2 U+179A		xEjmr	U+1781 U+17C2 U+17D2 U+17D2 U+179A *

* These are incorrect results

[glasseyes]

For the khmer_angkor keyboard, I'll give baseline results with gedit, then Terminal and Firefox. It's important to understand that when I type xEjm, the khmer_angkor keyboard reorders the output at the time the m key is pressed, to give the equivalent of xjmE. The r is extraneous (but forms part of the complete word)...

* Ubuntu 18.04

* Firefox 63.0.3 64-bit

Baseline, gedit.

This is what is expected (and what I consistently see in gedit).
Correct order key Correct order result vs Alternate keying order Alternate order result
x U+1781 x U+1781
xj U+1781 U+17D2 xE U+1781 U+17C2
xjm U+1781 U+17D2 U+1798 xEj U+1781 U+17C2 U+17D2
xjmE U+1781 U+17D2 U+1798 U+17C2 xEjm U+1781 U+17D2 U+1798 U+17C2
xjmEr U+1781 U+17D2 U+1798 U+17C2 U+179A xEjmr U+1781 U+17D2 U+1798 U+17C2 U+179A

Terminal

Correct order key Correct order result vs Alternate keying order Alternate order result
x U+1781 x U+1781
xj U+1781 U+17D2 xE U+1781 U+17C2
xjm U+1781 U+17D2 U+1798 xEj U+1781 U+17C2 U+17D2
xjmE U+1781 U+17D2 U+1798 U+17C2 xEjm U+17D2 U+1798 U+17C2 *
xjmEr U+1781 U+17D2 U+1798 U+17C2 U+179A xEjmr U+17D2 U+1798 U+17C2 U+179A *

• These are incorrect results

Thanks for the fuller explanation.
The issue in Terminal is how it handles the coeng and backspace.
'm' generates U+1798 U+17D2 U+1798 U+17C2
The first deletes the U+1798
The second doesn't just delete the expected U+17D2 but deletes U+17C2 as well
Then the third back deletes U+1781 erroneously

Can you try a virtual terminal emulator on windows and see if that is general cross-platform virtual terminal behaviour and not just a problem on linux?

[glasseyes]

For Firefox and other apps such as Chrome/Chromium where ibus can't get the surrounding context:

I can confirm that it is a race condition on backspaces. I can't find a way to do a backspace in series with the text commit so there are no guarantees about when they'll happen

I don't like it at all but I can't find a way to speed up the backspace and the only way I've found to slow down the text commit enough to reliably[1] enter that text is to sleep for a realisticish keypress of 1/100s after each forward_backspace press and send a backspace key release after that.

[1] saying that, there is another race problem if you start typing too soon after starting ibus or switching to a keyman keyboard, at least on a slower computer

[mcdurdin]

1. Does the same race condition problem arise with (original) KMFL? I am assuming it does?
2. I don't think timing- or delay-based solutions are going to be robust. They will fail on loaded systems, and they also cause trouble for rapid input.
3. Therefore, I suspect the solution for the race will be similar to the one we engineered for Windows (https://blog.keyman.com/2018/10/the-keyman-keyboard-input-pipeline/).
4. I am not so worried about the race condition on switching input methods at this time. That's something we can live with.

[mcdurdin]

Can you try a virtual terminal emulator on windows and see if that is general cross-platform virtual terminal behaviour and not just a problem on linux?

Yes, this problem arises with Mintty on Windows (BKSP on U+1798 U+17D2 deletes both).

[glasseyes]

Can you try a virtual terminal emulator on windows and see if that is general cross-platform virtual terminal behaviour and not just a problem on linux?

Yes, this problem arises with Mintty on Windows (BKSP on U+1798 U+17D2 deletes both).

Good, as expected then so we can't do anything more about terminals.

[glasseyes]

1. Does the same race condition problem arise with (original) KMFL? I am assuming it does?

I haven't replicated it but the code path to forward the backspace is the same so I would expect it to.

2. I don't think timing- or delay-based solutions are going to be robust. They will fail on loaded systems, and they also cause trouble for rapid input.

I agree.

3. Therefore, I suspect the solution for the race will be similar to the one we engineered for Windows (https://blog.keyman.com/2018/10/the-keyman-keyboard-input-pipeline/).

Unfortunately I can't see a way to do something similar here. SendInput can send unicode in the generated key event but all I can do is send a virtual keycode, either forwarded back to ibus-keyman or generate a new one via the accessibility system.
ibus-keyman has no access to the Application window so it cannot send text itself, but tells ibus to send it.

4. I am not so worried about the race condition on switching input methods at this time. That's something we can live with.

OK

[mcdurdin]

After discussion, we will postpone this to v12 given it will require significant architectural design and perhaps upstream changes to ibus. This is not ideal but the same problems are present in KMFL and other ibus-based systems so it is a known limitation.

[nikita-moor]

A similar problem with Anki (Qt-based) and EuroLatin (SIL):

1. Start Anki
2. Press Add to add a new flashcard
3. Type xxxx in any field
4. Put the cursor in the middle of the string (xx|xx)
5. Start typing -a for "ā"

The result is xx-xā rather then expected xxāxx. Both engines, ibus-kmfl and ibus-keyman, produce this error. However, direct typing as xx-axx is correct (xxāxx).

P.S. Notepadqq is Qt-based but has no this problem.

[ermshiperete]

See also the Firefox part in #4429.

[ermshiperete]

I was able to reproduce this problem pretty consistently in a Bionic VM. A good site to test in Firefox might be https://www.editpad.org/.

[darcywong00]

Also likely relates to serializing input for #4029, #4030, #5154