Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Rapid touch inputs on the OSK produce inconsistent output #818
Once thought to affect all of KeymanWeb mobile + embedded, it turns out this particular error pattern happens solely within the iOS app.
Basically, what the title says - pressing two keys on a keyboard within a very tight timeframe leads to rather inconsistent output. Sometimes one of the two keys will have its output doubled, sometimes only one key is output once, etc. It's almost certainly something in the way that touch-event handlers are processing keyboard input, hampering Keyman's usability.
referenced this issue
May 4, 2018
Interesting... there are some differences among the platforms.
iOS suffers far more from this issues than does native KMW... there must be something in the iOS app causing this. Additionally, I can get the popup preview key from iOS to stick in situations where I cannot do so in native KMW.
iOS app preview key instructions:
Type the following keys in sequence on the default EuroLatin2 keyboard, only releasing each key after the next key in the sequence has been pressed. Go at a moderate speed, just quick enough to avoid generating the long-press menu:
Upon releasing the 'j' in a similar manner to the others, you should see the 'j' key's preview remain if input was well-timed, while also seeing the full output string 'alskdj' in the text area.
This should generalize to other keyboards and keystroke patterns - I simply chose this pattern for ease of typing and general readability.
Chances are, when keystrokes get sped up, something similar to this preview bug's cause likely underlies the overall issue for the iOS app.
changed the title from
KMW Mobile-mode: Rapid touch inputs on the OSK produce inconsistent output
Rapid touch inputs on the OSK produce inconsistent output
May 7, 2018
To test out this hypothesis further, I used one of our test iPads. After all, 'phone' form factor devices use key previews while 'tablet' form factor devices do not, all within the same version of the app.
iPad (no key preview):
iPhone (with key preview):
On a side note, I did get to see one issue highly related to #716 in action by accident when clearing the text area. I'll document it there.
On the other hand, I just ran another test using our test iPhone 5c, having neutered the key previews. That is, I blocked the embedded
That seemed odd, so I went on to run another test. Same device, but this time I forced the embedded KMW to believe the phone was a 'tablet' so that it would never even attempt to generate key previews.
It turns out that the iPhone 5c still suffers the same issues; there's some clear lag in how it attempts to handle input events.
OK. Rigged up an event-logging embedded variant of KMW to get us a better look at what's happening.
The 'touchstart' and 'touchend' handlers both are rigged to report which touch point is most relevant for the event, and
Intended input: "the quick brown fox"
It appears that the 'r' of 'brown' and the 'o' of 'fox' were not detected for some reason. Other errors arose from other sources.
The result, from Xcode logs of an on-device iPhone 5c experiment run:
It would appear that
It'll take some time to analyze this log more thoroughly, but what is alarming is that certain events that should occur sequentially in a synchronous manner appear to have other events transpiring in the middle of their code.
For comparison, the same "actual input" typed far more slowly, resulting in correct output:
Noting the information on https://docs.kioskproapp.com/article/840-wkwebview-supported-features-known-issues, we know that the
I'm hesitant to say this, but it looks like the touch event handlers themselves are operating asynchronously, possibly in parallel with each other.
Consider the following block from the "rapid touch" log, with irrelevant info omitted:
Suppose, for a moment, that we could assume these log messages all occurred sequentially, with no race conditioning reordering involved.
If we start with this assumption, then the third and fourth lines appear to be out of order - the second line (end: key = 'default-K_R'`) should have immediately triggered the fourth line, as that log message is in a function called by the same function giving the second line's log.
If we assumed instead that these lines were merely reordered due to asynchronous logging, however, that gives us a major question for the seventh line (timestamp 13:00:48.803): where did the second
If we instead operate from the original assumption, we have a valid explanation... but it's bad news. If the third line's function reaches a certain point before the second line's function reaches its critical point, regarding the property
One more run in an attempt to further confirm the behavior - this time I put similar logging in place, but this time collated all the log messages within the WebView's embedded page via JS
Note the first few keystrokes - this time, ala race condition, the T's 'end' event did start the processing first... but it didn't clear
Since we're purely in JS land now, note that each alleged thread must output
It appears that this behavior is due to legacy use of
Nixing this isn't enough to solve all the issues, however, as some of the OSK touch events aren't recognizing the intended key. This has been determined as related to #863.