Editing Korean letters with NVDA and braille display issue. #5640

Closed
khsbory opened this Issue Dec 30, 2015 · 27 comments

6 participants

@khsbory

Hello, I’m Hyongsop Kim in Korea.
In Notepad or another word program, if I type just English letters, there is no problem.
But if I type Korean letters, my braille display cannot be scrolled to prior or next line, just panning within same line.
If I go to prior or next line, I have to press keyboard arrow keys, not braille display scroll key.
Also in word program, if I type some Korean letters and type English letters, the English letters are not appeared in my braille display.
So if I check, I have to move keyboard arrow keys.

Please check and fix this issue.

Thank you.

@josephsl

Hi,
Confirmed.
Technical: This is because of the fact that input composition window gets focused, and panning the display shows this window. Panning up and down has no effect due to focus moving to this composition window.
@michaeldcurran and @nishimotz: Is there anything (like a hack) we can do for braille users using East Asian languages who may experience this issue? Thanks.

@jcsteh
@josephsl
@khsbory
@jcsteh
@khsbory
@jcsteh
@khsbory
@jcsteh

Also in word program, if I type some Korean letters and type English letters, the English letters are not appeared in my braille display.

Is this fixed in the latest snapshot for the "next" branch? I'm guessing this is #4145.

@khsbory
@jcsteh
@josephsl
@jcsteh
@josephsl
@jcsteh
@khsbory
@josephsl

Hi,

English characters disappear when typing them after typing Korean.

STR:
1. Type soething in English.
2. Switch to Korean and type something in Korean.
3. Switch back to English.
Expected: English characters are displayed.
Actual: English characters are missing.
4. Type some more English letters.
5. Switch to Korean and type something in Korean.
Result: The missing English characters are then shown.
Thanks.

@jcsteh

When the English characters are missing, if you alt+tab out and back in again, do they appear? Can you give me a quick key sequence to get some Korean characters for testing?

@josephsl

Hi,
Yes, English characters appear when I press Alt+TAB to switch from and to Word.

Some keystrokes I used:
1. After installing Korean input method, press Windows+SPACE to switch to Korean (Windows 8 and later; I'm using Windows 10).
2. Press right Alt to switch between Korean (native) and English (alphanumeric) inputs. Then type:

English: Hello (then no space)
Korean: xptmxld (testing)
English (no space after Korean): Hi

Note that braille output table should be Korean also.
Thanks.

@michaelDCurran

I have located the underlying issue:
in NVDAHelper.handleCompositionEnd: focus is moved back to the parent of the composition (the document) with an executeEvent. However, IAccessibleHandler, when routing caret events to the focus, only checks lastQueuedFocusObject. LastQueuedFocusObject is most likely still the composition object.

The easiest way to fix this is to use queueEvent instead of executeEvent. However, there may be side-effects to do with timing with input composition handling. We will have to just see.

A safer way might be to have executeEvent take an optional keyword argument, say "fromQueue" which would only be set to true when coming from the queue (i.e. not called directly by outside code). If fromQueue is False, executeEvent itself should update lastQueuedFocusObject as from an external point of view, the call is really the equivalent to queueEvent except that it happens straight away.

Any thoughts @jcsteh?

@michaelDCurran

A simple way to describe this issue, if not noted already, is that once composition ends, although the braille display again shows the document content, it no longer updates due to typed characters. Selection detection is also broken.

@jcsteh

in NVDAHelper.handleCompositionEnd: focus is moved back to the parent of the composition (the document) with an executeEvent. However, IAccessibleHandler, when routing caret events to the focus, only checks lastQueuedFocusObject. LastQueuedFocusObject is most likely still the composition object.

But handleInputCompositionStart also uses executeEvent, so why would lastQueuedFocusObject be the composition object? And if it isn't, what is it then?

The easiest way to fix this is to use queueEvent instead of executeEvent. However, there may be side-effects to do with timing with input composition handling. We will have to just see.

I'm actually a bit worried about the fact that we use executeEvent at all. executeEvent isn't supposed to be called from a background thread. That means speech and braille could be happening in the background thread and we only guarantee that those need to work in the main thread. Also, there could be race conditions with focus events between the two threads.

A safer way might be to have executeEvent take an optional keyword argument, say "fromQueue" which would only be set to true when coming from the queue (i.e. not called directly by outside code).

If there's some special reason we need to use executeEvent for input composition, we could also just set lastQueuedFocusObject directly before calling it. Obviously, we shouldn't be tweaking eventHandler internals like that, but it's probably safer to deal with a special case internally rather than exposing something publicly which might be frequently misunderstood. Yes, anyone could tweak lastQueuedFocusObject, but if they do, we can say "you tweaked stuff you shouldn't have; you're on your own".

@khsbory
@josephsl
@nvaccessAuto

Incubated in cd543f6.

@jcsteh jcsteh added a commit that closed this issue Feb 1, 2016
@michaelDCurran michaelDCurran Fixed problems with braille display output when entering Korean chara…
…cters.

NVDAHelper.handleInputCompositionStart: the new input composition NVDAObject's parent is set to the most correct focus (i.e. objectWithFocus). However, if this is equal to NVDA's focus object, use that instead, in order to still maintain any possible state.
Fixes #5640.
b41ea09
@jcsteh jcsteh closed this in b41ea09 Feb 1, 2016
@nvaccessAuto nvaccessAuto removed the incubating label Feb 1, 2016
@jcsteh jcsteh added this to the 2016.1 milestone Feb 1, 2016
@nishimotz

Sorry for slow response.
The use of Microsoft Japanese IME (or Office IME) and braille display causes similar issue, which still occurs the try build nvda_snapshot_try-i5640-11798,c38150f.exe.

Recently Japanese Team have confirmed that this issue happens with 2013.3jp and later versions.
(not yet tested with prior versions)

It should be filed as new issue, because Japanese IME may be something different from Korean IME.
Note that Japanese braille translation is only supported by NVDA JP and not yet supported by NVDA itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment