When on the Japanese IME in Rōmaji input mode, compose key creates halfwidth katakana

[ghost]

Platform: Windows 10 Home x64, US (but computer is in Japanese); WinCompose v0.7.7 on Menu key

Steps to reproduce:

• Set keyboard to Japanese IME, Rōmaji kana input mode, but set to the Rōmaji input mode
  • This can be accomplished by pressing alt-` to switch into hiragana input, pressing ctrl-shift-capslock to switch into one-key-per-character input mode, and repeating alt-` to go back into Rōmaji mode.
• Try and use WinCompose.
  • For example, typing <Multi_key> a yields ﾁ.

[samhocevar]

Thanks for the detailed information, I was able to reproduce the bug, which is good news! But for now it seems very bizarre, I’ll have to investigate a bit.

[ErikAnderson3]

First off, THANK YOU for writing and sharing this tool. Microsoft's user experience is ... woeful, sometimes.

I came here to report what seems to be the same issue as elyatai has encountered.

Technical details:

WinCompose v0.8.2
Windows 10 v 1709 (build 16299.967), Enterprise edition
Locale for non-Unicode-aware programs configured to Japanese (Shift-JIS).
Keyboard set to Japanese IME. EN-US keyboard removed (deliberately by me, since the JA IME includes EN-US functionality, and the EN-US keyboard options just get in the way).

WinCompose debug log:

The top red section shows the key combo working correctly. The bottom red section shows some kind of interference, producing an unexpected and unwanted result.

Symptoms:

Since receiving Windows Update KB4486996 on February 25 (pushed by my employer's IT department, not directly from MS), I've noticed various odd behavior, I finally had time to isolate some of the issue with WinCompose today. NOTE: This issue comes and goes. I'm really stumped why.

I have CAPSLOCK configured for my Compose key. I used to type CAPSLOCK + [compose sequence] to get the glyph I wanted. Since updating, occasioally this behavior will only work as expected in WinCompose itself. When things go funny (for reasons yet unknown), in everything else, something (the IME? some other Windows component?) interferes, so that CAPSLOCK + [one key] inserts a half-width katakana. Longer compose sequences result in a half-width katakana + the regular keyboard character for that key.

For example, CAPSLOCK + --- (three hyphens) should produce an em-dash:

—

When things are misbehaving, I instead get:

ﾎ--

This is a half-width "ho" katakana plus two hyphens.

On a Japanese keyboard in kana mode, the hyphen key is also the key for the "ho" kana.

Somehow, some key combination or key-capture race condition appears to be tripping up either WinCompose, or the IME, or both, resulting in the IME switching momentarily to half-width kana mode and direct kana input mode. Once conversion is complete, the IME kicks back into regular romaji (i.e. Latin alphabet) input mode.

Notably, halfway through writing this bug report, and in the process of researching the issue, the aberrant behavior has stopped. I can now use WinCompose as wanted and expected, without the IME interfering. The thing is, I didn't do anything. It just stopped acting funny.

Here's a new screenshot of the debug log, typing in the same CAPSLOCK + --- combo here in the browser window:

Okay, that is super freaky. I restarted WinCompose to flush the debug log for a fresh screenshot, and now the bad behavior is back.

Opening the WinCompose "Show Sequences" window and typing CAPSLOCK + --- again seems to have fixed it.

Here's a final debug log screenshot after things are now working correctly:

—

Here's hoping this helps.

[ErikAnderson3]

To provide a bit more context for the specific kana produced when this buggy behavior is happening:

• It seems that <Multi_key> is being captured by the IME, rather than by WinCompose.
• When this happens, the IME interprets the keypress as a command to change from:
  • Half-width Alphanumeric input (i.e. English), with the Input Method option set to Romaji Input (i.e. the regular ASCII values of the keys, be they QWERTY or DVORAK), to
  • Half-width Katakana input (i.e. Japanese), with the Input Method option set to Kana Input (i.e. the kana values of the keys, as if the keyboard itself is a Japanese keyboard).

For reference, here's a picture via Google Image Search of a Japanese keyboard. We can clearly see that "A" corresponds to hiragana ち, or in half-width katakana ﾁ (as reported by elyatai), while "-" (the hyphen) corresponds to hiragana ほ or half-width katakana ﾎ (as I reported).

So it seems the glyphs being produced are not mysterious and are actually the expected values, once we recognize that the IME is capturing the <Multi_key> and interpreting that as a command to change input modes.

As to why the IME is capturing the <Multi_key> press, and only sometimes at that, I have no idea.

[ErikAnderson3]

Confirmed that this issue is still present in the just-released v0.9.0. Also, although opening the "Show Sequences" window and typing in there does show the expected and wanted sequence, doing this no longer fixes the problem in any other window -- I'm still getting ﾎ-- instead of em-dash here in Chrome, even after getting — in "Show Sequences". With v0.8.2, typing in "Show Sequences" would somehow force the correct behavior in other apps, but no longer.

[samhocevar]

I deliberately released 0.9.0 before tackling this bug because I fear it will require some heavy code modifications. I'll keep you posted. Many thanks to @ErikAnderson3 for the detailed analysis!

[samhocevar]

I think I found a reasonable workaround. Only a single function (ToUnicode) was misbehaving, so I tricked it into believing it uses a standard US keyboard. Please see the release pages for a downloadable preview version that features the fix.

@ErikAnderson3 since you deliberately removed the EN-US keyboard layout from your preferences, there is a possibility that the patch does not work. If it indeed doesn’t work, can you please try to re-add that layout as a second test? Thanks!

[ErikAnderson3]

Hmm, I've been unable to reproduce the issue since rebooting on Monday (2019-03-18), due to a corporate network credentials issue -- no changes to Windows. I've noticed other issues with Windows in the past, where leaving it running over extended periods gradually leads to odder behavior. I wonder if that's some of what was happening with WinCompose?

Currently still on Windows 1709, build 16299.967. WinCompose version 0.9.0.

I'll post again if I can reproduce the odd behavior. Thank you!

[samhocevar]

Yes, it’s part of what made it so difficult to debug: the behaviour is non-deterministic and seems loosely related to the order in which the WinCompose and the IME keyboard hooks get loaded. It’s not random but it seems to depend on many unknown parameters.

[samhocevar]

WinCompose 0.9.1 was released and fixes this issue. Thanks for your patience!