JIS (Japan Industrial Standards) key mapping issues (mac -> win)

[kmk-matsui]

Bug Description

The conversion key and non-conversion key on the JIS keyboard are not working.

How to Reproduce

When inputting the English and Japanese keys from a Mac, the non-conversion key and conversion key are recognized on Windows.

Expected Behavior

When keydown the Eisu（英数） key on a Mac, the non-conversion（無変換） key are recognized on Windows.
When keydown the Kana（かな） key on a Mac, the conversion（変換） key are recognized on Windows.

Operating system(s) on local side and remote side

MacOS(Ventura 13.4.1(c)) -> Windows 11

RustDesk Version(s) on local side and remote side

1.2.1 -> 1.2.1

Screenshots

Reference: Wikipedia - Language input keys

Additional Context

No response

[rustdesk]

@fufesou

[fufesou]

@kmk-matsui Hi, thanks for your feedback.

But I can hardly support this issue. The inputs across OS is based on the map of "key-code".

I've tested to grab the key codes on my macOS. The "converion key" and "non-convertion key" can not be grabbed, no codes are shown.

I don't know if it's because my macOS is not targeted at Japanese users.

Can you please run the attached process to test the key codes ?

You can also use the code here if u use rust. https://github.com/fufesou/rdev/blob/master/examples/grab.rs

Just download the attached file, and then

tar -zxf grab.tar.gz && ./grab

And then press the keyboard.

It will print somethings like:

It would be very grateful if you could run all the corresponding keys on your keyboard on your computer and post the key-code map here.

grab.tar.gz

[fufesou]

The picture below is the soft keyboard layout of my macOS.

[kmk-matsui]

@fufesou

I think the non-conversion key and the conversion key on Windows correspond to the 英数(eisu) key and the かな(kana) key on MacOS.

press a key "non-convertion key"(英数キー)

name: Some(UnicodeInfo { name: Some(" "), unicode: [32], is_dead: false }), type: KeyPress(Lang2), code: 0x66, scan: 0x0000
name: None, type: KeyRelease(Lang2), code: 0x66, scan: 0x0000

press a key "converion key" (かなキー)

name: Some(UnicodeInfo { name: Some(" "), unicode: [32], is_dead: false }), type: KeyPress(Lang1), code: 0x68, scan: 0x0000
name: None, type: KeyRelease(Lang1), code: 0x68, scan: 0x0000

[fufesou]

@kmk-matsui

The soft keyboard cannot be grabbed by the process.

I've plugged in a physical JIT keyboard, but the "non-convertion key" and "converion key" can not be grabbed.

Thanks for the quick and useful reply. I'll add the key-code map later.

[rustdesk]

The fix is merged, please test with the nightly build. https://github.com/rustdesk/rustdesk/releases/nightly.

[kmk-matsui]

@fufesou
I installed the nightly version on both macOS and Windows.
And then I tested the "non-convertion key" and "converion key" keys on the website (https://anysweb.co.jp/keycheck/#google_vignette) using Windows, but they were not recognized.

[kmk-matsui]

@fufesou
Operating system(s) on local side and remote side: Windows10 -> Windows11
In the above case the "non-convertion key" and "converion key" keys were recognized.
version 1.2.1 on local side is not nightly version.

[fufesou]

There's no "key-code" map between the same OSs.

I'll try to find the map later by another test method.

[fufesou]

@kmk-matsui Hi, I've tried mac -> win.

My mac cannot detect key event of "Non-Convert" and "Convert". I'll try to find the reason.
It's strange that the "grab" process works on your computer.

I can send Non-Convert and Convert from my mac computer to windows computer by exchanging the mapping of "A" with "Non-Convert" and "B" with "Convert". It works fine.

[kmk-matsui]

@fufesou
On Mac, the keys "Non-Convert" and "Convert" are not recognized. Instead, there are keys labeled "Eisu" (英数) and "Kana" (かな). These keys are used to switch between input methods.
On Windows, the keys labeled "Eisu" (英数) and "Kana" (かな) correspond to the "Non-Convert"" and "Convert" keys.
The key codes for "Eisu" (英数) and "Kana" (かな) keys are different from those of "Non-Convert" and "Convert" keys on Windows.

[fufesou]

Thanks.
But I can't detect keypress events on either side of the spacebar. 🙃
I'll try to find out why they cannot be detected later.

[My1]

@kmk-matsui Hi, I've tried mac -> win.

My mac cannot detect key event of "Non-Convert" and "Convert". I'll try to find the reason. It's strange that the "grab" process works on your computer.

likely because "convert" and "nonconvert" arent present on on mac keyboards, and Mac maybe doesnt know what these even are and ignores them.

I can send Non-Convert and Convert from my mac computer to windows computer by exchanging the mapping of "A" with "Non-Convert" and "B" with "Convert". It works fine.

you mean by exchanging the mapping on windows, or in rustdesk, or where?

---

it seems the Keymap handling is similar or the same as the W3C KeyEvent.Code
https://www.w3.org/TR/uievents-code/#key-alphanumeric-functional

since mac uses a different set of keys (kana/Eisu - which is alphanumeric, which map as Lang1 and Lang2 events) next to the spacebar, while windows has an Eisu on the Caps Lock Key (which doesnt seem to have its own function code but just seems to call "CapsLock")

or how do the keys compare to each other and what would be the best option to send between OSes?

also does Mac Handle the Japanese keys on non-apple keyboards at all and if yes, how so?

so ideally would be maybe a filling of this table.

from mac to windows

Key on Mac	send to JIS (ideal)	send to JIS (current)
かな (kana)	convert	nothing
英数 (Eisu)	nonconvert	nothing

is what I grasp from OP

Key on JIS	send to Mac (ideal)	send to Mac (current)
英数(Eisu/Caps Lock)
カタカナ / ひらがな / ローマ字(Hira/Kata/Romaji)
変換 (henkan/convert)
無変換(muhenkan/nonconvert)
半角 / 全角 / 漢字 (han/zenkaku/Kanji - Half/Full Width/Kanji)

[My1]

@kmk-matsui can you confirm I have the state of how it should be and is for mac -> JIS keyboard in the above comment correct? (and maybe just in case someone cares think about the JIS -> Mac Section)

[fufesou]

I can send Non-Convert and Convert from my mac computer to windows computer by exchanging the mapping of "A" with "Non-Convert" and "B" with "Convert". It works fine.

you mean by exchanging the mapping on windows, or in rustdesk, or where?

I changed rustdesk.
On the mac side, when I pressing "A", I send the "Non-Convert" key event to the remote side.

I mean, the key mappings from mac to win are right. But I cannot detect the key events on my mac.

[My1]

I can send Non-Convert and Convert from my mac computer to windows computer by exchanging the mapping of "A" with "Non-Convert" and "B" with "Convert". It works fine.

you mean by exchanging the mapping on windows, or in rustdesk, or where?

I changed rustdesk. On the mac side, when I pressing "A", I send the "Non-Convert" key event to the remote side.

I mean, the key mappings from mac to win are right. But I cannot detect the key events on my mac.

meaning you dont get to see the Lang1 and Lang2 when pressing the keys on a mac keyboard? or are you using a standard JIS kb on a mac?
because ideally if you can see the Lang1 and Lan2 events you should be able to tell rust desk, that if it gets an incoming Lang1/2 and the remote side is not running on Apple, that the remote rustdesk understands Convert/nonconvert

[fufesou]

meaning you dont get to see the Lang1 and Lang2 when pressing the keys on a mac keyboard? or are you using a standard JIS kb on a mac?

Yes, there are no key events when I pressing or releasing the keys aside to the space key.

It may be caused by the system of my macOS, or the implementation of detecting the keys. Windows can detect the keys.

[rustdesk]

Does AnyDesk work?

[fufesou]

I've tried AnyDesk (mac -> Win, Map 1:1 mode), the keys aside the space also do not send key events.

[fufesou]

But the keys can be detected #5027 (comment) by @kmk-matsui

I've already added the code mappings. But I cannot understand why there's still an issue here.

On Mac, the keys "Non-Convert" and "Convert" are not recognized. Instead, there are keys labeled "Eisu" (英数) and "Kana" (かな). These keys are used to switch between input methods.
On Windows, the keys labeled "Eisu" (英数) and "Kana" (かな) correspond to the "Non-Convert"" and "Convert" keys.
The key codes for "Eisu" (英数) and "Kana" (かな) keys are different from those of "Non-Convert" and "Convert" keys on Windows.

RustDesk uses the position code mapping, the keys' function will not affect the mapping.

[rustdesk]

I've tried AnyDesk (mac -> Win, Map 1:1 mode), the keys aside the space also do not send key events.

If so, should we continue to spend time on an unsolvable problem?

[My1]

not sure if it's unsolvable or anydesk didnt have enough japanese customers needing this (if at all)

#5027 (comment)
the part that is interesting tbh is how this key grabber can see the keys, but not rustdesk.

[rustdesk]

tracked by #4367 to mark as low priority, hope Japanese dev can work on this.

[rustdesk]

anydesk didnt have enough japanese customers needing this (if at all)

I don't think any commercial company would like give up a big market like Japan.

[fufesou]

not sure if it's unsolvable or anydesk didnt have enough japanese customers needing this (if at all)

#5027 (comment) the part that is interesting tbh is how this key grabber can see the keys, but not rustdesk.

It is grabbed by the demo app #5027 (comment) , which is also used by RustDesk.

[My1]

anydesk didnt have enough japanese customers needing this (if at all)

I don't think any commercial company would like give up a big market like Japan.

I wouldnt say give up, but just that some things may not be seen unless someone asks and it is a bit specific as you need to both be japanese and go mac -> win.

[My1]

aside from being weird af, one thing that might need to be checked is how this works in mac to mac connections, I mean if the local mac's rustdesk doesnt even see the Lang1/2 keys in the first place it couldnt send the keys to the remote mac either.

[kmk-matsui]

Similarly, Lang1 and Lang2 were not recognized by chrome remote desktop, so I mapped F11 and F12 to Lang1 and Lang2 in karabiner and assigned the non-conversion and conversion keys to F11 and F12 in the chrome remote desktop keymap.

[kmk-matsui]

If rundesk has a keymap assignment function like chrome remote desktop, it would be possible to support the above.

[rustdesk]

We do want to add keymap, though not high priority.

[fufesou]

If rundesk has a keymap assignment function like chrome remote desktop, it would be possible to support the above.

But the real problem is that Lang1/2 keys are recognized on your computer and then there is something wrong with the mapping.

Maybe this issue could also be solved later sometime, not now.

[rustdesk]

on your computer

So this issue has to be left to be solved by an Japanese dev, since we do not have the keyboard and super low efficiency.

[My1]

or at the very least a dev/hacker with access to a mac and a japanese apple keyboard.