Key mapping issue with AZERTY keyboard

[pidobeuliou]

• I have read the FAQ.
• I have searched in existing issues.

Environment

• OS: [Windows]
• scrcpy version: [from 1.15]
• installation method: [Windows release]
• device model: XIAOMI Mi9T, XIAOMI Redmi Note 10
• Android version: [at least 10 and 11]

Describe the bug
Up to scrcpy-win64-v1.14, the keyboard mapping with AZERTY keyboard is perfect. All characters are displayed correcly on the smartphone, even with accents.
From scrcpy-win64-v1.15 up to the last version scrcpy-win64-v2.0, the keyboard mapping with AZERTY keyoard presents the following bug:
The keys 1, 2, 3, 3 (upper keys above the first line of characters azerty) are not well mapped. The displayed characters on the Smartphone correspond to the QWERTY keyboard: ! in place of 1, @ in place of 2, # in place of 3, etc.
Some others keys are also not correctly mapped: > in place of "."

[rom1v]

Here is the changelog of 1.15: https://github.com/Genymobile/scrcpy/releases/tag/v1.15

The difference is probably that Shift is forwarded to the device.

I use an AZERTY keyboard both on my computer and on the device, it works perfectly for me 😕

What is your computer keyboard layout? What is your device keyboard app? Does the problem also occur with another device keyboard app?

Run scrcpy --no-audio -Vverbose, and:

• press Shift
• press 1
• release 1
• release Shift

What is the output in your console?

Here is my result:

VERBOSE: input: key down code=59 repeat=0 meta=000041
VERBOSE: input: text "1"
VERBOSE: input: key up   code=59 repeat=0 meta=000000

[pidobeuliou]

Hi,
Thank you for your answer.
What is your computer keyboard layout? => FRA: French (France) French keyboard
What is your device keyboard app? => Microsoft SwiftKey 9.10.14.17 (last available version)
Does the problem also occur with another device keyboard app? => You found the issue! With Gboard keyboard, no problem. The mapping is correct

With Verbose, here is my result:
VERBOSE: input: key down code=59 repeat=3 meta=200041
VERBOSE: input: text "1"
VERBOSE: input: key up code=59 repeat=0 meta=200000

How do we explain that with scrcpy version lower than 1.15, the mapping is correct even with Microsoft SwiftKey keyboard app?

[rom1v]

With Gboard keyboard, no problem. The mapping is correct

👍

With Verbose, here is my result:

VERBOSE: input: key down code=59 repeat=3 meta=200041
VERBOSE: input: text "1"
VERBOSE: input: key up code=59 repeat=0 meta=200000

That is correct.

There is a confusion with the Shift key by your keyboard app the way scrcpy is injecting events.

How do we explain that with scrcpy version lower than 1.15, the mapping is correct even with Microsoft SwiftKey keyboard app?

Because before v1.15, scrcpy did not forward Shift to the device, so this did not trigger the issue.

I added a log:

diff

diff --git a/server/src/main/java/com/genymobile/scrcpy/Controller.java b/server/src/main/java/com/genymobile/scrcpy/Controller.java
index 59fae6024..1cae937b4 100644
--- a/server/src/main/java/com/genymobile/scrcpy/Controller.java
+++ b/server/src/main/java/com/genymobile/scrcpy/Controller.java
@@ -192,6 +192,7 @@ public class Controller implements AsyncProcessor {
             return false;
         }
         for (KeyEvent event : events) {
+            Ln.i("=== " + event);
             if (!device.injectEvent(event, Device.INJECT_MODE_ASYNC)) {
                 return false;
             }

Could you please replace this binary in the scrcpy v2.0 folder:

• scrcpy-server SHA-256: acab58397a246952ad17153268cceb3115718c083aeba403e5f79bfcc53c2e7c

Run scrcpy --no-audio -Vverbose with your custom keyboard app, and type 1, then post the console output.
Do the same with the Gboard keyboard.

[pidobeuliou]

I have replaced your binary scrcpy-server in my scrcpy v2.0 folder.

Here is the result:
With Swiftkey keyboard app: I press shift 1, but the screen displays "!"
VERBOSE: input: key up code=59 repeat=0 meta=200000
VERBOSE: input: key down code=59 repeat=0 meta=200041
VERBOSE: input: text "1"
[server] INFO: === KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_1, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=1090023359, downTime=1090023359, deviceId=-1, source=0x101, displayId=-1 }
[server] INFO: === KeyEvent { action=ACTION_UP, keyCode=KEYCODE_1, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=1090023359, downTime=1090023359, deviceId=-1, source=0x101, displayId=-1 }
VERBOSE: input: key up code=59 repeat=0 meta=200000

With Gboard keyboard app: I press shift 1, and the screen displays "1"
VERBOSE: input: key up code=59 repeat=0 meta=200000
VERBOSE: input: key down code=59 repeat=0 meta=200041
VERBOSE: input: text "1"
[server] INFO: === KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_1, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=1090113162, downTime=1090113162, deviceId=-1, source=0x101, displayId=-1 }
[server] INFO: === KeyEvent { action=ACTION_UP, keyCode=KEYCODE_1, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=1090113162, downTime=1090113162, deviceId=-1, source=0x101, displayId=-1 }
VERBOSE: input: key up code=59 repeat=0 meta=200000

I don't see a lot of differences between the 2 [server] INFO...

[pidobeuliou]

For information, I tried other keyboard applications such as "Typewise", "Yandex Keyboard", "Bobble Keyboard", and they work correclty. Pressing Shift 1, displays "1".
The issue seems to be only with Microsoft Swiftkey...

[rom1v]

OK, so probably Microsoft Switfkey incorrectly interpret the shift state when it should only consider the keycode :/

[pidobeuliou]

OK, so probably Microsoft Switfkey incorrectly interpret the shift state when it should only consider the keycode :/

Should-it be possible to run scrcpy with an option to not forward "Shift" to the device?

[rom1v]

Shift is useful for text selection for example (Shift+left/right).

That would add another option for shift (and maybe yet other options for other keys), just because one keyboard app does not handle it correctly :/ Maybe it should be reported to that keyboard app so that it is fixed on their side?

[thefaboss]

Hi there, thx a lot for your great app :)

I've got the exact same issue than pidobeuliou with Swiftkey.
I vote up for a command line option to disable shift key if this is really the issue's origin.

Like pidobeuliou said in his description, there may be an confusion between azerty and qwerty keyboard... I don't understand why shift key would be the cause...?