Allow keyboard infomation override for xrdp connections

[TOMATO-ONE]

Hi all

This discussion is similar to Discussion #1933 and PR #1934
But this one is for xrdp itself.

I am open to suggestions.

I'm sure there have been many xrdp issues reported relating to keyboards.
This is a user interface issue that has a serious impact.

The difficulty in fixing this issues is that there are many different keyboards and keyboard layouts for many different languages in the world.
The issue reporter and the developer do not necessarily have the same type of keyboard.
The same is true for languages.

In the course of PR #1934 we added an option to override the keyboard information sent to the remote in NeutorioRDP Proxy mode.
I propose to add a similar option to the xrdp itself.

The ability to set the keyboard information to an arbitrary value would be useful for debugging and troubleshooting.

What do you think?

[TOMATO-ONE]

I think we need to discuss and implement this in a few steps.
Here's what I have in mind

• 1st one: add an option to force the Client to overwrite any keyboard information sent to it with the specified value for debugging purposes.
• 2nd one: Allow to set other than 0x00000409 (English-US) as xrdp default key layout when keyLayout value is 0x00.
• 3rd one : Special support for Japanese keyboard.
• 4th one: Special support for various RDP clients.
• 5th one: add keyboard information to be overwritten in each xrdp.ini section.

For the first and third one I have the draft code.
The second one I haven't coded yet. But I know where to put it.
The fourth problem is how to identify the RDP client.
The fifth I have not been able to demonstrate. I don't even know if it's possible.

[matt335672]

I agree the 4th and 5th are problematic.

• The RDP protocol does not allow for client identification in any way that I'm aware of - it's a capability-based exchange. That gives you a challenge for no. 4.
• I can't see at the moment how no. 5 would interact with the login screen.

I'd also like to see clearer debug info on where the keyboard is being selected from. Are we taking the client selection, or are we defaulting on receiving keyboadLayout=0x0? This should be pretty quick to implement and will help users with diagnosing issues.

[TOMATO-ONE]

thanks.@matt335672
I've put the fourth and fifth in a separate thread.

I agree about the logs.
I will incorporate it into the first implementation.

[TOMATO-ONE]

The intention of each step is supplementary.

1st one:
Overwrite the keyboard information in the xrdp connection with the values in the xrdp.ini file.
This makes the keyboard information fixed for the whole rdp system.

This removes versatility from the system, but we think it is useful for debug purposes.
You can set any keyboard information and see how the system works.

I also believe that xrdp can be used as a work-around when used on personal Linux systems.

[matt335672]

Agreed - for debugging, removing versatility is fine.

Can you explain your last sentence a little more please? I'm not understanding it.

[TOMATO-ONE]

I am assuming that only one user will be using the system, for example a Linux system on Virtualbox or WSL.
In such cases, if keyboard layout problems occur, we believe that fixing the keyboard layout information on the xrdp side will not be a problem.

[TOMATO-ONE]

I have proposed PR #1952 for this Step.

I've also added code for the following points.

I'd also like to see clearer debug info on where the keyboard is being selected from. Are we taking the client selection, or are we defaulting on receiving keyboadLayout=0x0? This should be pretty quick to implement and will help users with diagnosing issues.

[TOMATO-ONE]

I have implemented Draft PR #1952 and tested it with a variety of RDP clients.

Overriding the keyboard layout worked exactly as I wanted.
However, I have found that in some cases it does not work.

Here are some interesting results

• Pattern 1: RDP client is sending keyboard layout 0x0000000
  Overriding the keyboard layout does work.
  This is the case with the Microsoft Remote desktop client on macOS Bigsir.

• Pattern 2: The xrdp client is misinterpreting the keyboard layout
  Overriding the keyboard layout does work.
  This is mstsc.exe on Japanese Windows with a US keyboard connected.

• Pattern 3: The RDP client is sending a different keyboard layout value than the actual one.
  It doesn't work the way I want it to.
  To be precise, it overwrites the wrong layout with another layout, resulting in a strange key layout that doesn't match any language.
  This is the case with Microsoft Remote Desktop client 10 on a Chromebook with a Japanese keyboard.

[TOMATO-ONE]

2nd one:
Some RDP clients send the keyboard information as 0x00.
This is meant to use the default setting on the RDP server side, but currently the default value for xrdp is fixed at 0x00000409.
This is not a big problem for most people in the world, but it can be a problem in some environments, such as Japanese.
I would like to be able to set an arbitrary default value in xrdp.ini.

[matt335672]

This should definitely be configurable.

[TOMATO-ONE]

Thank you.

I will create a draft code.

[TOMATO-ONE]

I've followed the code. I studied it.

As a result, I realized that there is a place to define it in xrdp_keyboard.ini.
I haven't tried it enough, but this second step may not be necessary to implement.

xrdp/xrdp/xrdp_keyboard.ini

Lines 42 to 52 in 34e56f3

	[default]
	; keyboard_type and keyboard_subtype is not read for default section. It
	; is only a placeholder to keep consistency. Default model/variant are
	; platform dependent, and could be overridden if needed.
	keyboard_type=0
	keyboard_subtype=0
	; user could override variant and model, but generally they should be inferred
	; automatically based on keyboard type and subtype
	;variant=
	;model=

[TOMATO-ONE]

3rd one:
This is based on the discussion in PR #1934.
KeyLayout information can be undocumented on Windows in a Japanese environment.
This is not a problem if the RDP server is windows, but requires special handling for xrdp.

This feature is already implemented in xrdp.
It works fine on Japanese keyboards (PC106/109) with kbd_'type:0x07'.

However, if you are using Japanese on an English keyboard (PC101/102) with 'kbd_type:0x04', the current code will give a false positive.
As a workaround, you can switch the "language" and then connect, and @metalefty also recommends that method.

I find this operation unfamiliar to the average Japanese Windows user.
I would like to implement a special support for this.

[matt335672]

You might also want to look at the logic around the xrdp_keyboard.ini file. I've only just discovered this little corner so I can't say I understand it much. I notice however that there's a rdp_keyboard_jpwhich is triggered on keyboard_type 7. It might be you can add a similar section for keyboard_type 4.

I'm agreed the language dialog is unfamiliar to many users. As a UK English user, I only found it after the discussion I had you and @metalefty. One simple workaround might be to document this on the xrdp wiki.

[TOMATO-ONE]

Thank you.

I'll consider the code as you advise.

[TOMATO-ONE]

As for the fourth.
@matt335672 I certainly don't think there is any way to fully identify a client.

I know it's incomplete, but you can use client_info->version or client_info->build as a criteria.

The RDP Client on iOS/Android/macOS changes versions frequently.
We don't think it's possible to include special support for each in the xrdp code.

I thought of a similar mechanism to xrdp_keyboard.ini.
Read in a user defined file for the conversion and
The idea is to overwrite the keyboard layout information for a specific version of the value.

Such a process is a big challenge for me as a C beginner.

[TOMATO-ONE]

I'll explain the purpose of this step in a little more detail.

I have a lot of devices.

• MacBook Air (11-inch, Early 2015) Japanese Keyboard
• iPad mini4 with bluetooth keyboard (US)
• HUAWEI Android 7.0 Tablet
• ASUS Chromebook (Japanese Keyboard)
• CHUWI 2-in-1 Windows10 tablet with US Keyboard

I have been testing these devices and various RDP client software to connect to xrdp.

The keyboard information sent by the RDP Client is device and application specific. The values may change when the version of the application changes.

I have come to believe that it is very difficult to cover all of this with the automatic determination of xrdp.

Although the information sent by each RDP client does not fully identify the device and application, the value of client_info->build seems to be different for each.
I am thinking that this might be used to make special support for RDP Clients with specific values.

[TOMATO-ONE]

In the process of testing the first one, I felt that this part of the implementation was unnecessary.
It doesn't always give the results you want, and it makes the system more complicated.

[TOMATO-ONE]

As 5th one:
@matt335672 you are right.

The login screen is shown first, so the keyboard info is set then.
I don't know if it is possible to change the keyboard info when switching sections.

I'm trying to create some test code to try it out, but I haven't managed to create the code itself yet.

[TOMATO-ONE]

I'll explain the purpose of this step in a little more detail.

I was thinking of a way to let the user choose their own keyboard information if it is not possible to automatically determine it.
Specifically, I wanted to allow the user to toggle the keyboard information when switching between sections on the login screen.

For example, we could write a section in the xrdp.ini file that looks like this

[xorg]
name=xorg

[xorg-JP]
name=xorg-JP-Keyboard
xrdp.override_keyboard_type=0x07
xrdp.override_keyboard_subtype=0x02
xrdp.override_keylayout=0x00000411

[matt335672]

Hi @TOMATO-ONE

That could work, but you'd end up with a lot of sections. Each time a new keyboard type was added you'd need to duplicate all of your sections to add the new type.

I can't hep feeling adding some kind of support for keyboard selection on the login screen itself would be a better idea.

[TOMATO-ONE]

Hi @matt335672 Thanks for your comment.
Thanks for your comment.
I don't think it's a good idea to create a separate section for each language either.

This plan of mine was intended to be used as a work-around in a multi-user environment.
Basically, it uses automatic key layout recognition by xrdp.
Most users should have no problem with this.
Only some RDP clients with key layout mismatch problems will be asked to log in using a dedicated section.

I also think that keyboard layout selection on the login screen is a good idea.
However, a GUI needs to be implemented. It requires more man hours and skills.
If we can override the keyboard layout for each section, it will be easier to implement in the future.

[matt335672]

I can see where you're coming from, but once we put something in the ini file it becomes hard to remove, as users will start to depend on it.

For the case you mention above, can the same effect be achieved by editing xrdp_keyboard.ini ?