AltGr key combinations are not working

[thardeck]

I have tried using AltGR key combinations in Chromium and Firefox under Linux against QEMU and they have never worked. This is quite an issue since on the Germany keyboard "|", "" and "@" are only available via AltGr.

The issue is partly described under https://github.com/kanaka/noVNC/wiki/KeyEvents .

I think I have seen a solution here in a branch some time ago were you could activate a checkbox to activate AltGr and then disable it again.

For my project I have created a solution which does add an HTML button which activates the AltGR key press for the next key entered by the user.

I am basically changing _handleKeyPress to achieve this. Is it worth creating a pull request for this and incorporating my changes properly in the noVNC code or is there a reason why this was never merged/implemented?

[jalfd]

Which version are you using?

It should work just fine in a recent version. Last summer I did a complete overhaul of the keyboard handling, because the original implementation didn't work well with non-US keyboards.

As far as I know, AltGr works fine in all major browsers on Linux and Windows. (Firefox on Linux currently has some issues with dead keys, but AltGr should work)

[thardeck]

I checked out master and connected to QEMU via websockets. It didn't work with the German AltGr keys but I didn't debug what actually was sent. The chars shown were the ones you would have gotten without pressing AltGr.

But I don't think it is just QEMU since with my manual AltGr button the keys are inserted without issues. What could be is that other VNC implementations can handle AltGr being sent as Alt + Ctrl but Qemu doesn't but, like I said, I haven't debugged the stream.

[jalfd]

Can you try opening input.html, and enter various key combinations there? (And then post the output here)

[astrand]

RFB is based on keysyms, so the VNC client should send XK_at. Probably QEMU does not interpret this correctly, if certain keys (AltGr etc) is not sent beforehand. This is a bug in QEMU.

[thardeck]

I have pressed
AltGr + q = @
AltGr + < = |
AltGr + ß = \

0: Display initialized
1: raw key event keyup (key: 116, char: 0, which: 116)
2: raw key event keydown (key: 225, char: 0, which: 225)
3: RFB keypress down keysym: 65027 key name: 65027
4: raw key event keydown (key: 81, char: 0, which: 81)
5: raw key event keypress (key: 64, char: 64, which: 64)
6: RFB keypress up keysym: 65027 key name: 65027
7: RFB keypress down keysym: 64 key name: 64
8: RFB keypress down keysym: 65027 key name: 65027
9: raw key event keyup (key: 81, char: 0, which: 81)
10: RFB keypress up keysym: 64 key name: 64
11: raw key event keyup (key: 225, char: 0, which: 225)
12: RFB keypress up keysym: 65027 key name: 65027
13: raw key event keydown (key: 225, char: 0, which: 225)
14: RFB keypress down keysym: 65027 key name: 65027
15: raw key event keydown (key: 220, char: 0, which: 220)
16: raw key event keypress (key: 124, char: 124, which: 124)
17: RFB keypress up keysym: 65027 key name: 65027
18: RFB keypress down keysym: 124 key name: 124
19: RFB keypress down keysym: 65027 key name: 65027
20: raw key event keyup (key: 220, char: 0, which: 220)
21: RFB keypress up keysym: 124 key name: 124
22: raw key event keyup (key: 225, char: 0, which: 225)
23: RFB keypress up keysym: 65027 key name: 65027
24: raw key event keydown (key: 225, char: 0, which: 225)
25: RFB keypress down keysym: 65027 key name: 65027
26: raw key event keydown (key: 219, char: 0, which: 219)
27: raw key event keypress (key: 92, char: 92, which: 92)
28: RFB keypress up keysym: 65027 key name: 65027
29: RFB keypress down keysym: 92 key name: 92
30: RFB keypress down keysym: 65027 key name: 65027
31: raw key event keyup (key: 219, char: 0, which: 219)
32: RFB keypress up keysym: 92 key name: 92
33: raw key event keyup (key: 225, char: 0, which: 225)
34: RFB keypress up keysym: 65027 key name: 65027

[thardeck]

Btw. do we now have an implementation for keyboard conversions, where we can select the current keyboard layout? The keyboard in the code was not static but I didn't find any layouts.
I have also seen a pr for this some time ago.

[astrand]

7: RFB keypress down keysym: 64 key name: 64

This is XK_at (0x40). So no problem here. Well, the debug printouts could be a little better, for example keysyms in hex. The "name" is also wrong.

Btw. do we now have an implementation for keyboard conversions, where we can select the current >keyboard layout? I have also seen a pr for this some time ago.

Should not be necessary. RFB/VNC is layout independent. Virtual machines, however, needs to have a layout option in order to translate from high level keysyms back to low level scancodes.

[thardeck]

Thx, so I am closing this and will debug the issue in QEMU when I have time.

[jalfd]

@thardeck It is possible/likely that QEMU expects Ctrl+Alt instead of AltGr. Windows generates that key combination when you press AltGr, so if the server expects that, I guess receiving AltGr instead might have unexpected effects.

[thardeck]

No, that can't be it because I have added for testing purposes a method to rfb.js to just send the AltGr key before the next key and it works fine. It is a little hacky but it did the job, even with QEMU.

// overwrite existing _handleKeyPress method to allow the usage of our AltGr
// key button
RFB.prototype._handleKeyPress = function (keysym, down) {
    if (this._view_only) { return; } // View only, skip keyboard, events
    if (this._key_down_code != 0) {
      Util.Info("Sending key code (down): " + this._key_down_code);
      this._sock.send(RFB.messages.keyEvent(this._key_down_code, 1));
    }
    this._sock.send(RFB.messages.keyEvent(keysym, down));
    if (this._key_down_code != 0) {
      Util.Info("Sending key code (up): " + this._key_down_code);
      this._sock.send(RFB.messages.keyEvent(this._key_down_code, 0));
      this._key_down_code = 0;
    }
}

RFB.prototype.holdAltGr = function () {
    if (this._rfb_state !== 'normal' || this._view_only) { return false; }
    Util.Info("Holding Alt-Gr");

    this._key_down_code = 0xFE03
}

[thardeck]

I don't think that the issue is QEMU's fault. According to the log, AltGR is not down while the at key is sent.

3: RFB keypress down keysym: 65027 key name: 65027
4: raw key event keydown (key: 81, char: 0, which: 81)
5: raw key event keypress (key: 64, char: 64, which: 64)
6: RFB keypress up keysym: 65027 key name: 65027
7: RFB keypress down keysym: 64 key name: 64
8: RFB keypress down keysym: 65027 key name: 65027

The reason why regular VNC sessions don't have an issue is since they don't have to do a later conversion and can just use the at key.

But nevertheless noVNC sends an AltGR key up while AltGR was still hold down so this is a bug.

This does also explain why my hack works, even with QEMU.

[astrand]

It's true that "most" VNC clients will let AltGr stay down while sending XK_at, but it's still a bug in QEMU if it requires AltGr to be down. For example, consider that you have a physical keyboard with "@" on a separate physical button, on the client side. In this case, you do not need to use any modifier at all in order to generate "@" = XK_at. Over VNC, XK_at is the only thing that will be sent. Then, it's up to QEMU to do any "fake" modifier keypress, it required by the keyboard map configured on the server side.

[jalfd]

That's intended behavior. Logically, what happens on the client is that the @ key is pressed.

The alternative would be to either:

• tell the server that AltGr + q is pressed (which the server can do nothing with, unless it knows the client's keyboard layout, which it doesn't. So it won't know to translate this to @), or
• tell the server that AltGr + @ is pressed, which is just incorrect. @ was generated as a result of AltGr + q being pressed. But AltGr + @ was not pressed (that key combination may have a different meaning on a keyboard where @ can be generated without using AltGr).

NoVNC pretends to release AltGr while sending the @ to get around this. Other VNC clients I've tested do the same thing, because all the alternatives would yield incorrect behavior.

At the root, the problem is that the RFB protocol doesn't distinguish between what the HTML DOM calls keyup/keydown (physical keys being pressed) and keypress (the character that is generated as a result of the keypresses). There is no way to communicate to the server that "AltGr was pressed and then Q was pressed, and this created the character @". The current behavior is the closest we can come to correct behavior.

I believe there's even a note about this in the RFB spec (although I don't have it at hand here and now to check). It is the server's responsiiblity to fake modifiers as needed.

[thardeck]

What would you think about an option in noVNC to enforce the behavior of sending the keydown key with the generated one?

[astrand]

I have never heard of any other VNC client having such a feature, so it seems kind of exotic to me. Why not try to fix the real problem instead - fix QEMU. Have you created a bug report in that project?

[thardeck]

Yes, it would be better to fix this in QEMU but it is not as easy I suppose. I haven't created a bug report yet but I talked to some QEMU developers.