Modifier key events have their modifier bits set differently on Linux vs OSX

[neurocyte]

Edited by @kovidgoyal to add the poll:

We are discussing the best approach to take for fixing this issue, the two options are below with updated votes.

1. The modifier bits should reflect the modifier state taking into account the current key event. This is the platform behavior of macOS , Windows and Qt ( @neurocyte @j4james @dankamongmen @kovidgoyal @rockorager )
2. The modifier bits should reflect the modifier state as it was before the current key event. This is the platform behavior of Linux and GTK on Linux (havent been able to test GTK on other platforms). (@kchibisov @mitchellh)

(1) requires no changes to the existing kitty keyboard protocol beyond specifying the behavior. (2) requires adding a new "fake key code" and sending these events to update the modifier state.

Original issue is below:

On linux the modifier bit is clear on press events and set on release events. OSX is the other way around.

For example:

Linux:
LEFT_CONTROL PRESS
CSI 57442 u

ctrl+LEFT_CONTROL RELEASE
CSI 57442 ; 5 : 3 u

macOS:
ctrl+LEFT_CONTROL PRESS
CSI 57442 ; 5 u

LEFT_CONTROL RELEASE
CSI 57442 ; 1 : 3 u

This comes from a discussion with ghostty dev on which behaviour should be considered correct.

See: https://github.com/mitchellh/ghostty/issues/1082

[kchibisov]

Option 1 without modifier for release I'd consider as compromise. At least firefox works like that on when you try event tester where they have shift attached for press, but never for release, even when you have multiple pressed. Maybe it's a firefox's Wayland issue though, but that's what I get:

This behavior can be done when your stack is doing 2.

[ngortheone]

@kovidgoyal not a terminal app, but all Jetbrain's IDE's have Shift+Shift shortcut for Search Everywhere feature
https://blog.jetbrains.com/idea/2020/05/when-the-shift-hits-the-fan-search-everywhere/

EDIT:
And it is pretty popular feature. It is not crazy to think that some other app might want to mimic that.

[kovidgoyal]

On Fri, Dec 15, 2023 at 08:37:43AM -0800, Ihor Antonov wrote: @kovidgoyal not a terminal app, but all Jetbrain's IDE's have Shift+Shift shortcut for Search Everywhere feature https://blog.jetbrains.com/idea/2020/05/when-the-shift-hits-the-fan-search-everywhere/

And have you ever held down shift then switched to the IDE and then pressed shift again?

[ngortheone]

@kovidgoyal you asked for an example:

Still waiting for an actual example where you have bound shift+shift to do something in some application

So here you have it. I am sure there are plenty more if you care to open your mind and look around.

And have you ever held down shift then switched to the IDE and then pressed shift again?

This is not a fair question.

• First of all, you are trying to appeal to me personally, but I can't speak for the majority of users. A poll "What does the user expect to happen" would be more appropriate.
• Secondly, you are trying to re-enforce your case by leaning on the default broken behavior on some commercial platforms (OSX, Windows, and Qt which primarily caters to the first 2). The fact that these platforms behave that way does not mean it is the right way. We are doing opensource because we want to make software the right way, don't we? What you are suggesting is to accept badness that commercial platforms happened to have and propagate it to open source software. To me this goes against the spirit of opensource software. We can and should do better.
• Third, the way your question is phrased suggests that if users are not doing this today - they shouldn't do it in the future, which ridiculous. Perhaps a mirror will help here: "have you ever used ESC in your terminal as a hot key or a as part of key combination? Wat? It doesn't work? Well, it means you shouldn't because some iterm or cmd.exe doesn't support it, and we decided it is for your own good"

Last, but not least, I'd like to question the very fact of your stewardship of the protocol.

• You have demonstrated numerous times unprompted disrespect to other participants in this and related threads.
• You have also expressed unfair and groundless critics of other platforms
• You've unilaterally made changes to the protocol without discussing them with the community.
• You continue pushing for option 1 even after realizing and admitting that it is technically impossible to implement on wayland.
• You are trying to create a cross-platform protocol but actively discriminate against one major platform.

I don't think your judgement is sound and you are not acting in the interest of the entire community.

It's patently absurd to expect me to do all the work of creating the spec and then on top of that all the work for notifying your lazy ass

As a protocol steward this is exactly what you should do.

P.S. I will just aggregate some of your comments here:

...to put up with your crazy and badly designed protocol...
... And note that this protocol does not support sticky keys which is a linux only abomination....
... It's patently absurd to expect me to do all the work of creating the spec and then on top of that all the work for notifying your lazy ass.
...Tomorrow, when Linux people get bored of Wayland and invert FooBar display server that decides to implement keyboard events in some other fashion...

[gpanders]

@kovidgoyal not a terminal app, but all Jetbrain's IDE's have Shift+Shift shortcut for Search Everywhere feature https://blog.jetbrains.com/idea/2020/05/when-the-shift-hits-the-fan-search-everywhere/

EDIT: And it is pretty popular feature. It is not crazy to think that some other app might want to mimic that.

It looks like the JetBrains shortcut is pressing Shift twice in quick succession, which is different than what's being discussed in this issue (pressing both Shift keys simultaneously).

---

From reading the comments in this thread it seems to me that the division between options 1 and 2 is mostly to do with whether one cares more about key presses or key releases. Both options result in ambiguity in one or the other case (but only when dealing with the admittedly esoteric scenario of detecting simultaneous modifier keys).

With Option 1, there is ambiguity in the key press:

1. Shift key + Shift modifier  <-- First shift key pressed
2. Shift key + Shift modifier  <-- Second shift key pressed
3. Shift key + Shift modifier  <-- First shift key released
4. Shift key + No modifiers    <-- Second shift key released

Note that (1) and (2) are identical.

With Option 2, the situation is reversed:

1. Shift key + No modifiers    <-- First shift key pressed
2. Shift key + Shift modifier  <-- Second shift key pressed
3. Shift key + Shift modifier  <-- First shift key released
4. Shift key + Shift modifier  <-- Second shift key released

Now (3) and (4) are identical.

Now completely disregarding any issues about actual OS implementations or display protocols (which I admit I have essentially no knowledge of), could a 3rd option be a compromise between Options 1 and 2 where the modifier bits are set differently for key presses and key releases?

Thus for key press events we employ option 2 (modifier bits reflect the modifier state as it was before the current key event) and for key release events use option 1 (modifier bits reflect the modifier state taking into account the current key event).

[kovidgoyal]

On Fri, Dec 15, 2023 at 11:02:44AM -0800, Ihor Antonov wrote: @kovidgoyal you asked for an example: > Still waiting for an actual example where you have bound shift+shift to do something in some application So here you have it. I am sure there are plenty more if you care to open your mind and around.

Except, that your example was not an actual example, as pointed out in the previous post. Which once again goes to the obvious point, that this is a made up use case trying to put lipstick on the pig of this broken ass protocol. And if you open YOUR mind for even half a second to the possibility that the Linux keyboard protocol is actually poorly designed tripe, not a sacred cow, you will realize that tracking global modifier state is completely orthogonal to (1) or (2). It requires separate *events* for updating modifier state *independent* of key events. As such the choice of (1) vs (2) *which is what is under discussion here* is completely unaffected by the issue of tracking global modifier state of which latched keys is just a subset. If someday someone actually comes up with alegitimate use case for tracking global modifier state, it can be added to the protocol regardless of the decision made here. The Linux protocol is simply bad in a networked context, introducing arbitrary latency between a key event and the change in modifier state it causes is bad, a priori. Deal with it.

> And have you ever held down shift then switched to the IDE and then pressed shift again? This is not a fair question. - First of all, you are trying to appeal to me personally, but I can't speak for the majority of users. A poll "What does the user expect to happen" would be more appropriate.

What crock, find me *one* user and *one* use case that does this. It doesnt have to be YOU personally.

- Secondly, you are trying to re-enforce your case by leaning on the default broken behavior on some commercial platforms (OSX, Windows, and Qt which primarily caters to the first 2). The fact that these platforms behave that way does not mean it is the right way. We are doing opensource because we want to make software the right way, don't we? What you are suggesting is to accept badness that commercial platforms happened to have and propagate it to open source software. To me this goes against the spirit of opensource software. We can and should do better.

I agree wholeheartedly and the badness is in the open source software this time.

- Third, the way your question is phrased suggests that if users are not doing this today - they shouldn't do it in the future, which ridiculous. Perhaps a mirror will help here: "have you ever used ESC in your terminal as a hot key or a as part of key combination? Wat? It doesn't work? Well, it means you shouldn't because some xterm or cmd.exe doesn't support it, and we decided it is for your own good"

I see so now we have to design a protocol that caters to every imaginable future use case including made up ones at the cost of making current actual existing use cases WORSE.

Last, but not least, I'd like to question the very fact of your stewardship of the protocol.

Go do some actual work and make your own protocol in that case.

- You have demonstrated numerous times unprompted disrespect to other participants in this and related threads.

Unprompted! @kchibisov started this whole waste of time by calling me unprofessional. And you are now following up with more personal attacks.

- You have also expressed unfair and groundless critics of other platforms

Except they are extremely fair and fully grounded.

- You've unilaterally made changes to the protocol without discussing them with the community.

Go read the hundred plus comments in this thread before making unfair and groundless personal attacks

- You continue pushing for option 1 even after realizing and admitting that it is technically impossible to implement on wayland.

Yup, the fix for wayland's broken design is to fix waylanf not force its mistakes on everybody else.

- You are trying to create a cross-platform protocol but actively discriminate against one major platforms.

Yup, that *minor* platform got it wrong and should fix its mistake.

I don't think your judgement is sound and you are not acting in the interest of the entire community. > It's patently absurd to expect me to do all the work of creating the spec and then on top of that all the work for notifying your lazy ass As a protocol steward this is *exactly* what you should do.

Says you. And let me respond with this, you continue you personal attacks on me one more time, you will be banned from this repo. Good bye and good luck.

[kovidgoyal]

@gpanders the problem is modifiers are treated as state in all the underlying platforms, independent of which key if any caused the state change.

Treating them differently in this protocol would basically be adding friction everywhere. The one outstanding issue here is tracking global modifier state changes independent of key events. No one has yet come up with a use case for this, but even if they did, the choice of (1) vs (2) is irrelevant to it, as it requires separate events that just set modifier state independent of key events. With those separate events both (1) and (2) would work fine.

Meanwhile (2) has the problem that in a networked context it introduces arbitrary latency between a key event and notification of the corresponding modifier state change. (1) does not suffer from this issue.

The issue with (1) is that it is theoretically true that it cant be implemented robustly in Linux. But practically it can be, as evidenced by the fact that Qt does exactly that and has been doing that for decades without any issues. The possible issues that implementing (1) in Wayland could cause are entirely theoretical and extremely unlikely. No one has presented any evidence to the contrary. It seems like some people just assume that because Linux is open source it must be better. Roll eyes.

And on the subject of Linux's keyboard protocol, I will even go so far as to say that treating global modifiers as state is fine, possibly a good thing, it allows for more use cases (even though they are not actual practical ones). However, not updating the modifier state in the key event that changes it is bad, particularly so in a networked context. These two issues are orthogonal. Linux could have had its global modifier state events and also updated modifier state in key events. It chose not to do that, and no one has presented any reason or justification for that. Personally I would guess its historical accident.

[kovidgoyal]

@gpanders Sorry I forgot to address your actual proposal while rambling :)

Your idea would certainly resolve the ambiguity but it would still be "impossible" to implement in Linux so it wont fix @kchibisov's objection. Basically, he will be satisfied only by forcing this protocol to follow linux and for terminal emulators to implement various hacks on other platforms to simulate linux instead of following the other platform's more sensible design and forcing terminal emulators to use various hacks to simulate their behavior on linux.

[kchibisov]

Basically, he will be satisfied only by forcing this protocol to follow linux and for terminal emulators to implement various hacks on other platforms to simulate linux instead of following the other platform's more sensible design and forcing terminal emulators to use various hacks to simulate their behavior on linux.

Maybe you don't actually read what I wrote where I said that I'm fine with doing 1 when you don't attach modifiers for releases at all, e.g. what I see in firefox and what you did initially as a compromise.

And as I said, the only issue I have with 1 is when everyone doing 2, is a lot of work when you need to do releases modifiers, since you need to track each key press user doing including latched modifier upon focus.

[kovidgoyal]

On Fri, Dec 15, 2023 at 09:41:37PM -0800, Kirill Chibisov wrote: >Basically, he will be satisfied only by forcing this protocol to follow linux and for terminal emulators to implement various hacks on other platforms to simulate linux instead of following the other platform's more sensible design and forcing terminal emulators to use various hacks to simulate their behavior on linux. Maybe you don't actually read what I wrote where I said that I'm fine with doing 1 when you don't attach modifiers for releases at all, e.g. what I see in firefox and what you did initially as a compromise.

I never did not *report* modifiers in release events. So not sure what you are saying. I think you mean dont *update* modifier state in release events?

And as I said, the only issue I have with 1 is when everyone doing 2, is a lot of work when you need to do releases modifiers, since you need to track each key press user doing including latched modifier upon focus.

Again, don't follow. Who is this everyone that is doing (2)? Linux is the only one that does (2). To get (2) on any other platform you have to store the last reported modifier state globally in your application and then deal with latched modifiers on focus changes on your own as well. As for *lot of work*, I am fairly sure this can be implemented on Linux in about 20 lines of C. Proof will be when I implement it.

[kchibisov]

I never did not report modifiers in release events. So not sure what you
are saying. I think you mean dont update modifier state in release
events?

See the picture here, alacritty/alacritty#7435 (comment) it's clearly kitty, and there's no control modifier for the first release. That behavior would be fine by me actually.

Edit: Ah, sorry, it's demo of kitty protocol on macOS in ghosty after your changes to spec, I got confused by the title of app.

Who is this everyone that is doing (2)?

I mean, toolkit used by terminal, like GTK, winit, etc. They all hide that from the user of the library, because they have 2 behavior, so you either need to get out of your way and patch them just to work with some terminal protocol, etc.

As for lot of work, I am fairly sure this can be implemented on Linux in about 20 lines of
C. Proof will be when I implement it.

I'm not denying the fact that it could be done in that amount of you work with libxkbcommon directly and self patch glfw. Do that in that amount in e.g. gtk 4.

[kovidgoyal]

I never did not report modifiers in release events. So not sure what you
are saying. I think you mean dont update modifier state in release
events?

See the picture here, alacritty/alacritty#7435 (comment) it's clearly kitty, and there's no control modifier for the first release. That behavior would be fine by me actually.

kitty's current release behaves on Linux as (2) and macOS as (1) that is wha thtis issue is about. We want to settle on either (1) or (2) on all platforms.

Who is this everyone that is doing (2)?

I mean, toolkit used by terminal, like GTK, winit, etc. They all hide that from the user of the library, because they have 2 behavior, so you either need to get out of your way and patch them just to work with some terminal protocol, etc.

Qt is a toolkit used for many terminal emulators, off the top of my head, konsole, qterminal, contour. And it follows (1).

As for lot of work, I am fairly sure this can be implemented on Linux in about 20 lines of
C. Proof will be when I implement it.

I'm not denying the fact that it could be done in that amount of you work with libxkbcommon directly and self patch glfw. Do that in that amount in e.g. gtk 4.

No, I will implement it without patching glfw and not using libxkbcommon in the implementation. Essentially the logic is simply:

store the current state of the left/right modifier keys. And use that state to set the modifier bits only for modifier key events. For all other key events continue doing what you do currently.

This will implement (1) on Linux. And yes, it will fail for modifiers already pressed before focus change and if the wayland compositor decides to do something funky with modifier keys. As already discussed, these are both, theoretical and highly unlikely problems. And the failure applies only to the modifier bits associated with the actual modifier key events. All other key events are unaffected.

[gabyx]

@kovidgoyal : I think your points are perfectly fine. At the end we want all a protocol which can be implemented on all platforms correctly, with of course some small drawbacks, and the latched key thing and focus change, and funky modifier changes inbetween are such drawbacks, which are ok IMO because tackeling would mean we do not reach the first goal which is far more important.

I have a question on the implementation, I maybe understanding things not quite right, but thats ok:
Could you explain what happens in your implementation on Wayland when we get a key press event CTRL_L (which is corresponding to ctrl modifier per default assumption in your keyboard layout in xkb..., we also know that Wayland sends modifier information after the key event CTRL_L which we do not have in this example).

So for (1) you would need to directly set the modifier bit ctrl when you kitty-encode this press event, right?
Are you saying that you detect the press of CTRL_L and then directly with your "state" know that you need a modifier ctrl to encode as well?

[gabyx]

@kovidgoyal : Another thing: where is that protocol change recently which was mentioned? Any link to it? Thanks :-) Just interessted what changed, because that should be reflected in wezterm as well if we missed something etc...

[kovidgoyal]

@gabyx Yes I will basically force the assumption that the ctrl key will change the modifier state in the standard way. This will work fine if the user uses the standard mechanisms to remap the ctrl key as something else. That something else should be identified as the control key by libxkbcommon hopefully. More esoteric changes wont work, but they should be vanishingly rare and would anyway break the usage of the modifier bits to detect what key is a modifier. Fundamentally, on X11/Wayland modifier state is completely detached form keys. You can literally have anything change that state. X11/Linux wants to completely separate modifier state from key events treating them as separate things. This is fine in principle, but in the real world nothing actually works like that.

What protocol change are you talking of?

[gabyx]

@kovidgoyal Does that mean you need to call xkb (after ctrl key has been pressed, or before starting up kitty) and gather the activated real modifier and use that one, once the user presses CTRL_L? If yes that basically means you do the same thing or something similar as the keyboard probing loop on wayland when kitty starts up, correct?. So the task is: which modifier from ctrl, shift, lock, mod1...mod5 is mapping to CTRL_L, such that you can kitty-encode this modifier when CTRL_L is pressed?

[kovidgoyal]

On Sat, Dec 16, 2023 at 12:18:26PM -0800, Gabriel Nützi wrote: @kovidgoyal Does that mean you need to call xkb (after ctrl key has been pressed, or before starting up kitty) and gather the activated **real modifier** and use that one, once the user presses `CTRL_L`? If yes that basically means you do the same thing or something similar as the keyboard probing loop on wayland when kitty starts up, correct?. So the task is: which modifier from `ctrl, shift, lock, mod1...mod5` is mapping to `CTRL_L`, such that you can kitty-encode this modifier when `CTRL_L` is pressed?

No, I'm not going to bother. I will just assume that whatever xkb identifies as the l_ctrl and r_ctrl keys is what changes the state of the ctrl bit in modifiers in the standard way. If there is some more complex scenario needed, it will be an edge case and I can look into it in the future. For example, to be more sophisticated one could actually dynamically change this assumption after the first time the user presses a key that causes modifiers to change in the next modifiers update event. But frankly, I doubt this will be needed.

[kovidgoyal]

Voting was 5 to 2 in favor of (1). I have implemented (1) in master.