Support "binding" press and hold for some combos, like Cmd-tabbing

[joshgoebel]

Currently combos are seen as "discrete units" in that you can easily do the following on the output:

• Remap CMD-tab to Ctrl-Tab
• CMD-tab
• CMD-tab
• CMD-tab

This will Ctrl-Tab 3 times (fully pressing AND releasing all the keys every time). What you cannot do is:

• Remap CMD-tab to Ctrl-Tab
• Press and hold CMD (to hold a switcher open)
• Hit tab (while CMD is steadily down)
• Hit tab (while CMD is steadily down)
• Hit tab (while CMD is steadily down)

The "and hold" part is impossible with xkeysnail 0.4 and is the reason Kinto.sh currently uses a patched xkeysnail (that has it's own issues).

---

The plan is to fix this "better" by implementing smarter sticky keys such that the keymapper understands when you might be triggering a "sticky" combo and holds the key down on the output until you are finished.

[joshgoebel]

Copying this from another thread:

Held keys half work now. I currently only works (goes sticky) for simple A -> B keymaps, like CMD -> Ctrl. So:

• If you hit CMD-tab (and hold CMD) CMD->Ctrl is marked as sticky (and the release of control is bound to CMD)
• If you hit say CMD-shift-tab nothing magical happens.

But if you START with CMD-tab and then you start shifting it'll work because the CMD is sticky (at that point) and it doesn't matter that that shift will be reasserted each time. How many people start tabbing backwards? Might be some wouldn't even notice....

But if there are other places that have X-Y-tab => X-tab or X-tab => X-Y-tab those don't currently work.

[joshgoebel]

@RedBearAK The problem is with multiple keys involved things get very fuzzy... Take this mapping:

K(Ctrl-Alt-Tab) => K(CMD-Shift-Tab)

So lets say we just sticky bind whatever we can... so after pressing the combo we have a stick table that looks like:

• (Ctrl, Alt) => (CMD, Shift)

Both sets of keys are sticky/linked to each other (as a group)... so as long as you hold Ctrl and Alt we'll hold (CMD, Shift) on the output... if you release BOTH, that's easy - we release both on the output. But say you only release Alt... Since we can't be certain the keys map individually (you could have A => B,C) the only thing we're certain of is that now the output combo is wrong - but we have no idea what we should replace it with.

• Should we release CMD?
• Should we release Shift?
• Should we release both?

So you perhaps we analyze the sequencing (and require people be picky about order), so our sticky table:

• (Ctr) => (CMD)
• (Alt) => (Shift)

Now at least there are no ambiguous bindings and we can lift the keys individual or together at any time and have that state maintained across the input<->output. Then you have the edge case:

• A, B, C mods=> D mod - releasing any of A B or C would release D
• D mod=> A, B, C mods - releasing D would release A B and C

---

The edge case gets harder though because in the former edge case (A, B, C=> D).... if we release C... what is to be done about A, and B? Perhaps they just go back into suspension waiting to see what's next?

I was thinking we had a case where we had to hold some keys on output AND hold some keys in suspension, which isn't currently possible but perhaps I was imagining that scenario.

I should state I'm assuming here that only one sticky table can be active at once... so I guess the final edge case is when you hold some keys and switch from one key combo to another which have incompatible sticky tables?

[joshgoebel]

K(Ctrl-Alt-Tab) => K(CMD-Shift-Tab)  #blue
K(LCtrl-LAlt-B) => K(RCtrl-Ralt-B)  #green

So you start tabbing (blue) and sticky table becomes =

• (Ctlr) => (CMD)
• (Alt) => (Shift)

Then you release Alt and (while still holding Ctrl) trigger the (green) combo... it seems to apply the new sticky table would "break" the Ctrl -> CMD (releasing CMD on the output)...

So perhaps there isn't... oh I see the issue now... it's when you release Alt then repress Alt. The release breaks the sticky, but when you press it again we don't know what to do with it (we can't assume it's shift anymore) - it should be suspended (hidden from the output temporarily) - but we're already asserting Ctrl=>CMD on the output...

So I'm starting to think I need per-key suspensions... and then the second alt press would be "deferred" until we know what to do with it...

[joshgoebel]

@RedBearAK If you don't follow all that, that's ok, the notes are largely for myself.

[joshgoebel]

@rbreaves Are all your sticky key needs K() => K() ie:

K("RC-Tab"): K("M-Tab"),                    
K("RC-Shift-Tab"): K("M-Shift-Tab"),        
K("RC-Grave"): K("M-Grave"),                  
K("RC-Shift-Grave"): K("M-Shift-Grave"),     

Is there anywhere you've needed sticky + lots more combos...

K("RC-Tab"): [K(...), K(...), K(...), K(...)]

If it's just one on one I think I've covered all the cases above... I'm still not seeing where (if this is done correctly) it couldn't just be on for default - for every combos... making the keyboard behavior far more realistic... but if that was problematic I think a sticky wrapper. I wonder if there are some people out there who are holding keys down but PURPOSELY enjoying the fact that we're sending discrete keys... if so enabling this by default would break those users expectations.

K("RC-Tab"): sticky(K("M-Tab")),                    

[RedBearAK]

@joshgoebel

I don't know if this will make sense, but I kind of feel like "sticky" is a misnomer in this context. There is a "sticky keys" setting in multiple desktop environments, but it's typically an accessibility option for people who have difficulty reaching multiple keys simultaneously. If enabled, "sticky keys" will treat a modifier like Shift as if it is being held down, even though it isn't. I think it's "sticky" until you press the key again, or hit Escape. Not sure, I never really used it.

In this case, there would be only the initial key event when the "sticky key" was pressed, and then another when it was pressed again to release it.

But with "held keys", shouldn't the system be constantly receiving key events for the key(s) that is(are) physically being held down? I mean, if there is no remapper being used, isn't that what the system would be seeing? Press-press-press, and then it just waits until it sees a Release event?

You're right about me not completely following the technical side just yet, but I'm mainly referring to the terminology regarding what we're trying to mimic. It could be confusing if someone associates it with the common accessibility feature that turns modifiers into locking keys without needing to be held.

Let me know if I'm completely off base.

[joshgoebel]

but I kind of feel like "sticky" is a misnomer in this context. ... It could be confusing if someone associates it with the common accessibility feature

Oh no disagreement now that you bring it up, we need a new/better name. I was only trying to get at the fact that we're "binding" (sticking) together the input/output keys such that the mapping becomes "seamless"... this would actually be 100% possible with only a single modmap... because we wouldn't have to worry about keys changing meaning midstream. It's helpful to have names for these concepts (to name variables, classes, etc).

But with "held keys", shouldn't the system be constantly receiving key events for the key(s) that is(are) physically being held down?

Yes, that was just assumed and works already... if a key is being "held" and bound then any input REPEAT events are simply passed on to the output as REPEATS... so holding the key continues to repeat it (well the mapped version of it) - just like it normally would. If a key is suspended then it's repeats are dropped and silently ignored.

[joshgoebel]

Is the word "bound" as in bound together better? I'm open to other suggestions if you have any.

[RedBearAK]

@joshgoebel

Is the word "bound" as in bound together better? I'm open to other suggestions if you have any.

I'm not sure I'm qualified to answer that since I think I'm fundamentally not understanding why the keys would need to be "bound" together in the first place.

I was just thinking earlier how with Kinto currently I can easily go back and forth between Cmd+Tab and Cmd+Shift+Tab and Cmd+Grave/Cmd+Shift+Grave, and the only thing that needs to be held down to keep the task switcher open and keep the final "action" from happening is Cmd.

we're "binding" (sticking) together the input/output keys such that the mapping becomes "seamless"... this would actually be 100% possible with only a single modmap... because we wouldn't have to worry about keys changing meaning midstream.

My only experience with the modmaps is what Kinto does, which has the one conditional remap of modifiers for GUI apps and another for terminals.

Probably a dumb question, but "seamless" as opposed to what?

[joshgoebel]

keep the task switcher open and keep the final "action" from happening is Cmd.

Yes, that doesn't happen by magic. Kinto very particularly patched xkeysnail to make that possible while also breaking many multi-key combos. I think there are some other hidden bugs also. When I say bind I mean how the software realizes that as long as you're holding Cmd [input] - to keep holding Ctrl [output] (without pressing and releasing it, which is what 0.4 does)... Kinto.sh does this TOO aggressively - even when it should be letting up on keys.

This is not a concept built into xkeysnail at all. With 0.4.0 it's literally impossible to "hold" any modifier down [that is part of a combo]...

Try something like an extended sequence of (like the one you describe) with the non-quiet output and debug logging for actual keyboard output events (you'll need to add that manually I think)... look at all the traffic... you press shift and release it yet Kinto will keep pressing/releasing it over and over forever until you finally release Cmd. You don't notice it because it does it "in the gaps", but this is still not correct, and leads to other issues.

Probably a dumb question, but "seamless" as opposed to what?

Kinto is not seamless (see my previous paragraph). 0.4.0 is not seamless - hence the need for Kinto to patch it, etc... nether behavior is correct, they're both broken in different ways.

[joshgoebel]

As I think I'm mentioned before bind should be the default... That's what a keymapper's job is - bind input and output keys together, even if they aren't the same key - such that you can't tell the difference... so I think 0.4.0 is wrong with it's release by default... but I'm worried there may be some people relying on that behavior.

But it's "un-natural" behavior... holding a key continuous while some layer pretends you're pressing and releasing it over and over... this isn't behavior you get with most (non-gaming) keyboards. It reminds me of the "fake repeat" buttons that some console controllers have... you just smash the button and the controller makes it appear like you're hitting it 100 times a second...

That seems unusual for most normal app usage though.

But (without a full engine rewrite) I think we can have both behaviors, ti's just a matter of which you get by default and which you have to manually ask for.

[RedBearAK]

@joshgoebel

Thanks, that was helpful.

I think that's a good analogy, with the gaming "fake" repeat button. That does seem to be what mainline (or is it just v0.4.0?) xkeysnail is doing.

But (without a full engine rewrite) I think we can have both behaviors, it's just a matter of which you get by default and which you have to manually ask for.

That would be nice. I'll do my best to provide useful feedback testing your branch.

[joshgoebel]

mainline (or is it just v0.4.0?)

I refer to the latest master branch of xkeysnail as mainline and current master also happens to point to 0.4.0.

[RedBearAK]

@joshgoebel

But was the “fire immediately” behavior the same in previous versions, like v0.3.0? That’s what I was getting at.

[joshgoebel]

No clue, before my time.

[joshgoebel]

@RedBearAK Feel free to test this now, you'll need to modify your bindings by hand:

    K("RC-Tab"): [bind, K("Alt-Tab")], 

bind simply refers to the new ComboHint.BIND which triggers auto-binding behavior on the first two keys on each side of the combo (input and output side).

[joshgoebel]

CC @rbreaves This is now explicit rather than auto... the key pairing (in <=> out) is still auto (first mod on both sides), but you have to declare that you want the sticky/bound behavior with bind... and if you do then the keys are bound after the first combo - until you release the input key.

I think that for your uses binding a single key (usually Cmd => something) should work... so now it should be easy to test and confirm whether that is indeed the case or not.

[RedBearAK]

K(Ctrl-Alt-Tab) => K(CMD-Shift-Tab)

This shortcut mapping alone (the example given early in the thread) is why I argued against the use of Cmd as an alias at all. I have no idea what is meant by this. If Cmd is RC, like under Kinto, this is mapping Ctrl+Alt to Ctrl+Shift, which makes no sense. If Cmd is Super/Win/Meta, as I assume it "should" be according to what it actually is, this is mapping Ctrl+Alt+Tab to Super+Shift+Tab, which again I have no idea why that mapping would ever happen. Unless this example is just random and completely unrelated to any kind of task switching shortcut on either macOS or Linux.

In short, the use of Cmd is super, ultra, mega confusing to me, especially when dealing with keyboards that are not Apple keyboards.

K("RC-Tab"): [bind, K("Alt-Tab")], 

I guess I will start keeping a keyszer.py config separate from kinto.py and start trying things like this.

[RedBearAK]

I hope this is what you meant. It seems to be working.

    K("Alt-Tab"): pass_through_key,                 # Default - Cmd Tab - App Switching Default
    K("RC-Tab"): [bind, K("Alt-Tab")],                      # Default - Cmd Tab - App Switching Default
    K("RC-Shift-Tab"): [bind, K("Alt-Shift-Tab")],          # Default - Cmd Tab - App Switching Default
    K("RC-Grave"): [bind, K("Alt-Grave")],                  # Default not-xfce4 - Cmd ` - Same App Switching
    K("RC-Shift-Grave"): [bind, K("Alt-Shift-Grave")],      # Default not-xfce4 - Cmd ` - Same App Switching

[joshgoebel]

Unless this example is just random and completely unrelated to any kind of task switching shortcut on either macOS or Linux.

It is random. You need some complex examples to talk about all the edge cases.

I hope this is what you meant

Yep... I wanted the bind on the left side but that was just too annoying to deal with, so right side it is. And one could make nicer syntax for this maybe... but the bind is just a magic command that says "bind the incoming combo to the next combo in this sequence"... and it hooks up the first mod on each side in a sticky pair.