Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast modifier+mouse clicks do not register (because of suspend) #16

Closed
joshgoebel opened this issue Jun 7, 2022 · 21 comments
Closed

Fast modifier+mouse clicks do not register (because of suspend) #16

joshgoebel opened this issue Jun 7, 2022 · 21 comments
Labels
bug Something isn't working help welcome Help/contrib is esp welcome

Comments

@joshgoebel
Copy link
Owner

joshgoebel commented Jun 7, 2022

Update: Of course this "just works" if you ALSO add your mouse as a device because the mouse buttons will trigger the modifiers to be un-suspended so that they can apply to the mouse button "keys".

If this works for everyone should this just be the solution? Finally a good reason to add add_device helper to the config file?


Output from start to trying to Ctrl+click a link. I held Ctrl for only a fraction of a second, which is how I frequently do it.

(--) CONFIG: /home/kris/.config/kinto/kinto.py
(+K) Grabbing AT Translated Set 2 keyboard (/dev/input/event4)
(--) Ready to process input.
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) suspending keys [<Key.RIGHT_CTRL: 97>]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) resume because of mod release

You should join the Discord if that would be any easier.

Not a big fan of the more "chatty" type applications like Discord, but if that's easier for you...

Originally posted by @RedBearAK in #14 (comment)

@joshgoebel
Copy link
Owner Author

@RedBearAK

You see in the log right there "suspending keys right ctrl"... that's why you can't use it to click with, it's suspended...

@joshgoebel
Copy link
Owner Author

Copying comments from other thread: I suppose we could start having flags - so people could pick and choose what type of broken they want, but I'd prefer to have no broken at all.

The problem is when you hit a modifier we have to wait until we know what you want to do with it (so we don't send the WRONG key to the output)... this works great with keys, but since the mouse is invisible we can't know you want to do a multi-key gesture using the mouse.

0.4.0 just sends the wrong keys to the output and then unpresses them later, but that's why it can trigger unintentional keypresses.

If we turn off suspend then #9 reasserts itself.

@RedBearAK
Copy link
Contributor

Copying from the other thread:

Hmm, I remember having that issue with Super in KDE, it kept opening the application launcher menu. You have to put a setting in a specific config file and restart Kwin to get it to unbind the Super key. For some reason they don't expose it in the GUI anywhere.

So your suspension should bypass the issue of menus getting triggered with Alt? Main places that has bothered me is Firefox and VSCode.

Isn't it primarily Alt, and perhaps Super to a lesser extent, that has this issue of unwanted actions? What if you don't do the suspension on Ctrl? Since Ctrl is used in so many direct shortcuts in Linux apps, and generally isn't used to do something on its own.

Now you're going to tell me about VIM or some other popular app that uses Ctrl all by itself to do something. LOL.

@joshgoebel
Copy link
Owner Author

Now you're going to tell me about VIM or some other popular app that uses Ctrl all by itself to do something.

Nope but that doesn't mean it doesn't exist. In general using modifiers as normal keys (without some sort of delay) is problematic and not a thing software should be doing - for this very reason... but that ship seems to have sailed long ago.

perhaps Super to a lesser extent,

Gnome Activity Center? Gnome binds super by default to pull up the task switcher apparatus thingy IIRC.

@RedBearAK
Copy link
Contributor

RedBearAK commented Jun 7, 2022

Gnome binds super by default to pull up the task switcher apparatus thingy IIRC.

Oh, right. I always install Kinto immediately after installing any Linux distro it's compatible with, and it always does a good job unbinding the Super key (except in KDE). So I tend to forget about that behavior in GNOME. I bend GNOME to my will. Mwahahahaha!

I am very used to modifier keys not doing things on their own, from Windows (pre-special Windows key days) and macOS, where modifiers just tend to be passive modifiers that change the action of other things. I found the action of Super in GNOME quite aggravating. Fortunately Kinto fixes that.

So suspend modifiers to keep them from doing things on their own, which I would generally be a fan of, but the modifier is needed to change the action of a click. This is a bit of a sticky problem, no pun intended.

With Kinto I guess I just stayed away from using the Alt key shortcuts too much with apps that have Alt-activated menus, so the defect hasn't really bothered me too much.

Doesn't xkeysnail have the facility to detect input from the mouse buttons and react accordingly? I thought I saw mousey stuff in key.py.

@joshgoebel
Copy link
Owner Author

joshgoebel commented Jun 7, 2022

Doesn't xkeysnail have the facility to detect input from the mouse buttons and react accordingly? I thought I saw mousey stuff in key.py.

Yeah, I just don't love the idea... it should work already... just run it and make sure it's reading from both devices: (you'll of course need your own file names)

./extended.sh -c kinto.py --device /dev/input/event3 /dev/input/event2

I can Cmd-Click on links in browser just fine.

@joshgoebel
Copy link
Owner Author

Of course now my mouse is slow again because it's an entirely different input device....

@joshgoebel joshgoebel added bug Something isn't working help welcome Help/contrib is esp welcome labels Jun 7, 2022
@RedBearAK
Copy link
Contributor

it should work already... just run it and make sure it's reading from both devices: (you'll of course need your own file names)

./extended.sh -c kinto.py --device /dev/input/event3 /dev/input/event2

Looks like it would be event4 and event9 here, but I don't know where this extended.sh is located. Is that relevant?

I ran it like this and it still doesn't Cmd+click without waiting past the suspension time.

./bin/xkeysnail -c ~/.config/kinto/kinto.py --device /dev/input/event4 /dev/input/event9

@joshgoebel
Copy link
Owner Author

Do you see the button clicks in the output? They should turn off suspend.

@RedBearAK
Copy link
Contributor

Do you see the button clicks in the output? They should turn off suspend.

Don't see any evidence of button clicks in the output, even if I hold the modifier past the suspension time. Maybe I'm using the wrong mouse device.

(--) CONFIG: /home/kris/.config/kinto/kinto.py
(DD) suspending keys [<Key.LEFT_SHIFT: 42>]
(DD) resume because of mod release
(OO) press LEFT_SHIFT
(OO) release LEFT_SHIFT
(+K) Grabbing AT Translated Set 2 keyboard (/dev/input/event4)
(+K) Grabbing ETPS/2 Elantech Touchpad (/dev/input/event9)
(--) Ready to process input.
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) suspending keys [<Key.RIGHT_CTRL: 97>]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) resume because of mod release
(OO) press RIGHT_CTRL
(OO) release RIGHT_CTRL
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) suspending keys [<Key.RIGHT_CTRL: 97>]

I'll try other devices from the list (which by the way I have to run the Kinto branch to see the whole device list like this, not sure how to get it out of your branch):

No keyboard devices specified via (--devices) option.
xkeysnail picks up keyboard-ish devices from the list below:

----------------------------------------------------------------------------------
Device               Name                                Phys
----------------------------------------------------------------------------------
/dev/input/event0    Power Button                        PNP0C0C/button/input0
/dev/input/event1    Sleep Button                        PNP0C0E/button/input0
/dev/input/event2    Lid Switch                          PNP0C0D/button/input0
/dev/input/event3    Power Button                        LNXPWRBN/button/input0
/dev/input/event4    AT Translated Set 2 keyboard        isa0060/serio0/input0
/dev/input/event5    Video Bus                           LNXVIDEO/video/input0
/dev/input/event6    Video Bus                           LNXVIDEO/video/input0
/dev/input/event9    ETPS/2 Elantech Touchpad            isa0060/serio1/input0
/dev/input/event10   Acer Wireless Radio Control         acer-wireless/input0
/dev/input/event11   Acer WMI hotkeys                    wmi/input0
/dev/input/event7    ELAN0504:01 04F3:312A Mouse         i2c-ELAN0504:01
/dev/input/event8    ELAN0504:01 04F3:312A Touchpad      i2c-ELAN0504:01
/dev/input/event12   HD User Facing: HD User Facing      usb-0000:05:00.3-1/button
/dev/input/event13   HD-Audio Generic HDMI/DP,pcm=3      ALSA
/dev/input/event14   HD-Audio Generic Front Headphone    ALSA
/dev/input/event15   py-evdev-uinput                     py-evdev-uinput

Okay, now enable remapping on the following device(s):

------------------------------------------------------------------------------
Device               Name                                Phys
------------------------------------------------------------------------------
/dev/input/event4    AT Translated Set 2 keyboard        isa0060/serio0/input0

I'll try the "i2c" devices.

@joshgoebel
Copy link
Owner Author

There is -h and --list-devices, it just doesn't happen if you don't ask for it.

@RedBearAK
Copy link
Contributor

Tried with event8 and unfortunately that completely takes over the mouse cursor. BTN and finger touch events show up in the log but the cursor will no longer move at all. The event7 device seems to have no effect on the cursor, and no output.

There is -h and --list-devices, it just doesn't happen if you don't ask for it.

Got it.

@joshgoebel
Copy link
Owner Author

joshgoebel commented Jun 7, 2022

BTN and finger touch events show up in the log

Then you need device 8 (you need the BTN events)... all events should pass thru... run evtest on device 8 and post a small snippet.

Ah I know, it's that we don't support those events on the other side...

@joshgoebel
Copy link
Owner Author

https://github.com/torvalds/linux/blob/master/include/uapi/linux/input-event-codes.h#L839
https://github.com/joshgoebel/xkeysnail/blob/main/xkeysnail/output.py#L34

You'll need to extend the UInput device to perhaps cover more of EV_REL or perhape EV_ABS... you should get an idea by running EVTEST...

@joshgoebel
Copy link
Owner Author

https://python-evdev.readthedocs.io/en/latest/_modules/evdev/uinput.html

You could also perhaps try UInput.from_device(devices)... to just add up the capabilities of your keyboard and touchpad and use that... it looks like you can pass the devices wrapped as InputDevice or as just strings.

At soe point too much capability is bad though as that's why we currently have code to prevent us from being mistaken as a joystick - although that's before my time and I dunno what happens if we are mistkaen as a joystick.

@joshgoebel
Copy link
Owner Author

I'd be tempted to try the from_device stuff properly but hotplug complicates all that since you have to change the capabilities of the device on the fly.

@RedBearAK
Copy link
Contributor

Pretty hard to get a "short" snippet of the touchpad device. It's very chatty in the output with even the slightest touch. But here's the tail end of some touches and clicks.

Event: time 1654633148.060271, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1203
Event: time 1654633148.060271, type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 2136
Event: time 1654633148.060271, type 3 (EV_ABS), code 0 (ABS_X), value 1203
Event: time 1654633148.060271, type 3 (EV_ABS), code 1 (ABS_Y), value 2136
Event: time 1654633148.060271, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 28000
Event: time 1654633148.060271, -------------- SYN_REPORT ------------
Event: time 1654633148.067174, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1198
Event: time 1654633148.067174, type 3 (EV_ABS), code 0 (ABS_X), value 1198
Event: time 1654633148.067174, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 35000
Event: time 1654633148.067174, -------------- SYN_REPORT ------------
Event: time 1654633148.074477, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1193
Event: time 1654633148.074477, type 3 (EV_ABS), code 0 (ABS_X), value 1193
Event: time 1654633148.074477, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 42000
Event: time 1654633148.074477, -------------- SYN_REPORT ------------
Event: time 1654633148.081257, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1190
Event: time 1654633148.081257, type 3 (EV_ABS), code 0 (ABS_X), value 1190
Event: time 1654633148.081257, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 49000
Event: time 1654633148.081257, -------------- SYN_REPORT ------------
Event: time 1654633148.088171, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1187
Event: time 1654633148.088171, type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 2137
Event: time 1654633148.088171, type 1 (EV_KEY), code 272 (BTN_LEFT), value 0
Event: time 1654633148.088171, type 3 (EV_ABS), code 0 (ABS_X), value 1187
Event: time 1654633148.088171, type 3 (EV_ABS), code 1 (ABS_Y), value 2137
Event: time 1654633148.088171, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 56000
Event: time 1654633148.088171, -------------- SYN_REPORT ------------
Event: time 1654633148.095050, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1184
Event: time 1654633148.095050, type 3 (EV_ABS), code 0 (ABS_X), value 1184
Event: time 1654633148.095050, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 63000
Event: time 1654633148.095050, -------------- SYN_REPORT ------------
Event: time 1654633148.101934, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1183
Event: time 1654633148.101934, type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 2138
Event: time 1654633148.101934, type 3 (EV_ABS), code 0 (ABS_X), value 1183
Event: time 1654633148.101934, type 3 (EV_ABS), code 1 (ABS_Y), value 2138
Event: time 1654633148.101934, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 70000
Event: time 1654633148.101934, -------------- SYN_REPORT ------------
Event: time 1654633148.107592, type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value -1
Event: time 1654633148.107592, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 0
Event: time 1654633148.107592, type 1 (EV_KEY), code 325 (BTN_TOOL_FINGER), value 0
Event: time 1654633148.107592, type 4 (EV_MSC), code 5 (MSC_TIMESTAMP), value 76000
Event: time 1654633148.107592, -------------- SYN_REPORT ------------

@joshgoebel
Copy link
Owner Author

#define ABS_MAX 0x3f

Yeah minimally you mway want everything up to ABS_MAX... you could probably do it briefly with something like:

    return UInput(events={ecodes.EV_KEY: _keyboard_codes,
        ecodes.EV_REL: set([0,1,6,8,9]),
        ecodes.EV_ABS: set(range(0,0x3f))
        })

@joshgoebel
Copy link
Owner Author

Thus allowing our virtual output device to communicate all those ABS touchpad events... I don't think MSC is important but it might be.

@joshgoebel
Copy link
Owner Author

I really wish there was a way to just subscribe to specific events from a device, but I don't think that is a thing - really we don't even need to grab the trackpad/mouse - we just need to listen to it. I think that's doable - but then I think we still have to listen to every single event it fires and filter them out in Python.

@joshgoebel
Copy link
Owner Author

Closing this in favor of #20... once you add the clicking device Cmd-clicking works just fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help welcome Help/contrib is esp welcome
Projects
None yet
Development

No branches or pull requests

2 participants