Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xcape features (double bind modifiers) #719

Open
timokau opened this issue Jun 20, 2016 · 42 comments
Open

xcape features (double bind modifiers) #719

timokau opened this issue Jun 20, 2016 · 42 comments

Comments

@timokau
Copy link

@timokau timokau commented Jun 20, 2016

Hi,
For me currently the main thing that stops me from giving wayland a try, is missing xcape support.
Now alols/xcape#67 is open for a while, and it doesn't seem like its going to come anytime soon or even is possible.
Is it correct, that the windowmanager would have to implement these features?
If so, have you considered implementing them?

My specific use-cas is:

  • Bind Caps to ctrl when I hold it, escape else
  • Bind Tab to alt-gr when I hold it, Tab else
@ddevault
Copy link
Member

@ddevault ddevault commented Jun 20, 2016

I think you should pitch this to libinput.

Loading

@ddevault ddevault closed this Jun 20, 2016
@timokau
Copy link
Author

@timokau timokau commented Jun 21, 2016

Thank you, I did.

Loading

@timokau
Copy link
Author

@timokau timokau commented Jun 24, 2016

They won't fix, apparently its too high-level for libinput.

Loading

@ddevault
Copy link
Member

@ddevault ddevault commented Jun 24, 2016

Ugh. I'll reopen this but I'm not sure how I want it to take shape in sway.

Loading

@ddevault ddevault reopened this Jun 24, 2016
@timokau
Copy link
Author

@timokau timokau commented Jun 24, 2016

Thanks
It wouldn't really be a good solution if every compositor had to implement this (although I think that especially people using tiling window managers may profit from this).
But if its the only one, I'm glad that you at least consider it.

Loading

@progandy
Copy link
Contributor

@progandy progandy commented Jun 24, 2016

Is this even possible in sway without changing wlc? I think the whole keymapping is hidden and not accesible.

Loading

@unrelentingtech
Copy link
Contributor

@unrelentingtech unrelentingtech commented Nov 22, 2017

Looks like this might be easier with wlroots.

Plus, here's an idea! Since every problem in Wayland is solved by inventing a protocol…

We need one that's kinda similar to xwayland-keyboard-grab, but:

  • obviously, not an exclusive grab — keypresses must be copied to the client;
  • the client should provide the set of keys it's interested in — the compositor can implement security policies based on that. e.g. sway can have two different permit levels, watch-modifier-keys and watch-all-keys, or just allow everyone to grab modifier keys (they're not that interesting after all) and require permit watch-keys for a keylogger;
  • and of course, the interface should also include a request that injects keypresses back.

Example flow:

  • the hypothetical wcape or waycape or whatever requests watch_keypresses(wl_seat, ['Shift_L', 'Shift_R', 'Control'])
  • the user presses and releases Control in a terminal window
  • the compositor sends an event for Control to the terminal (which ignores it), but also sends that event to wcape because it's registered for that key
  • wcape reads the event and requests inject_keypress('Escape')
  • the compositor sends an event for Escape to the terminal

Bonus idea: clients can request a "secure mode" like in macOS, so when you're entering a password, events would not be copied to wcape. (Not super necessary when only giving access to modifier keys, but would be great for, like, running a keylogger for typing statistics I guess :D)

Loading

@Timidger
Copy link
Member

@Timidger Timidger commented Nov 23, 2017

Seems annoying to have to define a whole protocol for this...but it is the most compositor agnostic option :\

Unless there was some other layer above libinput that's implemented that can implement this and all the compositors start using that. Though that seems mostly unnecessary and even less likely than all the compositors using the same protocol.

Loading

@unrelentingtech
Copy link
Contributor

@unrelentingtech unrelentingtech commented Nov 23, 2017

Yeah, just setting xcape style key presses is simple enough to do directly in compositors, but this protocol allows for more stuff, such as TextExpander-like apps, typing statistics, etc.

Loading

@unrelentingtech
Copy link
Contributor

@unrelentingtech unrelentingtech commented Nov 26, 2017

What do y'all think about this protocol?

Loading

@ddevault
Copy link
Member

@ddevault ddevault commented Nov 26, 2017

-1. This is not something I'm interested in doing with a protocol.

Loading

@unrelentingtech
Copy link
Contributor

@unrelentingtech unrelentingtech commented Nov 26, 2017

So it won't be possible to do UI automation/testing/macros (java.awt.Robot, Sikuli, xdotool etc.), text snippet expansion, keyboard typing statistics, etc. under Sway?

I guess I'll have to do this on evdev level then. It's possible to listen directly on evdevs and create a fake keyboard (uinput on Linux, cuse on FreeBSD) to inject events. I believe that a Wayland protocol would've been a much cleaner and better solution >_<

Loading

@ddevault
Copy link
Member

@ddevault ddevault commented Nov 26, 2017

I don't think that any of those use-cases justify putting in what is literally a keylogger protocol extension.

Loading

@unrelentingtech
Copy link
Contributor

@unrelentingtech unrelentingtech commented Nov 26, 2017

What if I want a literal keylogger to analyze my typing speed?

These features are useful. Global input access shouldn't be completely killed, it should be available to privileged applications. macOS uses accessibility APIs for this, you have to tick a checkbox in Preferences to allow an app to do this. Sway has IPC security already, which is pretty similar! permit /usr/local/bin/xcape keylogging

Loading

@ddevault
Copy link
Member

@ddevault ddevault commented Nov 26, 2017

Then you should patch sway.

Loading

@unrelentingtech
Copy link
Contributor

@unrelentingtech unrelentingtech commented Nov 26, 2017

What do you mean by "patch"? "maintain your own fork"? :)

Actually, I'll probably do the evdev-level thing because it will work with GNOME, KDE, etc. out of the box. Would help with bulletmark/libinput-gestures#27 as well.

Loading

@ddevault
Copy link
Member

@ddevault ddevault commented Nov 26, 2017

No, I mean patch. I run sway with some private patches myself. Patching software is not a big deal. Just do it until you get bored of gathering stats on your typing habits.

Loading

@oblitum
Copy link

@oblitum oblitum commented Dec 17, 2017

I'm author of this, may be interesting on this topic:

Specifically the caps2esc plugin covers "Bind Caps to ctrl when I hold it, escape else" and hideaway covers "Hide mouse cursor on inactivity". I use it on Sway (works on X as well).

Loading

@ddevault
Copy link
Member

@ddevault ddevault commented Dec 17, 2017

That's pretty awesome!

Loading

@unrelentingtech
Copy link
Contributor

@unrelentingtech unrelentingtech commented Dec 17, 2017

So, seems like the list of solutions is here: alols/xcape#67

My solution is evscript, it runs Dyon scripts with access to evdev/uinput.

Loading

@frsfnrrg
Copy link
Contributor

@frsfnrrg frsfnrrg commented Jun 2, 2018

I have a patch for sway which permits key event injection (and hence xcape behavior), although there are significant limitations. It requires that one have an identifier for the keyboard (for this example, 0:0:X11_keyboard press)† , and creates a new process every use, but does not require root.

0001-Add-fake_key-command.txt

Note that faked event timestamps are 0, and please read the warning.

Then in the sway config file, add a line like

bindsym --release Control_L exec swaymsg fake_key 0:0:X11_keyboard press_and_release 9

in which 9 is the xkb keycode for Escape. swaymsg indirection avoids a crash.

† You can find this by running with full logging (sway -d), and then reading the log for a line like

2018-06-02 11:13:59 - [sway/sway/input/input-manager.c:221] adding device: '1:3:example_keyboard'

Loading

@emersion
Copy link
Member

@emersion emersion commented Jun 2, 2018

Related: #1779

Note that you can get keyboard identifiers with swaymsg -t get_inputs.

Loading

kalbasit added a commit to kalbasit/shabka that referenced this issue Aug 4, 2018
Current blockers for SWAY adoption:
- Rofi support. Rofi currently does not play well with SWAY, it can't
  grab focus and it's annoying to have to click on it before being able to
  type.
- XCape support. See swaywm/sway#719
@avrelaun
Copy link

@avrelaun avrelaun commented Feb 20, 2019

I'm just going to drop my 2 cents here, I have been using this behaviour for more than a year, and it's awesome for tiling wm / vim / graphical apps with ctrl; left pinky finger has way less stress at the end of the day.

I used https://github.com/oblitum/caps2esc at first and packaged it to exherbo, then @oblitum moved it to https://gitlab.com/interception/linux/plugins/caps2esc as a plugin to a bigger interception-tools project.

Now that hideaway is built-in to sway, having such a generic input manipulation tool running just for the caps2esc behaviour seems a little overkill and being able to directly configure it from sway would be awesome.

Loading

@edrex
Copy link
Contributor

@edrex edrex commented May 24, 2019

@avrelaun if you just want to map caps to escape (rather than conditionally to ctrl or escape depending on whether chorded as caps2esc does) you can do that with a hwdb rule as in https://github.com/edrex/etc/blob/master/udev/hwdb.d/90-custom-keyboards.hwdb.

Loading

@emersion
Copy link
Member

@emersion emersion commented May 24, 2019

if you just want to map caps to escape (rather than conditionally to ctrl or escape depending on whether chorded as caps2esc does)

The preferred solution is to use the xkeyboard-config option.

Loading

@oblitum
Copy link

@oblitum oblitum commented May 24, 2019

For chording keys I think an external tool is preferred over something builtin. It doesn't look like a task for sway. For example, caps2esc has to manipulate key-down and key-up in a specific way to make chording natural and not cause problems like missing keystrokes.

Loading

@avrelaun
Copy link

@avrelaun avrelaun commented May 25, 2019

@edrex I need the exact behaviour with both ctrl and escape depending on the case.

Loading

@jcalonso
Copy link

@jcalonso jcalonso commented Jul 19, 2019

I came here because I am looking for a way to replace this xcape config:

xcape -e 'Shift_L=Shift_L|parenleft;Shift_R=Shift_R|parenright'

Any ideas if this is possible to accomplish with Sway? or other Wayland compatible tools?

Loading

@zsugabubus
Copy link

@zsugabubus zsugabubus commented Aug 11, 2019

Sorry for being off-topic, but if somebody has the same issue and it's the only thing that holds one back to switch, it can be super useful.

@jcalonso, not with (stable) Sway but at a lower level you can achieve it with the above mentioned Interception Tools (by @oblitum). If you could make that work and you don't want to reinvent the wheel you can use my general purpose plugin for a quick solution.

/* tap-rules.h.in */
{ .base_key   = KEY_LEFTSHIFT, /* change this key */
  .tap_key    = KEY_KPLEFTPAREN, /* what to do when pressed alone/pressed once */
  .repeat_key = KEY_KPLEFTPAREN, /* what to do when pressed for longer time (just remove it if you find it useless) */
  .hold_key   = KEY_LEFTSHIFT /* behave as this key as soon as other key pressed */  },
/* similar for right shift... */

@timokau, your config:

/* tap-rules.h.in */
{ .base_key = KEY_CAPSLOCK, .tap_key = KEY_ESC, .repeat_key = KEY_LEFTCTRL, .hold_key = KEY_LEFTCTRL },
{ .base_key = KEY_TAB, .tap_key = KEY_TAB, .hold_key = KEY_RIGHTALT/*altgr*/ },

Loading

@spectras
Copy link

@spectras spectras commented Oct 14, 2019

So, what is the current state of key event copying?

I am the maintainer of https://github.com/keyleds/keyleds. It implements visual feedback on gaming keyboards that have per-key lights, triggering animations to respond to key events.

How would I go about supporting sway/wlroot/wayland, whichever is the right level?

Loading

@emersion
Copy link
Member

@emersion emersion commented Oct 14, 2019

How would I go about supporting sway/wlroot/wayland, whichever is the right level?

We've already said we're not interested in a keylogging protocol. If you want to do this, read events with libinput, like e.g. wshowkeys. This requires a privileged process, because regular processes shouldn't have a way to read all key events.

Loading

@spectras
Copy link

@spectras spectras commented Oct 14, 2019

Let me get this straight:

  • The recommended way to have a secure environment is to force complex applications that interact with the desktop and run user-provided scripts to run as root?

I mean, that the compositor would ask the user for permission before granting access seems logical. That the feature would be completely missing…

Loading

@Kommynct
Copy link

@Kommynct Kommynct commented Jan 9, 2021

Sorry to bump such an old topic but I have found that SKR

https://aur.archlinux.org/packages/skr-git/
https://github.com/DavRack/skr

Should work well for this in principle.

Loading

@goetzc
Copy link
Contributor

@goetzc goetzc commented Jan 9, 2021

@Kommynct I think a better and more mature alternative is Interception Tools which was suggested in a previous comment, using the k2k plugin.

Loading

@spectras
Copy link

@spectras spectras commented Jan 13, 2021

While we're bumping that topic, any updates on how can gaming keyboard effects be implemented on Wayland?

Requirements are as follows. Keyboard management program needs:

  • To monitor key presses (sorry about that, but it's hard to give visual feedback to keypresses without the ability to know when keys are pressed).
  • To monitor either forground or focused application (lighting profiles are based on active application).
  • To run unprivileged.

As long as this cannot be done, there is a whole class of devices that cannot be used fully.
But I did not really follow updates, so maybe I missed some news?

(I could also mention all kinds of business use cases such as screen sharing with remote control, teamviewer, automated GUI testing, robotic process automation software, that just become impossible under Wayland).

Loading

@fluix-dev
Copy link
Contributor

@fluix-dev fluix-dev commented Jan 13, 2021

While we're bumping that topic, any updates on how can gaming keyboard effects be implemented on Wayland?

Most of this is not in the scope of Wayland (the protocols) or the compositor (Sway) itself:

To monitor key presses (sorry about that, but it's hard to give visual feedback to keypresses without the ability to know when keys are pressed).

Is this for changing colors? If so, there's already a variety of Linux tools for controlling Razer and other devices and their colours such as the Openrazer drivers, alongside RazerGenie or Polychromatic GUI tools. They are perfectly capable, probably through the Openrazer drivers (though I'm not too sure), at listening to key presses and modifying the visual effects.

The main point that people point out with adding some way of monitoring keys is that it's basically a keylogging protocol, which doesn't sound great without some better security implementations first.

To monitor either forground or focused application (lighting profiles are based on active application).

The IPC already allows this, you don't really need anything else from Sway.

(I could also mention all kinds of business use cases such as screen sharing with remote control, teamviewer, automated GUI testing, robotic process automation software, that just become impossible under Wayland).

Look at something like ydotool for automated typing and clicking.

Loading

@t184256
Copy link

@t184256 t184256 commented Apr 25, 2021

I think it sounds like a job for remappers like kmonad, not sway.

Loading

@spectras
Copy link

@spectras spectras commented May 1, 2021

Razer and other devices and their colours such as the Openrazer drivers, alongside RazerGenie or Polychromatic GUI tools. They are perfectly capable, probably through the Openrazer drivers (though I'm not too sure), at listening to key presses and modifying the visual effects.

Those need to run as root, and even as kernel module for some features.
Except for RazerGenie which relies on a root daemon to bypass wayland device security entirely.

That is indeed a path we could take. Is that really the security model Wayland wants to promote? Take features that could be implemented as regular user-space unprivileged code, and tell developers to make them ring0 and users to give them root permission?

Well I guess that is one way to see it, after all if you distribute the key to everyone, at least when you find your vault empty you can rejoice that is was not broken.

The main point that people point out with adding some way of monitoring keys is that it's basically a keylogging protocol

The main point is: some use cases require software to know a key was pressed.

  • It can already be done today, one simply has to ditch security and make everything run as root.
  • Or WMs can design a secure protocol, giving control to the user. “Application foo has requested permission to monitor your keypresses, [insert here whatever additional warning makes sense] — do you agree?” seems perfectly reasonable.

By refusing the latter approach, Wayland developers just force people to do the former. The result is a net decrease in overall security.

Also, having it done by the WM would make it possible to have fine control over which keypresses would be visible. Like MS Windows does, where keypresses directed to sensitive applications are not visible to hooks.


Anyway, as you see I am less active on this now. We just marked our application as incompatible with Wayland, and the editors we work with simply added "no Wayland" to their system requirements for their own software. Users are fine with that, so that works for us at the moment.

When Wayland is mature enough we can reconsider. Or just run everything as root, if it's really the Wayland's way as this conversation implies. So much for the promise of improved security.

Loading

@fluix-dev
Copy link
Contributor

@fluix-dev fluix-dev commented May 1, 2021

@spectras Since my comment I've read a bit more about the state of security in Wayland, so here's some stance/thoughts.

At the moment, as we've both stated, the way for direct event capture is through running something as root. I completely agree with you that this isn't ideal, but it requires at least some decision from the user.

I am against adding a protocol to globally grab key events at this moment because a protocol for proper security contexts hasn't been established yet (see here and here). Adding the key grabbing protocol before the security contexts would make unprivileged applications be able to log all keys: arguably worse than the current situation. Once a security context protocol has been merged, I think the protocol you're proposing definitely has some merit.

Note: I do not have extensive knowledge of what's discussed here, so take my comments with a grain of salt.

Loading

@kennylevinsen
Copy link
Contributor

@kennylevinsen kennylevinsen commented May 2, 2021

Indeed. Use of root is bad, but this is a net increase I'm security from X: rather than everyone and their mother being able to keylog, only root has the ability, and root is omnipotent anyway.

A Wayland protocol will be designed eventually for tasks like keybinding. We will probably still avoid full keylogging. The only reason this does not exist yet is due to other priorities.

There is nothing wrong with falling back on X until Wayland fits your use case.

Loading

@spectras
Copy link

@spectras spectras commented May 15, 2021

Hey both of you. Thanks for taking the time and energy on this topic.

fluix-dev> I agree, having security contexts makes sense in general, and listening keypresses, being sensitive, could live behind such a mechanism. I am not familiar with wayland development. Is there a way we can help make security contexts happen?

kennylevinsen> my point is the lack of granularity (root or nothing) pushes use cases that used to be user applications to root privileges. Which incurs a decrease in security from X.

In the end, it is the whole class of "user application that needs some special gui permission" that is missing from Wayland, and "make them all root" is hardly a solution. Keypresses are just an example of such a permission.
I think fluix-dev points to the right thing with security contexts, it does look like a clean approach to the more general problem.

Loading

@kennylevinsen
Copy link
Contributor

@kennylevinsen kennylevinsen commented May 15, 2021

In the end, it is the whole class of "user application that needs some special gui permission" that is missing from Wayland

We have plenty of privileged protocols already. Screen recording, layer shell, clipboard management are all protocols that are not intended for general use and, at some point, will be subject to restriction.

A dedicated key binding protocol would be in the same category and require approval, but a omnipotent key logging protocol is unwanted as abuse is unavoidable (just one popular proprietary application misusing it for key bindings is enough) and users would end up unable or unwilling to restrict it, leading to a security situation worse than what we have now: unprivileged keylogging free-for-all.

A key binding protocol is pretty easy to write, but as the primary usecases so far have all been proprietary applications such as discord and zoom that we cannot work on an implementation for (working implementations are a requirement to contribute protocols), no work has To my knowledge been made on this subject. It should be quite a simple protocol though.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet