SharpHook on Wayland #175
Replies: 1 comment 1 reply
|
Hello, As you might remember from #169, I ended up giving up on X11 and writing my own evdev hook + uinput synthesis for my project (Keysharp, a cross-platform hotkey and input automation tool). That's been shipping for a little while now, mostly to a few beta testers who broke it in interesting ways, so I've run into a decent number of the things you're planning here. One general caveat: some of my Linux code is verified on real hardware and some of it is only verified by code review and CI, so I'll try to note which is which where it matters. Additionally, this post is also powered by Claude - without AI I would just have ignored Wayland altogether... Suppressing eventsI do understand that there is real danger in that: I shipped a version where a beta tester registered a single harmless hotkey on Mint/Cinnamon (X11) and lost all keyboard and mouse input completely, only a reboot fixed it. But I consider the danger worth it: namely, how many uses can you think for just monitoring input events without capturing them, besides malicious ones like keylogging? Anything interesting usually requires blocking events, for example implementing hotkeys and key remaps. The fatal part isn't grabbing devices as such, it's grabbing devices in the same process that runs the user's callbacks. Your objection that a hook taking 1 second delays all input by 1 second with no way to unregister is exactly right for an in-process design, but it mostly goes away if you split it: a privileged broker process owns the grabs and the virtual device and runs no user code at all, and the application is just a client on a socket. The client can be slow, deadlock, crash or get SIGKILLed and the worst case is that suppression stops working. The things that made this survivable for me, all of which live in the broker and not in the app:
That said, if you still don't want it in libuiohook, could you design the extension point you mentioned (the one for how libinput opens the input files and how virtual devices are created) so that a caller can hand you already-grabbed fds plus a replay sink, rather than just open fds? That way libuiohook doesn't have to take any position on suppression, but someone who wants it can build it on top without forking. That seems like a much smaller commitment than implementing it yourself. A separate Linux input libraryA bigger version of that same idea, which may or may not appeal to you: it might be worth considering whether the Linux-specific work belongs inside libuiohook at all, or whether it wants to be its own project. The starting point is that the privileged parts can't really ship the way the rest of the library ships. Grabbing devices, creating uinput devices and prompting the user for permission all need a root-side install — a systemd unit, udev rules, possibly a resident daemon — and that's a distro packaging problem rather than something you can put in a NuGet package next to a But I think that argument extends further than just privilege, and this is the part I'd find most interesting. Most of the things Wayland won't tell you are obtainable from the compositor, if you can get an extension installed into it — a GNOME shell extension exposing a D-Bus method, a KWin script, the Cinnamon equivalent. Those can't ship in a NuGet package either, since they have to be installed into the user's desktop environment and enabled there, so they end up on exactly the same side of the seam as the daemon does. What follows from that is the useful part, I think. On Wayland the information you need is reachable either from below or from above, and neither one is reachable from an ordinary library. From below (evdev and uinput, with privileges) you get physical events, synthesis, key and button state, and idle time. From above (a compositor extension) you get the absolute cursor position, output geometry including fractional scaling, work areas and so on. Neither half is sufficient on its own, and they complement each other in fairly awkward ways — KWin for instance has idle timeout notifications but no way to query the current idle duration as far as I can tell, so that one has to come from evdev, while the cursor position can only come from the compositor. So something that owns both halves could be a complete Wayland input automation component rather than just a hook backend: global hooks, event simulation, blocking, absolute cursor position and warping, output enumeration with scaling, work area and idle time, behind one interface, with each piece answered by whichever half happens to be available on that machine. That's roughly the set of things that everyone who hits the Wayland wall currently reimplements separately, so it seems like something that would be worth existing on its own regardless of what libuiohook does with it. libuiohook's Linux backend would then just consume whatever subset of it is installed, and fall back to its own unprivileged path — legacy X11, or a read-only libinput hook on Wayland — when none of it is there. I didn't plan this shape, I ended up at it because the constraints pushed me there: a root daemon, plus per-desktop extensions, plus an in-process client that picks what's available at startup. So I suspect it's fairly forced by the problem rather than being a matter of taste. What I think it would get you:
There are definitely downsides to that, though. It's a third layer (SharpHook → libuiohook → this), and an external dependency - probably not that bad actually, because libuiohook already has external library dependencies. The extension half is also the part that needs the most upkeep, since GNOME in particular breaks its extension API fairly regularly — I maintain a GNOME extension, a Cinnamon one and KWin scripts for this, and keeping those working is easily the most annoying part of the whole thing. And nothing like this exists as a general-purpose thing today, so someone would have to actually build it. For what it's worth, if any of it turns out to be useful as a starting point rather than something to write from scratch, my projects' keysharp-inputd and the shell extensions are already BSD 2-clause, so if you wish to reuse them then you can do so. I don't have much interest in doing the split myself at the moment, but if it grew into an usable library then I could possibly switch to it myself too. libinput vs evdevlibinput does genuinely solve device classification for you, which is annoying in raw evdev — a gaming mouse advertises a full keyboard, a laptop has three or four nodes that all look like keyboards, power buttons show up as EV_KEY devices, and so on. But there are a few behaviours it changes that I think are worth knowing about first:
None of these are blockers, I'd just document them rather than let people discover them. The X11 backendI'd think twice about moving X11 onto libinput and uinput as well, and not really because of the privileges — the bigger issue I think is that it changes what the hook is able to see at all. XRecord records at the X protocol level, so as far as I know it sees everything the server dispatches, including events injected with XTest. An evdev or libinput hook only ever sees physical hardware, so every X11-level synthetic event becomes invisible to it: There's a smaller fidelity point too. XTest injection goes through the X server, so it respects the server's pointer button mapping and lands as ordinary core events, and What the change does genuinely buy on X11 is injected-event detection, since if you inject through your own uinput device you can tag it, whereas XTest events are indistinguishable from physical ones. That's a real gap and it's one of the reasons I gave up on X11 myself, so I do understand the appeal — but it's a fairly specialised need, and I don't think it's worth making every existing consumer require root just to keep working. So what I'd suggest is keeping XRecord + XTest as the default on X11 and offering the libinput/uinput path as opt-in for applications that specifically want injected-event detection, or want identical behaviour on both session types. I'd also have Detecting your own injected eventsI didn't see this mentioned in the plan and I think it's fairly important. Once the uinput simulator and the libinput hook are running in the same process, your own simulated events come straight back into your own hook, and as far as I can tell there'd be no way for the application to tell them apart from real input. Anything that simulates input in response to input runs into this — a macro where one key triggers a sequence of keystrokes, a turbo/auto-repeat feature, or even just an application counting keystrokes and silently counting its own. In the case where the response includes the trigger key itself it's an infinite loop. On Windows So the virtual devices need to be tagged and that needs to be exposed on the event. I identify my own device by name and vendor/product together rather than name alone (see the keyd note below for why), and libinput should give you what you need via One distinction that turned out to be useful in my case: I suppress the pass-through replay events from re-entering the hook, since those are just the user's own physical events being let through and re-reporting them would be a duplicate, but I deliberately don't suppress events from an explicit simulation request — I only flag them. That way the application can decide for itself whether its own synthetic input should trigger its own handlers, which is sometimes wanted and sometimes not. uinput gotchasDevice visibility. This one is silent and took me a long time to track down. My beta lockout above actually had two causes, and the second was that events replayed on my uinput node were unreadable by the session's display server for users who aren't in the I think this applies to your simulation path too even though you're not grabbing, because a uinput device created by a root helper is Device creation latency. Several applications at once. Reference-counting the devices within one instance makes sense, but the refcount is presumably per-process, so ten applications using SharpHook would mean ten sets of three virtual devices, each one costing a udev event and a device-added round trip in every compositor and libinput context on the machine. What I'd worry about more than the clutter is identity: if every instance registers its devices with the same name, vendor and product, then a hook in one application can't tell its own injected events from another SharpHook application's, which makes the tagging I mentioned above a lot less useful than it looks. Putting something per-instance into the device identity, like the pid in the device name or a per-instance product id, would fix that fairly cheaply. It might also be worth checking what happens when two plugins in the same process each load their own copy of the native library — I'd assume Other grab-and-replay tools. This was the first and actual cause of the lockout: Batch atomicity. Windows Absolute mouse positionI think this might be a bit too pessimistic. There's no portable way, but there is a way on most of the desktops people actually run, and I ship all of these:
The GNOME and Cinnamon ones need a shell extension, which is a reasonable thing for an application to ship and an unreasonable thing to ask of a library. So I'm not suggesting you implement all of these — more that you could expose a pluggable position provider interface so applications can supply one, with the evdev/absolute-device path as the default and a clean "unsupported" otherwise. That turns it from impossible into merely inconvenient for your users at almost no cost to you, and it's more useful than a hard KeyTyped and xkbcommonI do this on both X11 and Wayland with xkbcommon and it's honestly not that bad, maybe a day of work for the main path (build the keymap, resolve the keysym at the current group/level, convert to UTF-8) plus a longer tail. The tail is mostly dead keys — you need a table mapping dead keysyms to a (combining mark, spacing form) pair, then NFC-compose with the following base character and fall back to the spacing form when the sequence doesn't compose, so I do think it's worth doing eventually since it's what makes text expansion possible on top of the library, but I agree it isn't something to block v8 on, and I strongly agree with making it opt-in everywhere. PermissionsTwo things I'd add to that section. The first is active session gating. With raw The second is that the broker approach also answers the objection you raise about udev rules yourself — that granting your own user access means every process you run can keylog. A broker can authenticate its clients with Smaller things
ConclusionI think the crux of the issue is that Linux, compared to Windows and macOS, is fragmented and doesn't have a centralized API to implement most of libuiohook functionality. Possibly not the response you want, but I think the solution is a separate project to do that, and then let libuiohook (and Keysharp?) consume that. Possibly even two separate projects, because I couldn't find a good pre-existing library to manage permissions with enough granularity and a good UI. Let me know which direction you want to take, I'm happy to go into more detail on any of this ;) |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TL;DR
SharpHook currently doesn't support Wayland on Linux at all, and I'm planning to implement this support in the nearest future. This discussion describes the fundamental difficulties and solutions for them. Note that the initial Wayland implementation will not support some features - some of them will most probably be implemented later, and some of them will probably not be implemented at all. These changes are going to be in the next major version - v8.
Important: Starting with version 8, SharpHook will require elevated privileges on Linux for both X11 and Wayland by default. There will be a way to revert to the pre-8 mode which works only on X11, but it wil be regarded as a 'legacy' mode.
Also note that none of what is described here is final. A big part of this plan was created with the help of Claude Code, and even though it looks plausible, I can't be 100% certain until I actually implement it.
Context - Why This Is Difficult
Wayland is the last major piece of the cross-platform puzzle that's missing from SharpHook. And the main reason is that it's damn hard to do global input listening and input simulation on Wayland. See also the issue in the original libuiohook repo for details.
The main problem is that Wayland is just a protocol, not a display server like X11. Or to be more precise, it's a collection of protocols, and it's also wildly incomplete. Each Wayland compositor can choose which protocols to implement and how. For things that Wayland doesn't cover, each compositor has its own way of doing it.
Another problem is that Wayland is very restrictive - much more restrictive than X11. There is no protocol to listen to input events that are intended for other applications, and while there is a protocol for simulating events, it doesn't cover SharpHook's needs (for example, compositors can decide to ignore events simulated by SharpHook). There is absolutely no way to get the current global mouse position. And many more restrictions. This is great for users in that applications have much less wiggle room to do shady things, but it's very frustrating for application developers.
This is why we need to move to a lower level to handle input events and simulate them, and this requires elevated privileges (but not necessarily
rootaccess).Global Hooks and libinput
I've decided to use libinput as the engine for the global hook. It's installed on pretty much all modern Linux distros except probably some very esoteric ones. Another option was to use evdev, but libinput simplifies a lot of stuff, so I've decided to stick with it instead.
libinput requires access to
/dev/inputto work and there are several ways to give your application access to it. More on that below.Unfortunately, there is no way to get absolute mouse coordinates on Wayland without some drastic hacks which are user-noticeable. libinput only posts relative mouse motion events for mice (it can post absolute mouse motion events, but they are posted by touchscreens or by virtual machines usually, not by normal mice). So one major change will be that on Wayland, SharpHook will not raise events of type
MouseMoved- instead, it will raise events of typeMouseMovedRelativeToCursor. Your application won't have access to absolute mouse coordinates unless you implement getting them yourself.Since absolute mouse coordinates won't be available, mouse button events and mouse wheel events will not provide coordinates meaning that your application will not know where the mouse has been clicked.
Another missing feature will be that events of
KeyTypedwill not be available on Wayland, at least for now. I'm planning on making these events disabled by default everywhere since they cause system-wide side effects on Windows and X11, so care should be taken when using them anyway. I think it's possible to derive the character from the current keyboard state and the pressed key usingxkbcommon, but the planned Wayland implementation is complex enough as it is, and this feature is not part of core SharpHook functionality, at least in my opition. Note that while XKB started out as an X11 extension, it can be used without X11 and indeed all Wayland compositors use it for keyboard-related stuff.Event Simulation and uinput
Events can be simulated by writing to
/dev/uinput. This also requires elevated privileges. More on that below.Simulating events via
uinputwill require registering virtual devices (three of them - a keyboard, an absolute pointing device, and a relative pointing device). The lifecycle of virtual devices can be encapsulated in theEventSimulatorclass. Currently, an object of this class has basically no purpose other than being a collection of simulation methods, but this will change.Currently,
EventSimulatorcan be created usingnew:In the next version,
EventSimulatorwill only be creatable using the staticCreatemethod and will implementIDisposable:Setting up virtual devices and waiting for the OS to pick them up is quite slow, so ideally, an
EventSimulatorshould be created early in the application lifespan and reused. Event devices will be reference-counted in libuiohook, so creating a secondEventSimulatorwhile the first one is active will be cheap as it will not set up a second group of virtual devices.Text entry simulation will also be absent in the initial release. The implementation on X11 is already horrible, and on Wayland, it will be even worse, so I have to think it through separately. I think it should be possible to do, but no promises on the timeline.
System Properties
The
CreateScreenInfofunction (and the global hook itself) will use Wayland protocols directly to enumerate and keep track of screens, their sizes, and coordinates. It will use thexdg-output-unstable-v1protocol to support fractional scaling if this protocol is available (which it should be on all major Wayland compositors).GetAutoRepeatRateandGetAutoRepeatDelaywill get this info from Wayland directly.GetMultiClickTimewill return 400ms as a hard-coded value since Wayland does not provide this info at all, and this value is the multi-click time on GNOME and KDE.GetPointerAccelerationMultiplier,GetPointerAccelerationThreshold, andGetPointerSensitivitywon't work on Wayland and will always return-1as Wayland doesn't provide any way to get this info.Changes to the X11 Mode
On X11, SharpHook will also use libinput and uinput (unless you opt into the legacy mode). This means that SharpHook will also require elevated privileges where it previously didn't. This is a deliberate design decision. Firstly, this will make the Wayland and X11 implementations very similar to each other. You won't have to think about when to set up access to libinput or uinput - you will always have to do it. Secondly, this library is intended for legitimate applications, not anything shady, so I don't support having arbitrary access to input events for any application. This is similar to the model macOS uses - the user must explicitly opt into this functionality and it is up to you as the application developer to convince the user that your application deserves it.
Since libinput posts only relative mouse movement events for mice, the global hook will get the current absolute mouse position using
XQueryPointer. This runs the issue of not being synchronous - the global hook will get the mouse pointer position at the time of the hook's execution, not at the time of the event, which may drift. I think this is acceptable since once the motion stops, the last event will report the correct position, and it is usually the one that interests us.Other functionality should remain unchanged on X11.
Selecting the Mode on Linux
SharpHook will provide four different libuiohook modes or back-ends for Linux:
SharpHook will provide 4 libuiohook files on Linux instead of just one:
liuiohook.so will be a very thin wrapper and will simply load one of the three back-end files and redirect all function invocations to it depending on the selected mode. Selecting the mode will be possible using the new
SetLinuxBackendmethod. Importantly, once the wrapper has loaded one of the files, it will always redirect to it, so selecting the mode must be done before using any other SharpHook functionality.If you use the new modes, you can just delete libuiohook-legacy.so when packaging you application as it will never be used. If you use the legacy mode, you can delete libuiohook-x11.so and libuiohook-wayland.so, and you can even delete libuiohook.so and rename libuiohook-legacy.so to libuiohook.so so that there is no runtime wrapper. Each of the four files has an identical ABI.
The legacy mode may become obsolete in the future to simplify maintenance as Wayland becomes more popular, though there are no guarantees on the timeline or the very fact that it happens anytime soon.
Suppressing Events
In short, suppressing events won't be available on Linux in the next version, and will probably not be available in the future at all.
The original libuiohook will use a different strategy for Wayland support and it will support suppressing events, but I believe that the price for this is too high. It will use evdev instead of libinput. Using evdev is lower-level so more work is needed to set up a global hook using it, but it has a feature that libinput doesn't - grabbing devices. When a device is grabbed, only the process that grabbed it receives events from this device. The strategy in the original libuiohook is the following:
I believe that this is absurd. The price of a bug may be as large as bricking the user's system and requiring a restart because no input event may be processed at all for whatever reason (and let's hope that your application doesn't automatically run at login). There are no OS-level guardrails - if the hook runs for 1 second, then all user input is delayed by 1 second with no way to unregister a running hook for misbehaving. Thus, I believe that this feature is not worth it, and as far as I know, there are no other ways to suppress events on Linux.
Querying Feature Support
You won't need to check for feature support based on the current platform, SharpHook will provide a way to know whether a specific feature is implemented. In the next version, there will be the following features:
KeyTypedevents - enabled for Windows, macOS, and X11, disabled for WaylandPermissions
As stated previously multiple times,
/dev/inputand/dev/uinputrequire special permissions that users don't usually have. There are multiple ways to add them for your application.The easiest one is to run your application as
root. This is not recommended as users will need to unconditionally trust your application and that it won't be compromized.You can instead create udev rules which grant access to
/dev/inputand/dev/uinputto non-rootusers. You can grant these permissions to your own user, but this will mean that any application you run will be able to listen to all events and simulate events. You can also create a separate user specifically for your application, grant access to that user, and then run your application as that user, but that will complicate other stuff for your application.Or you can use the
inputgroup to control access to/dev/input, but udev rules are a more modern alternative, and you will need to use them for/dev/uinputanyway.SharpHook will also provide a way to cut into the way libinput opens the input files, and how virtual devices are created for highly customized solutions, but it's super advanced, and you probably won't need it. For example, this will allow to create a daemon that runs as
rootand provides accessible file descriptors to your application, and then SharpHook will use those instead of opening the files directly.Ultimately, SharpHook cannot recommend one way over the other since this is up to application developers to actually implement the needed plumbing like installer scripts which grant the necessary permissions.
Closing Thoughts
Implementing all this for Wayland is going to be hard. Using it will also be hard, but I don't see other ways to do this. At least I have a plan now, but it's still just a plan, it can still change.
Tell me whether you have any thoughts regarding any part of this plan. Maybe, you will have better ideas - after all, I'm not an expert on the Linux event system at such a low level, almost all of this is new to me.
All reactions