Wacom Tablet and Pen support issues

[ysooqe]

I would like to report a few (critical) issues with the new graphic tablet support.
I will try to give as much details as possible. Unfortunately, I am not a developer, so I don't know how to fix those issues and can only report them as best as I can.
First of all, some information about the setup:

Sway version 1.2-rc1-61-g352a3e1f (Oct 8 2019, branch 'master')

running on Arch Linux (using all the latest packages and the wlroots-git package from the AUR in order to build sway)

Tablet and Pen Wacom Intuos S with Pen (2018 Model)

Application to test tablet functionality Xournalpp (installed from the Arch Linux Community repo), using this since it is the only application I know of which runs natively under Wayland

I have built sway from the current state of the master branch and ran it from a tty.
Since this issue ticket regards one specific pull request (#4570) but there is more than one issue, I am listing the issues numerically and divide each issue into "expected behaviour", "actual behaviour" and "steps to reproduce" (it is assumend that the latest state of the master branch of sway is built and used, the tablet is connected via bluetooth and the first step is always to open Xournalpp).

Here are the issues I am facing:

1. First and foremost, for some reason, the "resolution" (for the lack of a better word) the tablet is drawing in, is really bad. So what happens is, that when there are lines drawn very quickly, the line that is drawn is just a succession of straight liner rather than a smooth curve. To see what I mean, look at this
   
   expected behaviour
   quick movements of the pen draw smooth curves
   actual behaviour
   straight consecutives lines are drawn
   steps to reproduce
   draw curves and lines very quickly and with a swift motion

2. For the buttons on the pen itself, the order is "reversed", so what is actually the first button on the pen is recognised as the second button by the application and the second button is recognised as the first button. So by that I mean, the first button is the one closest to the tablet. After setting this to Eraser in Xournalpp and setting the second button to the Hand-Tool, using each button it can be observed that button one is assigned to the Hand-Tool and button two is assigned to the eraser.
   expected behaviour
   buttons are assigned in the correct order
   actual behaviour
   button assignment is reversed
   steps to reproduce
   set the pen buttons in Xournalpp accordingly as test which button does what

3. This is the biggest issue so far: After assigning an action to the button(s) and using them, things start to "fall apart". The CPU usage spikes, everything becomes very unresponsive and slow and in the end its a hit or miss if the action that was intended is actually carried out, i.e. for example pressing the button to erase and then actually erasing some of the drawings. It is really hard to explain what I mean by that, therefore I tried to record a log as instructed.
   Sometimes, this does produce a crash of Xournalpp and even sway itself, but unfortunately, these crashes dont happen consistently, so I can not reproduce them reliably. I will therefore provide a stack trace as soon as I can find out how to reliably force this crashing.
   expected behaviour
   when using pen buttons, actions are performed as expected and everything works smoothly
   actual behaviour
   when using pen buttons, everything becomes unresponsive, the movement of the cursor itself lags behind and is slow and abrupt instead of smooth, quick and continuous, actions are not performed as expected, for example erasing using a button or moving the page is not working properly and sometimes not working at all
   steps to reproduce
   assign custom functions to the buttons of the pen in the settings of Xournalpp and try using them extensively

4. Pressure sensitivity, although supported by the tablet and pen and application, is not working.
   expected bevaviour
   pressure sensitivity is working as expected once it is enabled in the settings
   actual behaviour
   pressure sensitivity is not working, putting pressure on the pen does not change the thickness of the lines drawn
   steps to reproduce
   enable pressure sensitivity in the settings of Xournalpp

Please let me know if there is anything else that I could provide in order to help.

EDIT: Logs for issues 1 and 2: Log
For some reason, I can not reproduce issue 4 at the moment, so pressure sensitivity is working. I have not change anything since first reporting the issue(s), my computer has been in sleep-mode since then and I have not done anything else in between. I attached a log anyway and will update this once I can get a log when it is not working as expected.

[ddevault]

cc @jchv

[emersion]

Would be nice to collect WAYLAND_DEBUG=client logs for each of these issues, to figure out whether it's a sway bug or a client bug.

[ysooqe]

Would be nice to collect WAYLAND_DEBUG=client logs for each of these issues, to figure out whether it's a sway bug or a client bug.

So I start sway with WAYLAND_DEBUG=client sway -d 2> ~/sway.log and post that log for each action?

[emersion]

Just start the client with WAYLAND_DEBUG=client and reproduce the issue. Ideally, only post the part of the logs that are printed when you reproduce (skipping the part where the window is opened for instance).

[jchv]

Unfortunately, some of these problems may be related to libinput. It would be helpful to run sudo libinput debug-events and log some of it. If it looks like your tablet is being handled poorly in libinput, we’ll need to pursue fixes upstream.

[ysooqe]

Just start the client with WAYLAND_DEBUG=client and reproduce the issue. Ideally, only post the part of the logs that are printed when you reproduce (skipping the part where the window is opened for instance).

Also, I am really sorry, but I still dont understand what you mean by "the client". The client as in the sway instance?
Since the log files are quite big and I have absolutely no idea what exactly I am looking for, I do not know what the parts are that are printed when the issues happen :(
I will record the logs for each situation and update the initial ticket respectively.

Unfortunately, some of these problems may be related to libinput. It would be helpful to run sudo libinput debug-events and log some of it. If it looks like your tablet is being handled poorly in libinput, we’ll need to pursue fixes upstream.

I have done that, here is the log for issue 3 of the mentions issues. I have reproduced issue number three here and right after just closed the application and quit sway. The link contains both the log from libinput as well as sway.

[emersion]

Also, I am really sorry, but I still dont understand what you mean by "the client". The client as in the sway instance?

The client is the app you're using, Xournalpp.

WAYLAND_DEBUG=client xournalpp

[ysooqe]

First of all, sorry for the delay.
I tried to record some more logs today.
There I discovered, whenever I start Xournalpp with WAYLAND_DEBUG=client xournalpp, perform an action like drawing and then close the application with the sway keybinding (alt+shift+q), sway crashes. I tried to get a stacktrace of that. Since I have no idea what I am doing there or what that even is, I hope the information it provides is sufficient. I followed the instructions given when trying to open a new issue ticket.

I really want to help to improve this as best as I can, i.e. provide logs that actually help.
So before recording and posting endless lines of logs, I just want to confirm that the stuff I do record is actually information that can help you:
I start sway from a tty with Projects/sway/build/sway/sway -d 2> ~/sway.log
Then I open a terminal and open Xournalpp from it with WAYLAND_DEBUG=client xournalpp
Then I perform the action that is supposed to be logged. Then I close Xournalpp and sway.
After that I have a sway.log containing thousands of lines of information.
I have updated the initial ticket description with those logs. If this is not sufficient or does not help in any way, please let me know.
I am sorry if any of this is not the correct way of doing things, please let me know what should be done different in order to get to a solution.

[jchv]

I haven't forgotten about this issue, btw. I was able to crash things by pressing tablet pad buttons rapidly and just generally trying to cram a lot of inputs in. It seems to crash XWayland too and I'm kind of curious if maybe wlroots is the culprit in this one. I don't really see a whole lot of code on the Sway end that's suspect.

[hexd0t]

Any progress on this?
Running a number of sway builds including the new pen support is both awesome when it works but also tends to crash clients and or sway, as soon as my pen enters or leaves the range of the digitizer (n-trig with a Surface Pen 4 on a 2017 HP Spectre x360) - anything one can do to help resolve this?

[emersion]

You can help by making sure all crashes are reported, debugging them and sending patches to fix them.

[jchv]

I'm currently at a loss on these. Usually, it doesn't crash on my end, but when it does, it does so in a chaotic way that changes depending on whether valgrind is running. I can cause it to crash by doing something weird like spamming the tablet pad buttons a lot, but if I'm not doing that things are actually pretty stable for me.

Being able to narrow it down to a specific piece of code would be great, but I've not gotten any progress on doing so.

[jchv]

Hmm, well, this is interesting. I just pulled master again and tried to reproduce a crash. However, no matter what I did, it didn't seem to break at all. Everything now seems to work reliably...

I'm not really sure what might've happened.

[jchv]

Things crash a lot less now. It seems to be trivial to crash Krita's splashscreen by spamming tablet pad buttons. Usually Xwayland crashes, infrequently Sway crashes shortly after Xwayland does.

Here's a log with SWAY_DEBUG=1 and backtrace. https://gist.github.com/jchv/03acbfee73da34dd571d6e5c3e7607c5

Digging through, it happens because of this line:

	elm->prev = list;
	elm->next = list->next;
	list->next = elm;
>	elm->next->prev = elm;

...in wl_list_insert, called by wl_signal_add, called by xwm_set_seat. It seems that elm->next is NULL at that time, leading to a NULL pointer dereference. Anyone's guess as to how the list gets corrupted with a NULL pointer, however- I have no idea how that might happen. I don't think there's any data races to worry about because everything is single threaded (right?) and I don't really see what else could cause this. I know adding something to a list twice will corrupt it, but for one thing, I can't see where that might happen at to begin with.

I suspect this is probably not directly caused by whatever causes Xwayland to crash. Xwayland crashes in wl_proxy_marshal:

#0  0x00007fe0efb1ebe0 in raise () from /nix/store/qn76sklvyalzw9ilnxz6sh0020gl2qn6-glibc-2.27/lib/libc.so.6
#1  0x00007fe0efb1fdc1 in abort () from /nix/store/qn76sklvyalzw9ilnxz6sh0020gl2qn6-glibc-2.27/lib/libc.so.6
#2  0x000000000059f2aa in OsAbort ()
#3  0x00000000005a3c93 in AbortServer ()
#4  0x00000000005a4ad9 in FatalError ()
#5  0x000000000059c541 in OsSigHandler ()
#6  <signal handler called>
#7  0x00007fe0f0221167 in wl_proxy_marshal () from /nix/store/641prxh0dppg0d0cd1hkhcs83sq75g7b-wayland-1.17.0/lib/libwayland-client.so.0
#8  0x00000000004487c0 in xwl_seat_set_cursor ()
#9  0x0000000000448aa9 in xwl_set_cursor ()
#10 0x000000000048f85e in miPointerUpdateSprite ()
#11 0x000000000048fb46 in miPointerDisplayCursor ()
#12 0x000000000047c50b in CursorDisplayCursor ()
#13 0x00000000004ffd27 in AnimCurDisplayCursor ()
#14 0x000000000057076b in ChangeToCursor ()
#15 0x0000000000574772 in CheckMotion ()
#16 0x000000000057488a in WindowsRestructured ()
#17 0x0000000000592725 in MapWindow ()
#18 0x0000000000561a5b in ProcMapWindow ()
#19 0x0000000000567ab3 in Dispatch ()
#20 0x000000000056ba46 in dix_main ()
#21 0x00007fe0efb0bb8e in __libc_start_main () from /nix/store/qn76sklvyalzw9ilnxz6sh0020gl2qn6-glibc-2.27/lib/libc.so.6
#22 0x0000000000441d7a in _start ()

Krita also crashes, but I suspect that is a red herring and it is crashing exactly how it would if the X11 server disappeared in other circumstances. It dies inside an exit handler.

#0  0x00007f6112adea60 in KoResourceItemChooserSync::baseLength() () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritawidgets.so.18
#1  0x00007f611419df35 in KisViewManager::~KisViewManager() () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritaui.so.18
#2  0x00007f611419e479 in KisViewManager::~KisViewManager() () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritaui.so.18
#3  0x00007f611415f2d0 in KisMainWindow::~KisMainWindow() () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritaui.so.18
#4  0x00007f611415f379 in KisMainWindow::~KisMainWindow() () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritaui.so.18
#5  0x00007f61141781cc in KisPart::~KisPart() () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritaui.so.18
#6  0x00007f6114178369 in (anonymous namespace)::Q_QGS_s_instance::innerFunction()::Holder::~Holder() () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritaui.so.18
#7  0x00007f610f8fa351 in __run_exit_handlers () from /nix/store/qn76sklvyalzw9ilnxz6sh0020gl2qn6-glibc-2.27/lib/libc.so.6
#8  0x00007f610f8fa43a in exit () from /nix/store/qn76sklvyalzw9ilnxz6sh0020gl2qn6-glibc-2.27/lib/libc.so.6
#9  0x00007f610c16e5d0 in ?? ()
   from /nix/store/ah81nhbv1d97jgwdcl122dfymwyclclg-qtbase-5.12.5-bin/lib/qt-5.12.5/plugins/platforms/../../../../../kspf6ryzdxvr986qfi2fa02ij5lvks4p-qtbase-5.12.5/lib/libQt5XcbQpa.so.5
#10 0x00007f610c1a1943 in ?? ()
   from /nix/store/ah81nhbv1d97jgwdcl122dfymwyclclg-qtbase-5.12.5-bin/lib/qt-5.12.5/plugins/platforms/../../../../../kspf6ryzdxvr986qfi2fa02ij5lvks4p-qtbase-5.12.5/lib/libQt5XcbQpa.so.5
#11 0x00007f610f0b3d1d in g_main_context_dispatch () from /nix/store/msvdi3zrfdxah6arc1gahi9m2rv9qdci-glib-2.62.1/lib/libglib-2.0.so.0
#12 0x00007f610f0b3fb8 in g_main_context_iterate.isra () from /nix/store/msvdi3zrfdxah6arc1gahi9m2rv9qdci-glib-2.62.1/lib/libglib-2.0.so.0
#13 0x00007f610f0b404c in g_main_context_iteration () from /nix/store/msvdi3zrfdxah6arc1gahi9m2rv9qdci-glib-2.62.1/lib/libglib-2.0.so.0
#14 0x00007f611028fda4 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /nix/store/kspf6ryzdxvr986qfi2fa02ij5lvks4p-qtbase-5.12.5/lib/libQt5Core.so.5
#15 0x00007f61141353fa in KisApplication::start(KisApplicationArguments const&) () from /nix/store/w8hvrb1wn8xjzhyc624jg5w93ma7nqm1-krita-4.2.7.1/lib/libkritaui.so.18
#16 0x0000000000407fb9 in main ()

I am no longer able to crash Xournalpp running as a Wayland application. I am not sure what changed on that front.

[emersion]

Can you try compiling Sway, wlroots and Xwayland with debugging symbols to get better stack traces? Also the Sway stack trace is missing.

Once you get that, please report the Xwayland bug upstream.

[jchv]

FWIW, I haven't been able to reproduce this at all anymore. There is no more segfaults or weird behaviors when mashing tablet buttons. I have not experienced a random crash in over a month, and I've been actively using my graphics tablet with Sway. I suspect the bug was probably upstream, but I am not sure where: It couldn't have been XWayland, which hasn't had remotely relevant changes for months.

Sadly, having failed to triage it, it's hard to really know. This is, of course, my fault. I am still kind of clumsy with NixOS and simply was not able to figure out exactly how to get debug info for all packages.

Is anyone else still having trouble with Sway 1.4 and updated libinput/XWayland? Does anyone have a hunch for what may have fixed this? It may be time to close this bug if not.

[ysooqe]

Is anyone else still having trouble with Sway 1.4 and updated libinput/XWayland? Does anyone have a hunch for what may have fixed this? It may be time to close this bug if not.

I just updated everything and tried it.
Some of the issues remain unfortunately. I still have issue 1 of the initial ticket comment and a few other weird things. For example, regardless of the settings in Xournal, the pointer is not properly displayed (visual glitches, I could upload a recording of the screen if that would help somehow). Also, when trying to move the page up and down using the "hand tool" through the buttons and controlling the movement by moving the pen over the pad, it gets really unresponsive and takes at least a second to react to the input (CPU usage spikes aswell).

[jchv]

Hmm. I just tried Xournal++ and I am not seeing any issues.

This bug was tricky to begin with, so maybe it's just not manifesting for me right now/anymore. But I would keep an eye out for software updates (for related components.) I'm searching to see if I can figure out what has changed with my setup since the last time I crashed sway with tablet.

[Eragonfr]

Some of the issues remain unfortunately. I still have issue 1 of the initial ticket comment and a few other weird things. For example, regardless of the settings in Xournal, the pointer is not properly displayed (visual glitches, I could upload a recording of the screen if that would help somehow). Also, when trying to move the page up and down using the "hand tool" through the buttons and controlling the movement by moving the pen over the pad, it gets really unresponsive and takes at least a second to react to the input (CPU usage spikes aswell).

I'v a Wacom Intuos S and I have this bug too, but using any GTK+ app with GDK_BACKEND=1 but if I start the app in Xwayland, the pressure is fuck*d but... visual glitch and hight cpu usage disappear.
I'v tested my tabled and the pen with, Firefox, Sublime text, MyPaint and waybar and in all of these apps the glitch appear.
But with GIMP and Blender the glitch didn't appear.
So I don't know if it's related with GTK+, I think it is but somehow GIMP and Blender didn't have the bug.

[cristobaltapia]

@Eragonfr I have the same Wacom model and was seeing the same visual glitches in MyPaint and Xournalpp. A solution for me was to attach the wacom pen to its own seat with the following configuration:

seat seat_wacom attach "1386:788:Wacom_Intuos_Pro_S_Pen"
seat seat_wacom attach "1386:788:Wacom_Intuos_Pro_S_(WL)_Pen"
seat seat0 hide_cursor 10000

(Note: adapt your identifier if needed)
Note also that simply reloading sway sometimes does not change the seat of the input device, so you have to restart to be sure it works.

With that I get two different pointers: one for the normal mouse and one for the pen. Since the normal cursor will disappear after 10 seconds, you will only see one cursor after that time. You also wont be seeing those visual glitches. That was my experience at least.

However, some other problems appear with this configuration. The most annoying of them is that it somehow screws up the swayidle behaviour. Now, even if I am using my keybord the whole time, swayidle will kick in and lock my session. Not cool. So I had to disable it. Another related problem is posted in #5169.

[Eragonfr]

Thanks for sharing this solution, it's not what i'm expecting from sway(because i want only one cursor) but that work.

[cristobaltapia]

Good to know it worked for you. Yeah, I also think it is a strange behaviour. But at least I got to learn how to use seats :P

[Xyene]

Hi all,

There's a lot of issues in this thread that make it hard to track what's been fixed and what's still a problem. It would be great for organization purposes if separate tickets for each issue that is still present with sway/wlroots master could be opened instead.

A number of fixes for tablet input got merged recently, that should address some of the issues noted in this thread.

First and foremost, for some reason, the "resolution" (for the lack of a better word) the tablet is drawing in, is really bad.

Pressure sensitivity, although supported by the tablet and pen and application, is not working.

I cannot reproduce these in sway/wlroots master with Xournal++ 1.0.18, is this still happening?

For the buttons on the pen itself, the order is "reversed", so what is actually the first button on the pen is recognised as the second button by the application and the second button is recognised as the first button.

I can reproduce this, but I'm inclined to say this is an issue on Xournal's end. We send button 1 and 2 events (as per WAYLAND_DEBUG), but these are interpreted by Xournal in reverse order. The same thing happens on GNOME, though, so it's not sway-specific (and at least easy to work around).

After assigning an action to the button(s) and using them, things start to "fall apart".

Does this continue to happen? Unfortunately I can't attempt to look into this, as this issue with my tablet precludes actually using the buttons for anything.

I suspect this is probably not directly caused by whatever causes Xwayland to crash. Xwayland crashes in wl_proxy_marshal:

This sounds an awful lot like https://gitlab.freedesktop.org/xorg/xserver/-/issues/709, which should be fixed in Xwayland master as of 3 weeks ago.

Another related problem is posted in #5169.

This has been fixed in sway master, particularly #5240.

[jchv]

Thank you so much for looking into these issues and for fixing so many of them. I'll admit my debugging skills are just not quite sharp enough to understand some of these issues and I struggled to root cause a lot of the problems.

I can at least confirm that the Xwayland fix for pressing Wacom tablet buttons appears to have resolved that issue for me. I'll update my system to Sway master later on and torture test the tablet integration to see if I can see any issues.

[Geo25rey]

On my Lenovo Thinkpad L380 Yoga, the included pen and touch screen work flawlessly. The only config command I included is below (not sure if it is necessary). The buttons on the pen seem to work as left and middle click for the first and second button respectively. Xournal++ senses the amount of pressure I use when writing with the pen and when I use the first button it switches to the eraser temporarily as intended. Regarding the touch screen, long press registers as a right click too. Based on previous comments on this issue, I was expected a much worse experience. I'm curious to see if others are having a similar experience.

seat seat_pen {
        attach "1386:20823:Wacom_Pen_and_multitouch_sensor_Pen"
        hide_cursor 100
}

Edit: Forgot to mention that touch screen scrolling works as intended.

[Xyene]

I'm going to go ahead and close this issue in favor of filing one ticket per any outstanding issue. It seems like there aren't that many left 🙂