Surface Pro 3, 4 (N-Trig hardware): pressure sensitivity stops working after moving pen away from screen

[crydalch]

The latest changes have resolved some strange problems, thanks everyone!

Now, it seems the last major hurdle is getting pressure sensitivity to work. I realize this very well could be related to Gtk/Gdk upstream, but figured I'd open it up here:

If my stylus isn't near the screen, when I launch MyPaint, pressure sensitivity works! But once I move the pen away from the screen, I loose pressure sensitivity.

Any ideas where this might be coming from? I see two devices in the preferences: System Aggregated Pointer and N-trig DuoSense device Stylus. This is under Windows 10, btw.

Thanks!

[achadwick]

Not a clue, sorry. I don't use this hardware, and I don't run Windows 10.

Please can you paste in the versions of MyPaint you have tried, and the version of your "N-trig DuoSense device Stylus" driver. You will find both under "Add or remove programs" on Windows 7.

If you are using the very latest MyPaint beta, could you screencap the Devices tab of the Prefrences window after using all styluses, stylus ends, and all input devices on your system to draw something? Thanks.

Does it work in the GIMP (and if so, which versions?)

Please search your C: drive for "wintab". Do you have a wintab.dll or wintab32.dll or wintab64.dll on your C:\ drive? This is necessary, IIRC. The Wacom drivers park theirs under C:\Windows\SysWOW64 and C:\Windows\System32 for Windows 7. Ignore any in temp areas.

[supertobi]

I have a Surface Pro 3 with Windows 10, too and the pressure sensitivity works but the drawing in MyPaint is extreme slow.

@crydalch
You can get the wintab driver for the pen here:
https://www.microsoft.com/en-us/download/details.aspx?id=38826

[achadwick]

@supertobi lag issue discussion goes in #390, thanks! There is an experimental build I'm trying to get everyone to try as a fix for that bug, and feedback would be really useful.

[zsgalusz]

I am also experiencing the same issue with pressure sensitivity on the Surface Pro 4, using the latest beta (1.2.0-beta.3, both 32 and 64 bit versions). Using all the default drivers that shipped with the device and additionally the wintab drivers for the SP4 from Microsoft's website.

Screenshot for the device tab:

I disabled touch/mouse for painting but the pen pressure issue was the same even with it on.

I am not familiar with Gimp so I'm not sure whether it works there - I tried fiddling with the brush dynamics settings and I think it doesn't recognize the pressure but I might just be doing something wrong.

[achadwick]

This looks like one of those vague upstream issues that plague us constantly. And well, I don't have hardware available to test this myself. Let's see if we can uncover more about this popular hardware combination.

@zsgalusz I don't know if that's relevant to this issue - it might be a newer bit of hardware. Can you look in your devices list and pull out some information about the device in question, like hardware IDs or what Windows calls it?

@crydalch Are you able to retest against the most recent release (or a recent git revision)? v1.2.0 stable is available now. It contains some removal of proximity-in and proximity-out event handling since 2dd4812, and notably the transition you spotted is one that would be expressed as a proximity-out event. Sadly v1.2.0 had to be released with an earlier GTK due to the lag issue, so you may find it just doesn't work at all with your device.

You'll find that commit in MyPaint v1.2.0-beta.4 as well, which is built with the newer GTK.

[eduncan911]

Wanted to chime in saying I am experiencing the same issue (no pressure sensitivity). Followed a trick in another issue to reconfigure the "eraser" button to actually an eraser, so that now works. But, it did not at first.

Microsoft Surface 3 LTE
Microsoft Surface Pro 4 Pen (yes, it works with S3 as well)
Same N-Trig drivers for Surface Pro 3 and Surface Pro 4 (SP4 has 1024 levels of pressure though)
Windows Pro 10 x64, fresh install 1 week old
My Paint x64 1.2.0+git.f62444e.dirty <- same revision as 1.2.-0 stable I see. Odd version number though.

All 3 tablets use the N-Trig digitizer. And while they all use the exact same driver, the SP4 has increased levels of pressure to 1024. Again, same drivers for all three models: Surface 3, Surface Pro 3 and Surface Pro 4. The latest driver release for all models is November of 2015.

Windows Pro 10 x64 w/Nov 2015 update was a bare-metal fresh install recently.

Installed the November 2015 refresh of all drivers for the Surface Pro 3 and Surface 3 (they all use the same drivers).

Without WinTAB drivers, only the mouse is detected. Installing the latest Wintab_x64_1.0.0.20 drivers (the latest as of this posting, for both Surface 3 and Surface Pro 3 and Surface Pro 4) again shows the mouse - until you touch the canvas with both the Stylus and Eraser tips - then they both show up in the Preferences dialog.

And that Preferences dialog looks exactly as the same as the SP4 screenshot above: notice the "type" says "Pen" for both. The Surface 3 says the same thing.

Pressure sensitivity works in multiple other programs such as Sketch It and Fresh Paint.

On my Linux tablet (Lenovo ThinkPad Helix) running ArchLinux (Antergos distro) on tip with, it's Wacom digitizer works well (well, the eraser isn't always detected).

I wanted to note that on my Lenovo Helix, the Preferences page shows the "type" as Pen and Eraser separately - not as "Pen" and "Pen" as the SP4 screenshot above shows and my S3 shows.

^- Thinkpad Helix w/Arch Linux showing "Eraser" differently than S3, SP3 and SP4.

FYI: this Surface 3 will be wiped for ArchLinux (Antergos) as soon as 4.6 kernel comes out with the patches for the Surface 3 drm-intel chipset. Surface 3 Type Cover I am submitting patch for shortly.

But right now, the boss (wife) wants to draw - and I keep telling her that MyPaint is the best app for what she wants to do. Just no pressure sensitivity.

[eduncan911]

In addition to my last comment, in issue #420, you mention you are matching on:

As you can see in my Thinkpad Helix screenshot above, that is matched and recognized as an "eraser".

But, in the SP4 screenshot above (and in my Surface 3), the eraser is proper-cased "Eraser".

Maybe there's a regex ignore-case missing?

Because on my Surface before and after the format, the eraser never worked. I have to fight it to have it force an Eraser mode.

[achadwick]

@eduncan911
The code in question is here (I hope), and it uses re.I (ignore case): https://github.com/mypaint/mypaint/blob/v1.2.0/gui/device.py#L620. So if the device is called an eraser, it ought to be treated as an eraser (which actually doesn't mean much, beyond the initial selection of a device-specific brush for it).

The Type column is what GDK tells us. In this case, it is telling us that the device presents as a pen on Windows on the SP4 (:see_no_evil: Aha! "Surface Pro 4", not "Service Pack 4"... :stuck_out_tongue_closed_eyes:)

That's why gui.device.device_is_eraser() was written. Sigh. MyPaint should be treating this as a separate device on both platforms, and as a probable eraser. That's what your screenies are telling me, anyway. Are you able to flip over the physical pen and assign a different brush to the eraser end in the normal way on Windows?

(Anyway, this eraser discussion is a bit offtopic)

[achadwick]

Regarding pressure sensitivity dropouts on Surface Pro 3 and 4: this seems to affect GTK 3.14 (=MyPaint v1.2.0 stable for windows) thru 3.18-something-i-think (=MyPaint v1.2.0-beta.3 developer build for windows).

MyPaint is utterly reliant on the upstream GTK library's "GDK" module actually reporting pressure. The best people for reporting and chasing are you guys, since you actually own the hardware in question!

Please could somebody affected by this check the upstream tracker about this, and maybe report an issue there if they don't know about it yet? The hardware details above will doubtless be very useful to them. The initial search to get you started: https://bugzilla.gnome.org/buglist.cgi?quicksearch=surface+pro+pressure, but please try others. There's a bug in there which appears to have been fixed (bug 745697), somebody with one of these tablets should comment on it, since one of the screencaps above shows a detected device as Pen.

[achadwick]

I would like somebody to test whether 2dd4812 fixes this issue with recent GTK. Only v1.2.0-beta.4 contains both, or you could try building it yourself.

@zsgalusz
Can you check the proximity thing against the v1.2.0-beta.4 developer build as well please? Its GTK is recent and similar to v1.2.0-beta.3's, but its MyPaint bits contain 2dd4812. v1.2.0.beta.3 does not contain that commit.

The stable v1.2.0 contains the commit too, but its GTK is older (3.14.15).

[eduncan911]

@achadwick Is that commit 2dd4812 included with the latest stable 1.2.0 download from the Release page?

If so, that's what I am running (My Paint x64 1.2.0+git.f62444e.dirty) and it didn't work.

[achadwick]

@eduncan911
Yes, 1.2.0 is newer than 1.2.0-beta.4 and includes that commit. However as I said, testing with the stable MyPaint 1.2.0 build doesn't help diagnose this issue because its GTK is too old. @tumagonx built it with 3.14 for stability and non-lagginess.

Newer GTK includes tablet fixes. I suspect the fixes they included don't work perfectly, i.e. it seems pressure works until you take the stylus away from its pad (the "proximity thing"). But perhaps that only happens to MyPaint before 2dd4812? That's what I want someone to test.

All Surface Pro 3 and 4 users:
Please test with the developer build of MyPaint 1.2.0-beta.4 specifically. The build for that includes GTK 3.18.6, which may mean that it is laggy on your computer. However it also includes 2dd4812 which may fix the proximity issue.

[EDIT for clarity]

[zsgalusz]

@achadwick I just tested both 64 and 32 bit builds of 1.2.0-beta4, and both still exhibit the proximity issue.
Not sure if also caused by the same bug - I noticed that if I use the eraser side, after lifting it, it continues drawing for a while with the brush of the pen side. If my first stroke after starting mypaint was with the eraser, there is no pressure sensitivity at all. (I think the eraser side doesn't have pressure sensitivity, but it still breaks the pen side as well.)

(Regarding the somewhat offtopic eraser issue: it does work as a different brush, it's just not noticed as an eraser.)

[envision3d]

I wanted to join the conversation and offer my help in debugging, testing, experimenting, etc. in order to resolve this. I'm a long time user of MyPaint and have a SP4 on which I'd love to use it.

Please let me know any way I can help keep this moving forward!

[achadwick]

@envision3d (and anyone else willing to get a bit technical) – here's how to start:

Be sure you're seeing the same bug! 👓 ✔️

Basically, double-check your own experiences. Be sure you're as sure as you can be that it's happening, and that you know when it happens. Make sure you can reproduce it when you need to; we're about to step into active-doing territory.

Make really sure you have the most recent drivers for your tablet. This is so important. I cannot stress it enough.

MyPaint has an input tester! Help → Debug → Test Input Devices will tell you all about current pressure and tilt, and show you a stream of information about events as they come in on the main canvas. It's a bit basic, but it's a good toe-wetter for the next bit. That stream of bumpf that passes by is a stream of the events we're talking about... minus the proximity ones.

If pressure cuts out in this after you take the stylus away from the screen, and you see the dreaded "Pressure: (no pressure)" when you try again, that's this bug! As I understand it.

Ready to get technical?

Down the rabbit hole... 🐰

1. Set up a working MyPaint test environment: Instructions here.
   • This will allow you to test other code that uses GTK/GDK's native Windows ports too.
2. Get yourself to the point where you can launch the relevant MSYS2 shell, and launch an uninstalled (in-tree) compiled MyPaint from it.
3. Then get a copy of axischart.py too. By now you'll be able to run it (python2 ./axischart.py) because you have Python and all the other MyPaint deps.

You should now be in a position to get a really good handle on when these problems occur, and see how the code responds to it. Python's a really easy scripting language to learn. Just a few more pointers for debugging:

• Edit the code in a good text editor. Notepad++ is pretty good for this. Don't use Windows's built-in Notepad.
• Use print statements or logging (logger.info("value: %r", v)) to see values.
• (Advanced) If you get a proximity-in/proximity-out-event in axischart.py, try returning True (and False) from the handler. Does that change anything? Are the events being sent (don't forget to enable them: you'll need to allow Gdk.EventMask.PROXIMITY_IN_MASK and PROXIMITY_OUT_MASK somewhere.)

You can go deeper, and you probably will have to if my suspicions are correct. MyPaint 1.2.0+ is built on the GNOME project's GTK and GDK, version 3+.

• Read the docs: https://developer.gnome.org/references (look for GDK and GTK+)
• Look in the source code: https://git.gnome.org/browse/gtk+/tree/gdk

Hold on, isn't this an upstream bug? 🔌

We need to determine if MyPaint can be improved to help here, and how. However I still strongly suspect that it's really is an upstream bug (DO try and prove me wrong though).

If this is an upstream bug, it needs to be reported in their bug tracker by someone who owns the hardware in question:

• https://bugzilla.gnome.org/

They'll need the same details about your system as we do: make, model, version numbers of things, the tablet hardware you're using... and what GTK/GDK version you tried. Any strange results from the testing above would help them.

Please link any upstream bug reports here so that people can find them. If it's reported upstream, I can chime in too and help you explain. I'm occasionally active on their IRC channel (#gtk+ on GIMPNet), and that can be a good way to chase through backend-specific input bugs.

[scooter-dangle]

Similarly affected Surface Pro 4 user. Started a bounty for this issue: https://www.bountysource.com/issues/28004023-surface-pro-3-4-n-trig-hardware-pressure-sensitivity-stops-working-after-moving-pen-away-from-screen

[zsgalusz]

For the record - I installed 1.2.1-beta.1 on my SP4 and now I'm simply not getting pressure sensitivity at all, and my stylus is not recognized as such (only "System Aggregated Pointer" shows up under devices). I tried both the 32 and 64 bit builds, as well as reinstalling the wintab drivers.

[achadwick]

Bountysource is a good option, thanks @scooter-dangle!
Here's some more quick-start information for Surface Pro users who want to take this on:

• I believe that this is most probably an upstream issue in the win32 backend of GDK. If you fix that, you'll help a lot of people out.
• But there may be stuff we can do in MyPaint too to make this better. I'll eat my words and consider your changes for inclusion in MyPaint if they don't break other configurations.
• There's lots you can do in Python code right now to just diagnose this issue better.
• You might have to get your hands dirtty: How to set up gdb and make native Win32 debug builds of GTK.
• Planned docs work: Wiki: need an updated guide telling users how to diagnose tablet bugs, glitches, or omissions #674 "Wiki: need an updated guide telling users how to diagnose tablet bugs, glitches, or omissions". Stalled due to time/IRL issues right now 😢

[achadwick]

One important question to ask ourselves, from that planned docs work:

Q: can the input error be replicated with gtk3-demo (by running its Event Axes tester)?

If it happens for that too, it's definitely GTK/GDK that's at fault (or its interface with proprietary wintab.dll code). If not, it's the MyPaint code (or its interface with GTK/GDK).

[supertobi]

Where can I download the gtk3-demo.exe? I was only able to find a version with gtk 3.6.

[achadwick]

@supertobi The most productive way by far is to install the native packages of GTK on an MSYS2 deployment. The package names:

• mingw-w64-i686-gtk3
• mingw-w64-x86_64-gtk3

It's productive because there's also a corresponding debug PKGBUILD of the current git version available in MINGW-packages. They conflict, but in a nice way that lets you can swap in and swap out debug builds very easily with a bit of pacman -U magic on the command line.

[achadwick]

Possible related sibling-project bug? → https://bugzilla.gnome.org/show_bug.cgi?id=758500

[achadwick]

Updated version, needs testing

@supertobi @crydalch @zsgalusz @scooter-dangle @envision3d , everyone!

MyPaint v1.2.1 is now out, and the Windows build contains several fixes for tablet recognition (it's bundled with GTK 3.22.7). It also contains a bundled GTK3-Demo program, configured to open on the tablet tester page. Please can you:

1. Test MyPaint and see if this fixes the pressure dropout on proximity-out?

2. Completely close MyPaint and test with GTK3-Demo as well and see if the pressure dropout is reproducible there?

Gtk3-demo looks like this:
The red blob at the crosshair corresponds to pressure, and the angled lines shows tilt if your tablet has it. You'll be able to tell quite quickly if you're missing pressure suddenly.

The new launchers have these names:

[zsgalusz]

It's got even worse than before. Not only does the pressure sensitivity get lost after the first stroke, the pointer location also gets completely garbled now. To test, I tried to draw 3 circles on the left half of the visible canvas. The first circle appeared ok, but it was shifted significantly from where I was drawing. The second and third circles turned out not circles at all, but a jagged mess. (They appeared really slowly, too, about 2-3 seconds after I actually finished drawing them.) Here is a screenshot:

As an aside the pointer behavior is so bugged right now, that even the menus are inoperable by pen as they are affected by the same incorrectly perceived position of the pointer.

The Gtk3-demo reproduces the issue quite closely. When first using the pen, the pressure shows up, but the location is off to the side. After lifting the pen, the pressure goes away, too, and the location flickers slightly. Interestingly, after accidentally resizing the window the location seemed corrected (but the pressure was still gone) until lifting the pen again. The device name appeared to rapidly alternate between the system aggregate pointer and the Microsoft device stylus while I was holding down the pen.

For the record, I tested both 32 and 64 bit versions and there is no difference in their behavior.

[achadwick]

Upstream bug (GDK)

Please report to the upstream bug tracker

This is now confirmed as a bug in the GDK library: it affects GTK3-Demo too. We don't maintain the GDK library here ourselves. Somebody with the affected hardware needs to report this in the GNOME bugtracker, against the “gtk+” component so that the GDK developers know about it. Somebody with the affected hardware needs to take responsibility for getting this fixed with the upstream developers.

• Search GNOME Bugzilla for this bug first: wintab, ntrig. It may already have been reported!
• Use this form if you can't find it: Enter Bug: gtk+. You may need to create an account.
• Please post links to relevant upstream bugs here 😄

The GNOME Bugzilla is quite technical, but don't be discouraged. We can help you get it right here, and we have a few guides for reporting good bugs in our own wiki. See our wiki's Contributing section for details.

Give the upstream developers good details

• Paste a link to this bug into wherever you're commenting.

• The upstream developers are likely to want you to send them debugging output from gtk3-demo. This is best done by installing MSYS2 and its mingw-w64-x86_64-gtk3 package, and running gtk3-demo as:

  $ gtk3-demo --gdk-debug=input

  and attaching the output to the upstream bug report.

[achadwick]

@zsgalusz
Thank you for the testing, especially the bit about the rapid switching of device names. I'm sorry we can't take it further here.

One upstream bug I did find: https://bugzilla.gnome.org/show_bug.cgi?id=745697

Somebody needs to mention the completely broken tracking, though. It's clear to me that they're related.

[achadwick]

There is nothing we can do about this by changing the MyPaint code. This is a confirmed upstream fault with a plausible looking upstream bug report. Therefore I am closing the ticket.

This is not fixed. It merely cannot be fixed here.

Reward available for anyone fixing this!

There is a reward available on BountySource for anyone willing to approach the GTK developers and get the fault fixed (thanks, @scooter-dangle!). Here is the link:

💰 https://www.bountysource.com/issues/28004023-surface-pro-3-4-n-trig-hardware-pressure-sensitivity-stops-working-after-moving-pen-away-from-screen

Would it be possible to update the bounty to make it clear that these awful n-trig bugs are GTK issues, not MyPaint issues? A fix over in the GTK code would benefit more people (and we can't fix this in MyPaint).

[scooter-dangle]

Thanks for the update, @achadwick. I'll ping Bountysource support to ask whether the bounty can be transferred explicitly.

[Windmaedchen]

Created a bug report on GNOME Bugzilla:
Bug 778328

I hope it's not too bad, with my limited knowledge and all...

[achadwick]

@Windmaedchen
Thank you for chasing this!

Possible fix, please test!

We have a patched test-build ofr GTK in the upstream bug report, https://bugzilla.gnome.org/show_bug.cgi?id=778328#c18, and it looks promising so far.

Could anyone affected by this issue who's comfortable with a command line and our MSYS2 development environment please test out the build? You'll find instructions behind the links.

It's worth knowing that MyPaint 1.2.1 itself is available in MSYS2 repos now, so you can test in that too, provided you run it from a MINGW64 or MINGW32 shell 😄 Using MSYS2 means you can slot different gtk3 packages in and out to try different combinations without stuffing up your normal MyPaint install.

[crydalch]

@achadwick I will give the new Gtk fixes a try this week, I'll have a bit of time to give it a go! Cheers!

[crydalch]

Great news @achadwick, that patched build of Gtk3 seems to have fixed the problem! 🎉

The UI is quite small, but I'm guessing that's coming from/through msys/mingw, maybe they have a hard time with the hidpi resolution on a Surface? One question, is multi-touch navigation supposed to work? Or is that down-the-road?

Thanks/congrats to everyone to helped get this resolved!

[achadwick]

@crydalch
Nice, thanks for testing!

The UI should be double-sized on a Surface screen unless you have GDK_SCALE globally set to 1 or something. Other testers report that it autodetects now (a little too well in fact... it broke wintab again for HiDPI scales). Check your environment variables maybe (e.g. with env in MSYS2)?

You can experiment with different scales in MSYS2 by running mypaint as:

GDK_SCALE=2 mypaint -c /tmp/test2x
GDK_SCALE=3 mypaint -c /tmp/test3x
...
GDK_SCALE=1 mypaint -c /tmp/test1x

and so on. This won't affect the Windows environment settings.

Note that there are two patched debug builds of GTK floating around in the upstream tracker now. The other one fixes mis-positionings at 2x and greater. If you make your own build of GTK 's gtk-3-22 stable branch in its current state, it will contain all of the patches we've been working on. You can track the branch here and watch for the forthcoming release tag: https://git.gnome.org/browse/gtk+/log/?h=gtk-3-22 - expect 3.22.9 within a week or two, at the current rate 😄

MyPaint's master and stable v1.2.x branches contain a recent fix for a HiDPI glitch too. You may notice that it's a bit slower as a result; testing welcome 😐

---

No multi-touch support in GTK yet for Windows. For X11 and Wayland it's a different story and I need to get support for gestures into MyPaint somehow. But that is a different feature to add.

[scooter-dangle]

I've not tried to build anything on Windows. Any estimate on timeframe for releasing a build with the Gtk fixes?

[odysseywestra]

I've not tried to build anything on Windows. Any estimate on timeframe for releasing a build with the Gtk fixes?

We're in the process of changing how we build and distribute. So it's going to take a bit. However once we are done, we'll definably have much more frequent builds thanks to automating a bunch of it through Travis and Appveyor. We're making headway, but we still have a bit to go.

[marvin-droid]

I fixed the pressure of my Surface Pro 4 with the myPaint program, the problem was not the program, it was about the "wintab" controllers. I uninstall the wintab and then install the most recent version (it was actually the most recent version I had installed) but this time I installed wintab in repair mode and restart mu surface pro, when turning on and installing the wintab once more using in install mode default. Finally, I installed the myPaint program because before this process I had uninstalled the myPaint program, although I don't know if that matters at all

[marvin-droid]

I wanted to join the conversation and offer my help in debugging, testing, experimenting, etc. in order to resolve this. I'm a long time user of MyPaint and have a SP4 on which I'd love to use it.

Please let me know any way I can help keep this moving forward!

I fixed the pressure of my Surface Pro 4 with the myPaint program, the problem was not the program, it was about the "wintab" controllers. I uninstall the wintab and then install the most recent version (it was actually the most recent version I had installed) but this time I installed wintab in repair mode and restart mu surface pro, when turning on and installing the wintab once more using in install mode default. Finally, I installed the myPaint program because before this process I had uninstalled the myPaint program, although I don't know if that matters at all