Surface Slim Pen 2 may have mis-behaviour on side key and soft pressure of bottom button

[NP-chaonay]

Warning: The all progress specific in this page is freezed and no longer updated/active, see the bottom of the page for information

Issue Overall Progress: doing remained test/data-gathering with reading remain unread comments and doing something.

Remark: This comment may be updated later.
Tested on Ubuntu 22.04 LTS surface kernel quo's of both iptsd/ithc, Surface Laptop Studio Wayland

Behaviour (can request more test):
Bottom Button:

• when first, pen is hover near to screen but not touch, soft-pressure make continuous (continuous mean multiple click in single time, not holding click) right click
• AS I tested many times, while pen is hover/touch, if soft-pressure already in progress, pressing eraser button screen do not do thing (so action that is done by soft-pressure event).
• If pen is in touching, soft-pressure result in non-continuous right-click
• if hold soft-pressure key, then hover, it make many N click of right click.
  • and when touch, it make it single left-click
• if hold pressing eraser key, then hover, it have nothing
  • and when touch, it make left-click

Side Key:

• When pen is in touch/hover:
  • pressing side key cause right click. if hold pressing side key before touch the pen, it cause 1st right click (from hover event) and then left click ( from pen touch event).
• if hold side key, then hover, it make right click

when using eraser instead of pen tip for hover/touch:

• general
• • on carefully testing, seem that left click is only come from soft pressure of eraser (pressure thing of eraser tip) while hovering/touching. (I have test on Windows, this is same beahaviour as Windows, so can confirm nothing wrong ~>=99% IMO)
• when combine side key in testing: side key do nothing (both eraser hovered/touched (just very soft touch, try to not put much pressure of eraser pressure sensor) and both side key holding/1 click pressing)

Misc: Tested on Write app (not much test, test inside paper canvas):

• Settings
• • Pen button tool (I think it is side key): Select
• • Mouse Input mode: Pan
• • Single-Touch mode: Select
• • Multi-Touch mode: Pan and Zoom
• Eraser event works on both touch and hover
• hold right click and drag (panning) and hold pen side key and drag (select); so they have different behaviour
• side key
• • while pen hovering: holding or 1x press on key : seem do nothing
• • pen touch 1st then holding/just-press side key: discard pen line becoming selection
• • holding side key then let the pen touch to screen later: same as above
• bottom button
• • while pen in touch: holding or 1x press/soft-pressure to the button: discard pen line becoming selection
• • • while pen in hover: holding or 1x press/soft-pressure to the button: seem do nothing
• while using eraser:
• • pressing and holding side key: seem do nothing
• • using: normally on both soft/hard pressure
• (can request more testing for it)

My opinion:

• maybe there is something about pen/eraser pressure (or something).
• I think that if I holding side key / or holding soft-pressure of bottom button (only when pen is in touching) may == holding right click
• eraser tip may not have significant problem only found when using pen tip with eraser pressure-alike thing activated.

[quo]

So if I understand correctly, middle-click triggers if the pen is touching the screen, even if the side button is not being pressed?

Can you post a /dev/ithc capture from this happening?

Also, can you try increasing IPTS_DFT_BUTTON_MIN_MAG (in src/ipts/dft.cpp) to e.g. 2000 and then rebuilding to see if this helps?

[NP-chaonay]

UPDATED: I have edit from soft-pressing to soft pressure , in order to more understand

So if I understand correctly, middle-click triggers if the pen is touching the screen, even if the side button is not being pressed?

ok I think I cant not explain more clearly from above, so let shorten them and cut some details of issues.
middle-click triggered (updated: right-click) when pen is both touch and near screen in some conditions of:

• eraser soft-pressure (not press but little press) on eraser, it seem like it have force sensor at eraser tip
• side key

normal pen touch/near when no eraser soft-pressure and no side key press is normal

Can you post a /dev/ithc capture from this happening?
Also, can you try increasing IPTS_DFT_BUTTON_MIN_MAG (in src/ipts/dft.cpp) to e.g. 2000 and then rebuilding to see if this helps?

ok I will gather /dev/ithc for per each events/condition;
and later change IPTS_DFT_BUTTON_MIN_MAG and retry the test and also gather info from /dev/ithc
this would take time as in the near future time, I have low free-time.

[NP-chaonay]

(UPDATED 24/7/22: Progress is slow downed due to university academic)

Gathering/Testing progress (because I know it take time on it; however if I not super busy I will do test each entry below here everyday)

Key: S: skipped (decided to not gather/test result /dev/ithc from/related to the specific statement)

Environment used: SLS Surface SLim Pen 2 Kernel 5.18.5-surface GNOME-Shell
have 3 big part:

• Gathering Part A: /dev/ithc gathering before driver changed
• Gathering Part B: /dev/ithc gathering after driver changed
• [] Testing Part B: testing after driver changed (this should go quickly in 1-2 days, since it is fluent to test rather than to implement and doing "gathering part") you can see tested result instantly on what I have tested here.

details

Gathering Part B

Will be reported as percentage

Testing Part B

Keys: ticked=tested_OK_have_same_result S=Skipped_As_Aboved_Said W=HaveNegativeFeedback N=TestOKButSomethingUsefulNoticed N=TestOKButTestIsDifferrent

Behaviour (can request more test):
Bottom Button:

• when first, pen is hover near to screen but not touch, soft-pressure make continuous (continuous mean multiple click in single time, not holding click) right click
• AS I tested many times, while pen is hover/touch, if soft-pressure already in progress, pressing eraser button screen do not do thing (so action that is done by soft-pressure event).
• If pen is in touching, soft-pressure result in non-continuous right-click
• if hold soft-pressure key, then hover, it make many N click of right click.
  • and when touch, it make it single left-click
• if hold pressing eraser key, then hover, it make nothing
  • and when touch, it make left-click

Side Key:

• When pen is in touch/hover:
  • pressing side key cause right click. if hold pressing side key before touch the pen, it cause 1st right click (from hover event) and then left click ( from pen touch event).
• if hold side key, then hover, it make right click

when using eraser instead of pen tip for hover/touch:

• general
• • on carefully testing, seem that left click is only come from soft pressure of eraser (pressure thing of eraser tip) while hovering/touching. (I have test on Windows, this is same beahaviour as Windows, so can confirm nothing wrong ~>=99% IMO)
• when combine side key in testing: side key do nothing (both eraser hovered/touched (just very soft touch, try to not put much pressure of eraser pressure sensor) and both side key holding/1 click pressing)

Misc: Tested on Write app (not much test, test inside paper canvas):

• [S] Settings
• • [S] Pen button tool (I think it is side key): Select
• • [S] Mouse Input mode: Pan
• • [S] Single-Touch mode: Select
• • [S] Multi-Touch mode: Pan and Zoom
• Eraser event works on both touch and hover
• hold right click and drag (panning) and hold pen side key and drag (select); so they have different behaviour
• side key
• • while pen hovering: holding or 1x press on key : seem do nothing
• • [W:SameResult] pen touch 1st then holding/just-press side key: discard pen line becoming selection
• • [W:SameResult] holding side key then let the pen touch to screen later: same as above
• [W:Seebelow] bottom button
• • [W:SameResult] while pen in touch: holding or 1x press/soft-pressure to the button: discard pen line becoming selection
• • • while pen in hover: holding or 1x press/soft-pressure to the button: seem do nothing
• while using eraser:
• • pressing and holding side key: seem do nothing
• • using: normally on both soft/hard pressure

[NP-chaonay]

UPDATE NOTE: cancelled upload, file is kept in reference and potential usage
UPDATE NOTE: Due to re-test and many thing that make much complexity, I have to cancel file uploading, and uploading when everything is in good place

In case not want to wait or not to make thing slow down. I put completed files that just been done ahead of time.

File inside these except README/README2 is may not be need to updated further and can be analyse instantly without need of waitting above progress to be completed.
https://drive.google.com/drive/folders/1mMej0T6J5aZ7YK3WinOixuj7i-03F8xC?usp=sharing

README2 is regularly updated to tell description of each log files
for more info about .bin log file pls see README/README2

[qzed]

Setting IPTS_DFT_BUTTON_MIN_MAG = 4000 worked for me on the Surface Pro X.

[NP-chaonay]

Setting IPTS_DFT_BUTTON_MIN_MAG = 4000 worked for me on the Surface Pro X.

so your problem also affect on this same pen and on eraser button? but let this as a try for both side key and eraser button bug

PS: just test this as a try, but I think it would be the same, idk.

[qzed]

For some reason now that I want to re-test this I have a hard time reproducing the issue...

I've tested the both with the Slim pen (1) and the normal pen. The problem does not seem to happen with the normal pen, but 4000 works well on both. 2000 was a bit low leading to (less) wrong button presses and 5000 seemed to slightly interfere with the normal pen. 3000 works as well.

I haven't had any issues with the eraser functionality (i.e. flipping the pen around). With respect to the eraser button (if you mean the clicky thing on the end): That's handled via bluetooth, so not iptsd/ithc/... and I've had some issues with that not registering properly. But that seems to be a mechanical problem.

[qzed]

I'm not sure how much information we get about the pens (the old format had some serial number I think), so it might be possible to make these settings pen-dependent.

[NP-chaonay]

For some reason now that I want to re-test this I have a hard time reproducing the issue...

I've tested the both with the Slim pen (1) and the normal pen. The problem does not seem to happen with the normal pen, but 4000 works well on both. 2000 was a bit low leading to (less) wrong button presses and 5000 seemed to slightly interfere with the normal pen. 3000 works as well.

I haven't had any issues with the eraser functionality (i.e. flipping the pen around). With respect to the eraser button (if you mean the clicky thing on the end): That's handled via bluetooth, so not iptsd/ithc/... and I've had some issues with that not registering properly. But that seems to be a mechanical problem.

yes i know that the button click is handled via Bluetooth, but as I said in issue details above, there seem to be sensor of pressure detection at eraser tip which trigger right click which is misbehaving.

[qzed]

Ah, sorry for misunderstanding. I have not experienced that problem.

[quo]

@qzed
I had intentionally set IPTS_DFT_BUTTON_MIN_MAG a little lower than IPTS_DFT_POSITION_MIN_MAG, so that the button would be detected reliably even when hovering above the screen. So I'd rather not set it to a higher value by default. I think maybe it will work better to check if the button signal is about as big as the position signal (e.g. >80%), instead of setting an absolute limit.

Can you maybe upload a raw data dump (preferably in ipts-dump or ithc format)? Just pressing the button a few times with the pen touching the screen, and then slowly moving the pen away from the screen with the button still pressed should be enough. (Of course a data dump from the other pen would be nice as well, more data from different pens is always helpful :)

[qzed]

Those are raw data dumps read from the hidraw device via iptsd-dump (which as far as I can tell just directly reads from the hidraw device, so raw HID data). Looks like the current iptsd doesn't support dumping parsed data.

pen-dumps.zip

[qzed]

Also: I'm not expecting the defaults to work perfectly for every device. I think we should (at some point) rather add some config options, so we could have one set of options per device (or maybe even pen-model if we can identify that somehow).

[NP-chaonay]

STATUS: This section is completed

This section

Ok I have doing test wrong on part about middle-click/right-click. in Both Test part A (1st comment) and part B.

So I will re-test the related ones in my first comment. And after that the re-test have been completed, and continue doing remained task.
This have not affected to data gathering. (should not be affected, but I not gurantee, but maybe not too complex it it affected). (This statement is not 100% correctly, see below)

I will report here if mis-test parts have been re-tested.

PS: I have added tag "(NEED RETEST)" to indicate which statement requires either re-test/re-change-statement. I have to solve every statements prefixed with that tag in order to mark this re-test as completed.

PS2: for future reference, I have revert IPTSD suggestion-workaround by clone this repo, before doing 1st comment's re-testing. And for "Testing Part B", I will remind myself to apply that workaround before attempting doing the that next re-test.

PS3: The Gathering Part is also affected, but only README2 text file.

[NP-chaonay]

NEW_TEST_UNDONE tag job: ✔️ finished

UPDATE: I plan to add new two new tested (and further if it have)

now it is tagged as # NEW TEST UNDONE # and I will do the job for (NEED RETEST) first.

The jobs for NEW_TEST_UNDONE:

• Do Test on 1st comment
• Gathering Data Part A

and the remain is not in scope of the NEW_TEST_UNDONE tag. and handle like the normal job

[NP-chaonay]

STATUS: this section done

this section

Test edit required for statement

"on carefully testing, seem that left click is come from both touching of eraser tip and the soft pressure of eraser (pressure thing of eraser tip) while hovering. But seem that touch and do soft pressure at the same time cause nothing."

tagged with **# TEST EDIT REQUIRED # **

[quo]

@qzed

Those are raw data dumps read from the hidraw device via iptsd-dump (which as far as I can tell just directly reads from the hidraw device, so raw HID data).

Thanks. Was a little annoying to parse without any headers :) I assumed the report sizes are the same as on the SP7+, which seems mostly true, except on SP7+ report 26 has 2+7485 bytes of data, whereas in your case it looks like it has 2+7484 bytes. (Might be interesting to compare the HID descriptors. Here's the SP7+ descriptor. I made a parser a while ago to turn the descriptors into something more readable: https://github.com/quo/hid-descriptor-parser)

Anyway, the noise levels in these dumps look fairly normal, and well below the threshold for button presses. So I think the problem is not occurring anymore? It could also depend of the area of the screen you're using, if only some antennas have higher noise levels.

I also wonder if it's possible for the pen to go slightly out-of-sync with the touchscreen, which could allow the position signal to bleed into the button signal, since they use the same frequency.

Also, previously I thought that the "magnitude" field was some kind of combined signal magnitude from all antennas, but I just noticed that it is simply the squared magnitude of the center bin/antenna (real[4]*real[4]+imag[4]*imag[4]). It goes up and down quite a bit depending on whether the pen is right above an antenna or in between two antennas... So it really shouldn't be used with cutoff values the way I'm doing now. Instead it would be better to use the maximum of the fitted parabola.

Also: I'm not expecting the defaults to work perfectly for every device. I think we should (at some point) rather add some config options, so we could have one set of options per device (or maybe even pen-model if we can identify that somehow).

Yeah, that might be a good idea. Still, it should be possible to make things work reasonably well out of the box, without any tuning. Actual signals can be told apart from noise pretty easily because noise is roughly the same for each antenna, whereas the signal peaks near 1 or 2 antennas. It's just that the code currently doesn't take any of that into account.

As for pen identification: There is some kind of binary code transmitted in the other 3 rows of the button packet (9). Sequences of 14 bits, repeating after 11 values, which seem to be made up of: a 1 bit, a 4 bit index which loops from 0 to 10, 3 bits of data that's different for each pen, 3 bits of data that's the same for each pen, and 3 more bits of data that's different for each pen. Newer pens also have binary data in some of the other packets. Especially the data in packet 10 looks like some kind of unique id.

[qzed]

Here's the desc from the SPX: hiddesc.zip

Anyway, the noise levels in these dumps look fairly normal, and well below the threshold for button presses. So I think the problem is not occurring anymore? It could also depend of the area of the screen you're using, if only some antennas have higher noise levels.

Most of the time it happens intermittently. I think in the logs I posted it mostly worked but some times in the middle/end it mis-classified.

I also wonder if it's possible for the pen to go slightly out-of-sync with the touchscreen, which could allow the position signal to bleed into the button signal, since they use the same frequency.

I'd hope that'd be handled/ensured by the hardware... maybe?

Yeah, that might be a good idea. Still, it should be possible to make things work reasonably well out of the box, without any tuning. Actual signals can be told apart from noise pretty easily because noise is roughly the same for each antenna, whereas the signal peaks near 1 or 2 antennas. It's just that the code currently doesn't take any of that into account.

Makes sense. I was thinking of maybe using some basic noise tracking to have an adaptive threshold.

[quo]

Weird, the SPX descriptor is identical to the SP7+ descriptor. But on SP7+ the buffer is the right size (I just double checked), and in your files it's one byte too small. Wonder if that's a firmware bug or something else is happening.

Also, on closer inspection it looks like the button signal is almost always >10% higher than the position signal, so it's probably okay to increase the threshold a bit.

[NP-chaonay]

This issue is closed instantly due to this repo is have now deprecated.

So I will do my job remained here to be finished, then just going to reinstall official iptsd to see if problem is persist, if yes then I will making new issue then on there.

the new 2 job

• https://patrickhlauke.github.io/touch/pen-tracker/
• most tests but on windows

[NP-chaonay]

Update the progress: The issue testing is cancelled on quo iptsd version. And this page will be no longer updated.

I will testing the problem on latest official iptsd, if problem still exist, then issue will be created at linux-surface/iptsd