Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Windows Ink events being handled as touch events #5309

Merged
merged 9 commits into from Dec 27, 2023

Conversation

Susko3
Copy link
Member

@Susko3 Susko3 commented Jul 17, 2022

Depends on:


Previously (without SDL_HINT_TOUCH_MOUSE_EVENTS = "0") pen/touch inputs were sent as mouse events.
Disabling Use Windows Ink in wacom driver settings fixes the issue, as it reports inputs as mouse events.

This also fixes in-game tablet handling clashing with Windows Ink "touches", as InputManager will properly block unwanted mouse events (but won't block touch events!).

@peppy peppy self-requested a review December 17, 2023 16:03
@Susko3
Copy link
Member Author

Susko3 commented Dec 17, 2023

Can confirm that, on a touchscreen monitor, touch names are reported as "touch". I don't currently have a tablet to test with.

@Teages
Copy link

Teages commented Dec 24, 2023

I tested with my tool which use windows ink api to Inject pen event.

I never use .net before and I can't find any place showed the event name in both osu.Framework.Tests and properties of the event, so I added some code.

protected override void HandleTouchFingerEvent(SDL.SDL_TouchFingerEvent evtTfinger)
{
    Logger.Log($"pen event: {evtTfinger.type} {evtTfinger.x} {evtTfinger.y}", LoggingTarget.Runtime, LogLevel.Debug);
    if (evtTfinger.TryGetTouchName(out string name) && name == "pen")
    {
        // Windows Ink tablet/pen handling
        // InputManager expects to receive this as mouse events, to have proper `mouseSource` input priority (see InputManager.GetPendingInputs)
        // osu! expects to get tablet events as mouse events, and touch events as touch events for touch device (TD mod) handling (see https://github.com/ppy/osu/issues/25590)

        TriggerMouseMove(evtTfinger.x * ClientSize.Width, evtTfinger.y * ClientSize.Height);

        Logger.Log($"pen event: {evtTfinger.type}({name}) {evtTfinger.x} {evtTfinger.y}", LoggingTarget.Runtime, LogLevel.Debug);

// ...

looks good, at least name is correct

[runtime] 2023-12-24 09:17:43 [debug]: pen event: SDL_FINGERDOWN 0.659375 0.44346872
[runtime] 2023-12-24 09:17:43 [debug]: pen event: SDL_FINGERDOWN(pen) 0.659375 0.44346872
[runtime] 2023-12-24 09:17:43 [debug]: pen event: SDL_FINGERMOTION 0.659375 0.4445664
[runtime] 2023-12-24 09:17:43 [debug]: pen event: SDL_FINGERMOTION(pen) 0.659375 0.4445664
[runtime] 2023-12-24 09:17:43 [debug]: pen event: SDL_FINGERMOTION 0.659375 0.45005488
[runtime] 2023-12-24 09:17:43 [debug]: pen event: SDL_FINGERMOTION(pen) 0.659375 0.45005488

This bug really prevents me from playing lazer so I did this quick test, hope it helps you.
I can assist with testing if you need it

Redundant with the SDL2-CS update, see flibitijibibo/SDL2-CS#235.
@Susko3
Copy link
Member Author

Susko3 commented Dec 24, 2023

Thanks for testing! Looks like it's correctly recognising pen/windows ink events.
It would help if you could test with a local clone of ppy/osu. You can run UseLocalFramework.ps1 to use your local framework checkout (i.e. this PR) with osu!. (you may need to merge master into this PR locally to make things work.)

Some things to try out:

  • relative mouse mode on/off
  • mouse sensitivity != 1.0
  • touch handler enabled/disabled
  • in-game tablet drivers (I think your vTablet app may not work with them)
  • how does the touch device (TD) mod behave

@Teages
Copy link

Teages commented Dec 24, 2023

ok, I did test, my vTablet runs well with default setting

relative mouse mode on/off

I can't find "relative mouse mode", I think you mean High precision mouse
It is on by default, but if I turn it off, osu would not accept hover event
(just like in previous version)

mouse sensitivity != 1.0

The pointer would be locked at 0,0
It happens on stable when I turn raw input on

touch handler enabled/disabled

Enable Input/Touch or not doesn't effect to my app

in-game tablet drivers

I think it is based on OTD, so it doesn't works for me

how does the touch device (TD) mod behave

  • on the latest release: TD will auto turn on when pen down and turn off when pen up (song select page)
  • with this pr: I can't find TD mode on mode list

@Susko3
Copy link
Member Author

Susko3 commented Dec 24, 2023

 I can't find "relative mouse mode", I think you mean High precision mouse
It is on by default, but if I turn it off, osu would not accept hover event
(just like in previous version)

Hmm, this means that windows ink hover events are delivered trough WindowsMouseHandler. I wonder if there is some double-reporting happening when the pen is pressed down.

mouse sensitivity != 1.0

The pointer would be locked at 0,0

That's a bit unexpected, as the linked WindowsMouseHandler logic has sensitivity accounted for.

@Teages
Copy link

Teages commented Dec 24, 2023

I don't know how to get raw input event in osu, I played with pure pen and save a record, hope it include the data you need
Guest playing ONIMAI SISTERS - Himegoto_Crisisters (TV Size) (Yorita Yoshino) [Easy] (2023-12-24_20-24).zip

I build a golang program (thank gpt), then you can test it without a tablet.
test.zip

Usage: main.exe
  -duration int
        duration in seconds (default 1)
  -fps int
        frames per second (default 240)
Source Code
package main

import (
	"flag"
	"math"
	"time"

	"github.com/Teages/go-vdigi"
)

var (
	fps      int
	duration int
)

func init() {
	flag.IntVar(&fps, "fps", 240, "frames per second")
	flag.IntVar(&duration, "duration", 1, "duration in seconds")
	flag.Parse()
}

func main() {
	screens := vdigi.GetScreens()
	mainScreen, _ := screens.GetScreen(0)
	width, height := mainScreen.Width, mainScreen.Height

	pointer, _ := vdigi.CreatePointerForScreen(0)

	centerX, centerY := int32(width/2), int32(height/2)

	drawCircle(pointer, centerX, centerY, 100, 0)
	drawCircle(pointer, centerX, centerY, 50, 32767)

	pointer.Update(centerX, centerY, 0)
	pointer.Destroy()
}

func drawCircle(pointer *vdigi.Pointer, centerX, centerY, radius int32, pressure uint32) {
	frames := duration * fps

	for i := 0; i < frames; i++ {
		angle := float64(i) * 2 * math.Pi / float64(frames)

		x := centerX + int32(float64(radius)*math.Cos(angle))
		y := centerY + int32(float64(radius)*math.Sin(angle))

		pointer.Update(x, y, pressure)
		time.Sleep(time.Second / time.Duration(fps))
	}
}

Copy link
Sponsor Member

@peppy peppy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gonna get this in as we continue to get reports of this. The remaining issue looks like it can be solved separately.

@Susko3
Copy link
Member Author

Susko3 commented Dec 27, 2023

From my testing with the above program, windows ink input works with High precision mouse turned on and off.

The only issue seems to be with Precision != 1.0: hover events apply the precision, but "pen down" events don't. But that was an issue even before this PR, so it should be solved separately.

@peppy peppy merged commit 71610ef into ppy:master Dec 27, 2023
19 of 21 checks passed
@Susko3 Susko3 deleted the fix-windows-ink branch December 27, 2023 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants