Mouse touchpoint now properly marked as invalid

[jamesford42]

No description provided.

[nkast]

Is Invalid the right thing to do here?

I imagine it should emulate what touch does when your finger moves outside the screen/sensor.

• When a pressed mouse leaves the window send a Release with the previous position (=withinClient).
• Send a Pressed when a pressed mouse enters.

Invalid is returned only from TryGetPreviousLocation(). It's very unlikely that existing code will check for it. Most likely it will be treated as Released anyway.

Another thing..., does it work with Gestures? (tap,swipe, etc?) I don't think the Gesture detector expects an Invalid state.

[jamesford42]

I imagine it should emulate what touch does when your finger moves outside the screen/sensor.
Something like, send a Release with the previous position (=withinClient) when a pressed mouse leaves the window and send a Pressed when a pressed mouse enters.

But what you described is actually not how it works in xna/monogame. Consider, if it were, then an intentional release by the user would be indistinguishable to the code from an accidental (or intentional) drag off-screen without release.

This change will actually bring the mouse touch location behavior in-line with normal touch location behavior.

As far as gestures, yes any gesture which needs a release to be recognized (ie, tap, double-tap) would previously have (incorrectly) occurred for the mouse touch location when leaving the window but will now behave the same as a real touch location, which does NOT.

[nkast]

I suspect you are trying to communicate the MouseLeave event through the TouchPanel and then handle that as a special case in your code instead of focusing on actual emulation (which is possible as I describe above).

[nkast]

.. think of the mouse cursor as a finger that is pressed down when LeftButton is pressed and the window boundaries as a physical tablet or phone.

[jamesford42]

I agree, the mouse touch location is suppose to behave like a virtual finger. Do the test you are describing with your finger; the touch location goes from pressed to invalid and is never in a released state.

[nkast]

That actually happened? Strange!
What type of device did you use? A windows phone or WP emulator for example will do if I want to reproduce?

[jamesford42]

Normal windows xna pc build running windowed on a touchscreen monitor. I may have tested with WP emulator but I can't recall and I don't have that sdk installed on this computer anymore.

Edit: Actually I was matching the monogame behavior, I'm not sure if I tested actual xna, but any difference there would also apply to normal touch locations not just the mouse one.

[nkast]

I can't test WP7 nowadays, I need to install a Win7 partition just for that. On W10 everything in the SDK is broken (build fails and emulator reboots the OS!).

So, MG reports Invalid touch on a windowed app with a touch screen.
Looking at the OS events (CoreWindow.PointerExited, WM_POINTERLEAVE) it seems it treats PEN,Touch,Mouse the same way. You get events for everything. I wonder if that behavior of MG is by design or just a side effect of OS events. I can't see any place where the touch state is explicitly set to Invalid after a Leave event. Also Gesture detection nowhere checks for invalid state (does it somehow recover from 'open' gestures?).

On XNA/WP7 I never had to worry about Invalid touch positions and so says the MSDN documentation about TouchPositionState.Invalid. I bet that's also the case for fullscreen apps on MG and on any platform.

Thoughts? Did we just open the bag of Aeolus?

[jamesford42]

So, MG reports Invalid touch on a windowed app with a touch screen.

Good.

I can't see any place where the touch state is explicitly set to Invalid after a Leave event.

-edited-

Actually that reminded me, I did test on the WP emulator and found that pressing and then dragging off the edge of the physical device was reported as as the touch location becoming 'invalid' for one update before it is removed.

The only difference here is that our mouse touch point cannot be removed because the rest of the implementation depends on there always being a touch point with the magic-number-id indicating the special mouse touch point.

[ziriax]

I'm not sure what behavior you'll end up implementing, but IMHO moving the finger outside of the window is undefined behavior in XNA anyway.

That is, TouchPanel only works on Windows Phone, and you can't compare moving a finger out of a physical device with the mouse or finger being moved outside of the non-full-screen window, since you can't feel that.

So I would prefer that the MG mouse-to-touch emulation just captures the mouse when pressed (emulating TouchLocationState.Pressed), and then sends TouchLocationState.Moved events even when the mouse cursor (or finger) is moved outside of the screen. When the mouse capture is lost, or when the mouse is released, send a TouchLocationState.Released (one might argue that when the capture is lost, TouchLocationState.Invalid could be send).

I think this is what the user expects, when dragging the mouse the user might not be looking at its mouse cursor, but to the content being affected by the drag. So the user should not be penalized for accidentally moving the mouse outside of the window...

[nkast]

[...] but IMHO moving the finger outside of the window is undefined behavior in XNA anyway.

When you move your finder away from the touch sensor it sends a Release event. It's doesn't matter whether you lifted or just moved it outside the sensor.

[...] and then sends TouchLocationState.Moved events even when the mouse cursor (or finger) is moved outside of the screen.

That is actually an invalid behavior for the TouchPanel, it will never return values outside the TouchPanel.DisplayWidth/TouchPanel.DisplayHeight.

[ziriax]

Yes, but a window is not the same as the touch sensor. A window is a rectangular region of a larger touch sensor (ie a window on a surface pro or any otter touch screen). So I agree Release should be send when moving outside of the touch sensor, but not the window. Windows Phone did not have windows, XNA games were always full screen, so you can't really know what Microsoft would have done with TouchPanel in a windowed environment... I only know that since it is impossible to explicitly capture the mouse with the current MG API (or touch?) that the typical behavior used in almost all desktop applications is to capture the mouse when pressed.

[nkast]

Windows Phone did not have windows, [...], so you can't really know what Microsoft would have done with TouchPanel in a windowed environment...

That's a good point. So maybe we should mimic the XNA Mouse Class and report touchpoints even outside the window.

Actually WinForms capture the mouse and keep sending events for as long as LeftButton is pressed, even the MouseLeave event is delayed until you release it.
A fix would be as easy as to remove withinClient from the check here: https://github.com/MonoGame/MonoGame/blob/develop/MonoGame.Framework/Windows/WinFormsGameWindow.cs#L247

[nkast]

Thinking about it, TouchPanel is a bit more complicated than Mouse. Take Gestures for example,
should we recognise Tap and Hold that originate outside the window?

Or a pitch that's halfway into the window? If one touch is in the clientArea and the other is outside, this could either be a Pitch, or a Hold in the XNA window and a drag or a third app. While for mouseState the user could check the ClientBounds, for Gestures I don't see an easy way around it.
Maybe the key here is the window where the press first originated (whether we talk about Mouse or actual touchscreen). So we might need to keep a few more states, it's not as simple as removing withinClient as I though.

[ziriax]

Thinking about it, TouchPanel is a bit more complicated than Mouse. Take Gestures for example,
should we recognise Tap and Hold that originate outside the window?

Well, Tap and Hold can't originate outside of the window, since the initiating mouse-down will always occur inside the window. Or maybe you are talking about multiple game windows? For Tap and Hold that doesn't matter, but maybe for Pinching.

I guess you mean pinching? When emulating touch using the mouse, pinching seems impossible since it requires two touches. IMO when pinching with fingers the game window boundaries shouldn't matter either, you just keep tracking each finger after a touch occurs, disregarding the window. A user doesn't feel the window, he only feels the thing he's touching.

[tomspilman]

Fixed via #5641.