Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility mouse transition events should reflect a single logical mouse pointer #35

Closed
NavidZ opened this issue Feb 23, 2016 · 17 comments

Comments

Projects
None yet
6 participants
@NavidZ
Copy link
Member

commented Feb 23, 2016

@RByers @mustaqahmed

Looking at this example:

https://output.jsbin.com/bupaluf

Step 1. Move the mouse into the green box and leave it there.
Step 2. Touch the blue box and release.
Step 3. Now slightly move the mouse.

Here is the sequence in case it is hard to test it on Edge:

---- Step 1 ----
pointerover (id=1) on green
mouseover on green
pointerenter (id=1) on green
mouseenter on green
---- Step 2 ----
mouseout on green
pointerleave (id=134) on green
mouseleave on green
pointerover (id=134) on blue
mouseover on blue
pointerenter (id=134) on blue
mouseenter on blue
pointerout (id=134) on blue
mouseout on blue
pointerleave (id=134) on blue
mouseleave on blue
---- Step 3 ----
mouseover on green
pointerenter (id=1) on green
mouseenter on green

This sequence of events seem incorrect to me. Particularly looking at pointerevent with id=1 you can see the inconsistent sequence of events for it (i.e. 2 enters for green). Also mouse events (that are actually fired by moving mouse device) do no fully match the pointerevents with id=1 (i.e. the last 3 events).

The problem I see here is not quite addressed in the spec either. So I was wondering if something is missing or I have misunderstood something.

If there is actually a problem here, I can imagine the following solutions (which the first one looks cleaner to me but I don't know the impact).

  1. Do not try to match pointerevents with mouse events at all. So basically we left mouse in the green box and we touched blue box and fired compatibility mouse events for the touch event. Now if we move the mouse since the pointerevent with id=1 is still in the green box we shouldn't send any transition events (i.e. pointerenter/over) however since the mouse (i.e. the compatibility mouse which might include the actual mouse, touch, or pen) is in the blue box we should now send the mouse transition events to both blue box (mouseout/leave) and green box (mouseover/enter). This solution breaks the consistency of pointerevent with id=1 with mouse events but tells a consistent story looking only at pointer events or only looking at the mouse events.
  2. The second solution is when we touch somewhere and we fire compatibility mouse event we really move the mouse to there. So basically not only send the pointerevents with id=#RandomNumber for the touch event itself but also send the pointerevents with id=1 for the mouse. This solution makes sure the consistency of pointerevent with id=1 and the mouse events but in the cases like pen or touch events sends double pointer events which might not be desirable.

I was wondering what exactly Edge does as I couldn't quite figure out from the sequence of events. It is definitely neither of what I said.

@RByers

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2016

This is a really interesting question, thanks @NavidZ!

Basically I'd summarize this as:

  1. We'd want pointer events to represent a consistent enter/leave sequence per pointer ID
  2. For compatibility, mouse events should always present a self-consistent sequence
  3. But multiple different pointer IDs all map to mouse events
  4. And the spec implies there should be a 1:1 mapping from each pointer event to a compatibility mouse event

We can't have all 4 properties. In the case above Edge seems to violate 1, but that seems like a real shame since it means developers using the new multi-point aware API don't get a self-consistent event stream. I'd personally rather violate 4 and maybe even 2 if necessary in edge cases.

@teddink can you describe Edge's behavior here and what you consider by-design vs. a bug?

Note that this is related to issue #7 which we still need to address in the spec somehow.

@teddink

This comment has been minimized.

Copy link

commented Mar 4, 2016

Appears that we have a bug in Edge - thanks for the discussion.

The behavior should be #1 as stated above, which translates to:

---- Step 1 ----
pointerover (id=1) on green
mouseover on green
pointerenter (id=1) on green
mouseenter on green
---- Step 2 ----
pointerover (id=134) on blue
mouseover on blue
pointerenter (id=134) on blue
mouseenter on blue
pointerout (id=134) on blue
mouseout on blue
pointerleave (id=134) on blue
mouseleave on blue
---- Step 3 ----
mouseover on green

@NavidZ

This comment has been minimized.

Copy link
Member Author

commented Mar 5, 2016

But here if one only looks at the mouse events they have some events missing too. How come while mouse was on green on step 1 then all of a sudden blue div gets the mouseover/enter without green getting any out/leave in step 2?

@NavidZ

This comment has been minimized.

Copy link
Member Author

commented Mar 5, 2016

How about something like this event sequence:

---- Step 1 ----
pointerover (id=1) on green
mouseover on green
pointerenter (id=1) on green
mouseenter on green
---- Step 2 ----
mouseout on green
mouseleave on green
pointerover (id=134) on blue
mouseover on blue
pointerenter (id=134) on blue
mouseenter on blue
pointerout (id=134) on blue
mouseout on blue
pointerleave (id=134) on blue
mouseleave on blue
---- Step 3 ----
mouseover on green
mouseenter on green

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Mar 5, 2016

unless i'm misunderstanding, i'd say that it's important that pointer events are consistent in the sequence of events they dispatch, but that it would require quite a bit of extra work to somehow define that the compat mouse events also need to have a self contained mapping. fundamentally, you're trying to shoehorn the concept of mouse (a single pointer device) into a multi-pointer environment. it can certainly be done, but basically you want for every pointer to always "close out" the compat mouse event stream from the idealised single pointer and then "enter" the context of where the pointer just acted. a lot of effort/keeping track i'd say.

my view would be: if a developer is only looking at mouse events in a multi-touch/multi-input environment, events which are there primarily for compatibility, then...they can expect some rough sequence jumps.

@NavidZ

This comment has been minimized.

Copy link
Member Author

commented Mar 7, 2016

I agree with you Patrick. But in this spec we did mention mouseover/enter/out/leave events explicitly to be fired after corresponding pointer events. If you agree firing these events may already cause some inconsistency in the mouse event stream and to make them consistent we need more effort how about not mentioning those events in this spec at all.

Looking at this section:
https://www.w3.org/TR/pointerevents/#compatibility-mapping-with-mouse-events

the suggested solution has the problem we talked about in this issue. Although I can see it mentions that this way of sending compat mouse events are optional and agents can choose a way of sending compat mouse events that is has most compatibility but maybe not mentioning the solution can open future avenues to have a better compat mouse events definition in some other spec or something.

Also I was wondering if you are aware of any developers that depend on the suggested solution in the spec (i.e. the mouse/pointer events in that sequence) when they run on Edge.

@mustaqahmed

This comment has been minimized.

Copy link
Contributor

commented Mar 8, 2016

While waiting for more informed comments on the impact on developers, here are my two cents.

So far we have seen two possible solutions here:
Solution A Maintain strict 1:1 mapping between pointer & legacy mouse events, but break the enter/leave/over/out consistency within legacy mouse events (as in @teddink's event sequence above).
Solution B Make legacy mouse events consistent among themselves even in a multipointer environment, but break the strict 1:1 mapping between pointer & legacy mouse events (as in @navidzolghadr's event sequence above).

IMHO Solution B is better because it breaks only a suggestion in PointerEvent spec regarding "cross-spec" event sequences. On the other hand, Solution A breaks a behavior in an old spec which we may have to support forever (?).

@RByers

This comment has been minimized.

Copy link
Contributor

commented Apr 25, 2016

In practice I suspect either option will be fine for web compat. There are already other ways the mouse event sequence can be violated (eg. window covered temporarily or window moved, many browsers don't properly report mouse state transitions). I agree that solution B is cleaner, but it's more different from what Edge has already shipped. If Edge folks have a reason to prefer solution A that's OK with me. I agree that in any multi-modality scenario mouse events are always going to be a hack, so anything that is shown to be largely compatible with the existing web is fine (and arguably even preferable to changing behavior in a way that could change site behavior in a new way).

@teddink Can you please link to a public bug tracking this issue in Edge?

@jacobrossi

This comment has been minimized.

@RByers RByers self-assigned this Apr 26, 2016

@RByers

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2016

Thanks! Notes from discussion on the call today are here.

Our main concern is the potential for broken-looking UX when switching between mouse and touch input devices on a website designed only for mouse events (not at all an uncommon scenario). Eg. move the mouse over one hover menu then tap on another as in this example. With current Chrome and Edge behavior you shouldn't get into a state where both menus are open at once, but with Ted's proposal you would.

Also it seems wrong for the pointer events spec to define a sequence of mouse events which is inconsistent with the UI Events spec.

Perhaps we could simplify this by having the PE spec define the behavior of a virtual "legacy" mouse device, and leaving the specific generation of mouseover/enter/leave/out events up to the definition in UI Events? @mustaqahmed is going to propose something along these lines.

@RByers RByers changed the title Issue with compatibility mouse events and inconsistency with mouse pointerevent Compatibility mouse transition events should reflect a single logical mouse pointer (instead of being 1:1 with pointer transition events) May 4, 2016

@RByers RByers changed the title Compatibility mouse transition events should reflect a single logical mouse pointer (instead of being 1:1 with pointer transition events) Compatibility mouse transition events should reflect a single logical mouse pointer May 4, 2016

@RByers

This comment has been minimized.

Copy link
Contributor

commented May 4, 2016

On the call we agreed that this high-level direction was good. We'll need to write some good web-platform-tests to cover this behavior change, and @teddink says the Edge team is looking into how hard it will be to implement the changes (so may need to come back with implementation-motivated tweaks). But we're comfortable moving in this direction.

@RByers RByers assigned mustaqahmed and unassigned RByers May 25, 2016

@RByers

This comment has been minimized.

Copy link
Contributor

commented May 25, 2016

Spec is updated, leave the issue open pending some new / updated tests?

@mustaqahmed

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2016

Closing this since the spec issue is now resolved. I will create a new issue in w3c/web-platform-tests.

@mustaqahmed mustaqahmed closed this Jun 9, 2016

@mustaqahmed

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2016

Added the following web-platform-tests issue for the missing test:
Need tests for compat mouse events for multiple pointers.

@mustaqahmed

This comment has been minimized.

Copy link
Contributor

commented Jul 7, 2016

@RByers, @teddink: ptal at the PR for a new test.

RByers added a commit to web-platform-tests/wpt that referenced this issue Jul 7, 2016

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Jul 7, 2016

Just to check, this whole discussion is the reason why on Android, with a paired bluetooth mouse and Chrome 53 Dev with enabled PE, I get some "interesting" results, right? (or is this a separate weirdness, due to the fact that a mouse on Android also "fakes" touch events?)

https://patrickhlauke.github.io/touch/tests/results/#mobile-tablet-keyboard-mouse-events

moving mouse to my test button: pointerover > pointerenter > mouseover > mouseenter

clicking the mouse button (note the first 6 events, where the pointer is going out before coming back in, effectively): pointerout > pointerleave > mouseout > mouseleave > pointerover > pointerenter > pointerdown > touchstart > (gotpointercapture) > pointerup > pointerout > pointerleave > (lostpointercapture) > touchend > pointerover > pointerenter > mouseover > mouseenter > pointermove > mousemove > mousedown > focus > mouseup > click

second click (note the first 6 events, where the pointer is going out before coming back in, effectively): pointerout > pointerleave > mouseout > mouseleave > pointerover > pointerenter > pointerdown > touchstart > (gotpointercapture) > pointerup > pointerout > pointerleave > (lostpointercapture) > touchend > pointerover > pointerenter > mouseover > mouseenter > pointermove > mousemove > mousedown > mouseup > click

move mouse out of button: pointerout > pointerleave > mouseout > mouseleave > blur

@RByers

This comment has been minimized.

Copy link
Contributor

commented Jul 7, 2016

I think it may be related, but it's not the only issue you're seeing. We also just have a couple (what we've now decided are definitely) bugs on Chrome for Android with how we handle mouse. Once we've got those bugs fixed, we should retest this scenario to make sure we handle it properly. There's no reason Android should do anything different than Chrome desktop for these touch+mouse scenarios.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.