Should a captured pointer send boundary events by default?

[RByers]

Edge and the spec send pointerleave/pointerout when a captured pointer moves outside the bounds of an element (and it's descendants), and pointerenter/pointerover whenever it moves back in.

There's been some discussion in various places about the implications of this and whether it should change, so I offered to file a dedicated issue to summarize the arguments to see if we can come up with a plan here. In particular, this is a subset of Chrome's primary ship-blocking spec issue: #8.

I'll try to keep the above summary updated as we debate the details in the conversation below.

Advantages of sending transition events during capture

1. Edge already ships this, we shouldn't consider changing without a really good reason.
2. This is useful for capture scenarios like buttons where you want the button style to "pop back out" when the cursor is dragged off the capturing element.
   • But in that case, should the button really even be capturing? It needs to also know to ignore click events seen after pointerleave, so it's mostly acting like an uncaptured control, right?

Disadvantages of sending transition events during capture

1. It requires the UA to do hit-testing on each pointermove events during capture (Pointermove should not require a hit-test by default for touch #8)
   • Some fast paths may be possible in common cases, but detecting these cases is hard in itself. Eg. if it's possible for a capturing element to be occluded by another element (possibly even via an accelerated animation), then a simple bounds check is insufficient.
2. It means the semantics of the transition events are fundamentally different during capture
   • Normally an element will not receive pointermove events after it sees a pointerleave. As currently defined, when an element has capture the transition events no longer say anything about the event stream.
   • pointerover and pointerout events normally apply only to the immediate bounds of the element, but under capture they (currently) behave like pointerenter and pointerleave where they reflect the bounds of the captured element and all it's descendants. We're struggling to define this precisely (Clarified definition of capturing element borders #63, Determine when boundary events are fired after capture is released #15)
3. It means capture can't be defined simply as a modification of (the as yet unspecified) hit-testing algorithm.
   • A very conceptually simple model for capture would be to say that it's just an early-out from hit-testing - makes all pointer hit tests return the captured node. This is consistent with how implicit touch capture works in browsers like Safari and Chrome today.
4. In drag and drop scenarios (a common use case for capture) the events are only a potential hinderance
   • Typically in drag and drop the application will rely on it's own intersection testing (does the dragged item overlap a drop target at all) rather than built-in hit testing
   • If you start dragging near the edge of an element and drag slowly, you may receive a constant stream of pointerleave and pointerenter events as the dragged element is continually repositioned to remain underneath the mouse cursor.

Possible alternatives

1. Don't send transition events during capture at all
   • I.e. define capture to be just a modification of pointer hit testing (affecting both :hover and transition events).
   • Scenarios wanting capture with hover-style behavior (like the button scenario) would need to implement their own hit-testing (elementFromPoint) or boundary checks on pointermove listeners. This should be simple to do.
   • We'd need to do some compat testing to determine the potential for real-world breakage here. It's not clear how to measure this in practice (just because a pointerlevae handler is invoked during capture doesn't mean that the handler's behavior was actually desired by the developer). Thoughts?
2. Add a boolean option to setPointerCapture to specify the behavior
   • Depending on the default value, this would strike a tradeoff between the two options

What have I missed / got wrong in this summary? Other arguments / details? /cc @scottgonzalez @jacobrossi,

[dmethvin]

Hit testing and calling elementFromPoint inside the event handler itself are likely of equivalent cost? Both need to work from a clean layout. If you could push hit testing off to be handled for the rare cases where it is actually desired, rather than taking a constant penalty for it to generate transition events, that sounds like a good thing. It also avoids the need to worry about strange cases where elements change size under the pointer while it's captured and moving.

An extra arg to setPointerCapture for the caller to define whether they wanted transition events would take care of it if you don't want to make the event handler do its own elementFromPoint, although the "you may or may not get transition events" aspect of it might not be good from a documentation standpoint? I would lean towards taking transition events out entirely just to keep things simple and predictable.

[RByers]

Thanks Dave!

Hit testing and calling elementFromPoint inside the event handler itself are likely of equivalent cost?

Yes. However it's possible that manual use of elementFromPoint could trigger an extra layout (if you're not careful to order your DOM reads/writes) that (depending on if/how the browser aligns input with the document lifecycle) might be avoidable in the built-in hit-testing case. In the real pathological cases it's often the layout that's really expensive, not the hit-test itself. But I don't think this is likely to be a big issue in practice.

the "you may or may not get transition events" aspect of it might not be good from a documentation standpoint?

Agreed, that would be ugly and confusing (would only exacerbate the "transition events have 2 fundamentally different semantics" concern).

[scottgonzalez]

In the real pathological cases it's often the layout that's really expensive, not the hit-test itself. But I don't think this is likely to be a big issue in practice.

This actually just came up with a PEP user (unrelated to the transition events during capture though): jquery-archive/PEP#280

I think there will be a lot of work to educate developers about properly throttling DOM manipulation. The tools are available, but the community as a whole still has a long way to go. But this is a much broader issue than what's being discussed here.

[RByers]

We discussed this on the call today and nobody present could explain why the button scenario would want capture at all, it's most easily implemented by not using capture and having only a click listener and CSS :hover styling.

I argued there are 3 consistency principles we're trying to uphold here:

1. CSS hover matches pointerenter/pointerleave state
2. pointerenter/pointerleave state tell you what part of the tree will receive pointer events
3. during capture, only the captured element receives any pointer events
   The simplest model that does all this is the "capture overrides hit testing" concept. On capture, transition events behave as if the pointer moved to the captured element (gets pointerenter and CSS hover) and stays there for the duration of capture. On release behaves as if the mouse jumped to it's current position.

Chrome is going to proceed with implementing this model for now (since it's much simpler and we don't have a good understanding of why the more complex Edge behavior is useful) and get some data on web compat. In parallel @teddink is going to talk with @jacobrossi and see if they can explain the benefits of Edge's current behavior.

[RByers]

@scottgonzalez / @smaug---- suggest using the term "boundary events" to avoid confusion with transition events.

[teddink]

Here is a short video of the Windows 10 built in "modern" Mail app: Mail App Video
Notice how the pointer gets captured to the button that the is first touched, then no other buttons or anywhere else on the screen is getting pointer events. When the finger is returned to the original button and released, then the action for the button is completed. This is the button concept that we are trying to replicate on the web and why we need pointerenter and pointerleave events on the element that has captured the pointer.

Another example, although contrived and clearly not something that a designer would ever sign off on, can be seen here: http://teddin.azurewebsites.net/test/pointereventbutton.html
Pointerdown in the yellow div captures the pointer and changes the color and border. When the user moves outside that border, the div changes back to yellow, while the border remains to indicate that the div has capture. While captured by the bordered div, the green div never receives the pointerover events it is listening for. Moving back to the yellow bordered div, turns it blue again to indicate that the pointer could now complete the action that would be associated with the "button".

Does this better explain the button scenario referenced above?

[patrickhlauke]

Here is a short video of the Windows 10 built in "modern" Mail app

that is essentially what I was talking about in the last call (when mentioning also how this provides the ability for users to change their mind after they started to put their finger on the button, but then don't want to execute).

i think it was scott who then said that you wouldn't really do this with pointer events and capture, but just listen for click (for the action itself, which will only trigger when the end point when the finger was lifted is in the same element as where it started). probably throw in a touch-action:none. but (and i've not tested this) i think the actual hover state/highlighting couldn't be achieved with pure CSS ... which is why you may need to capture the pointer, react to enter/leave to apply/remove some CSS explicitly. i assume that the captured pointer would NOT fire a click if its final position once pointerup happens is outside of the original element (regardless of whether or not it was captured), right?

[NavidZ]

Although that particular UI seems weird to me but I guess that's just me and I totally agree we should make it possible for a developer to implement that UI.
@teddink, let me ask you this question regarding the boundary events in the case of that button since I got a bit confused how much of it is automated. Do you expect the developers to change the style of the button based on pointerleave/enter as @patrickhlauke suggested?
How about this as an alternative solution? Correct me if I'm wrong here. If we remove the hit testing while a pointer is captured (meaning we don't send the boundary events) developers who want to implement that UI can still do an element boundary check of "pointermove" events with a much smaller cost I assume in this case or the worst case just using elementFromPoint which has the same performance hit in the worst case as us always doing the hit test. So basically what I'm saying is that this UI you have in mind is still very well possible even if we don't send boundary events and of course not in a very hard way as it's just a boundary check of the element. I guess counting the lines this will probably be the same as adding the handlers to pointerleave/enter and toggling the style.

[scottgonzalez]

As far as I can tell, Edge doesn't implement that behavior for native <button> elements (there's no implicit capture). So this would be a custom implementation anyway, which gets us to the question that @NavidZ asks: is it important to have the browser do hit testing for you when you can do it yourself? The logic is extremely similar to drag and drop scenarios where you want to capture on the draggable element and do manual hit testing for the droppable elements. In this case, they just happen to be the same element, but the implementation is the same.

[teddink]

@scottgonzalez - Correct, Edge does not implement an implicit capture on the <button> elements.

Regarding the hit testing aspects of the discussion - I do not have any tests or data, but I do wonder if the UA can make more optimizations in its hit testing than the web developer can in JavaScript. To put it another way, I am questioning whether a hit testing loop in JavaScript is really equivalent to a hit testing loop inside the UA.

I do agree that the same behavior that I was able to produce in my sample could be accomplished with hit testing in JavaScript and does not specifically require the enter and leave events. The code does seem simpler to me with the enter and leave events - but that may just be me.

[RByers]

To summarize, I think we're all on the same page wrt. the use case, the debate is just centering around how easy/hard the different ways to implement that use case are:

1. Using pointer capture and boundary events (what works in Edge today but this issue is questioning) with custom click-detection logic
2. Using pointer capture and explicit hit testing (eg. elementFromPoint or simple bounds checks) with custom click-detection logic.
3. Not using pointer capture, but using boundary and click events. Suppress events reaching other targets with a temporary window-level capturing listener that does stopImmediatePropagation.

Does someone want to try writing the simplest possible code they can for these 3 scenarios so we can compare them concretely?

Of course ease of development is just one issue here, once we have some conclusion from that we also need to consider (arguably more important):

1. Complexity of the model (eg. this issue seems to make it hard for us to precisely define / reason about the behavior)
2. Compatibility impact of changing this

I do wonder if the UA can make more optimizations in its hit testing than the web developer can in JavaScript

That's not the case in blink at least. elementFromPoint is basically a direct path into the same hit testing algorithm we use internally, and I see that as an important property (hit-testing is a key primitive of the platform, if we internally had access to some more optimized version, we really should be exposing that to web developers somehow too).

[smaug----]

But you could optimize out most of the hit testing if you detected the relevant area in compositor side.
(however, I don't have a strong opinion what the spec should say about this.)

[scottgonzalez]

Implementations of various approaches to the button capture use case:

• boundary events: http://jsbin.com/zegubuyabi/1/edit?js,output
• elementFromPoint: http://jsbin.com/yeyiyawici/1/edit?js,output
• bounding box: https://jsbin.com/havubim/edit?js,output

The bounding box example doesn't work properly in Edge, event.offsetX and event.offsetY seem to be incorrect. I couldn't figure out how those values were being calculated.

[NavidZ]

Maybe using clientX/Y is better for that? I don't know if there is a case that this one missing
https://jsbin.com/havubim/edit?html,css,js,output

[scottgonzalez]

That works. I've updated the links in my previous comment. Thanks.

[RByers]

Thanks Scott. For the captured w/ boundary events case we also need a bit of custom logic to handle clicks, right? I guess it's just a click handler that bails out if the active class isn't applied? It's having to ignore some click events that makes using capturing feel a little hacky to me (the button doesn't really want all events, it just doesn't want anyone else to get them).

[scottgonzalez]

If you press down on button A, move to button B, and release, you shouldn't get a click on either of the buttons.

[RByers]

Oh, right - IMHO that violates the spec for click (which is defined to go to the common ancestor of the mousedown and mouseup targets. @NavidZ has been looking at that and will file a separate issue. In the "capturing alters hit-testing" model we're advocating, I would expect a click event on the captured node, regardless of where the pointer moves to (but we could define it as a special case if desired - explicitly doing a capture-independent hit-test on pointerup).

[scottgonzalez]

In the "capturing alters hit-testing" model we're advocating, I would expect a click event on the captured node, regardless of where the pointer moves to

Assuming you're in a sane scenario where the pointer is captured to an element that received pointerdown. In that case, both pointerdown and pointerup will occur on the captured element, regardless of where the pointer is actually positioned. As a result, you should get a click event without any changes to the spec.

[teddink]

Similar to my comment on #75, if an element gets both pointerdown and pointerup, I would expect that element to get the click event as well.

[teddink]

I have some initial data that we can analyze more deeply. Here is what I did - I grabbed the top 200 sites form our usetracker data that use pointer events. I then did static analysis from crawler data to find sites that had the text setpointercapture, gotpointercapture, and lostpointercapture. I then did searches across the usertrackerdata to figure out what sites both used pointer events AND had a capture related call somewhere on the site.

Pointer events in use AND setpointercapture found in static analysis:
yandex.ru
yandex.com.tr
www.trulia.com
vimeo.com
www.google.com

Pointer events in use AND gotpointercapture found in static analysis:
yandex.ru
yandex.com.tr
play.google.com

Pointer events in use AND lostpointercapture found in static analysis:
yandex.ru
yandex.com.tr
play.google.com
www.google.com
www.trulia.com
www.gumtree.com
www.wattpad.com

NEXT STEPS:

• I want to re-run the analysis with the top 1000 sites from our usetracker just to ensure that we have better coverage.
• I want to also run static analysis and comparisons for pointerleave, pointerout, pointerenter, and pointerover.
• We need to do deeper analysis and debugging on each site (thankfully it is a short list) to figure out how they are using pointer events to really figure out if they would be adversely affected by any changes we would propose to make in this area.

[teddink]

Oops - did not mean to close the issue - reopening.

[RByers]

In the meeting at TPAC we agreed that for now we're OK with not sending boundary events by default - as written in the reduce-hit-tests branch. We've got an open question about what is the best behavior for explicit capture (and whether the capture API should change) - filed #143 for that.