Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pointerType: 'dial' #152

Closed
appsforartists opened this issue Oct 31, 2016 · 20 comments

Comments

Projects
None yet
8 participants
@appsforartists
Copy link

commented Oct 31, 2016

@jacobrossi, do you know if there are plans for the Windows Wheel devices to emit pointer events?

This question may be a bit premature, since the three presently-defined pointerTypes have all been commercially successful for quite some time; whereas, the Surface Dial is the first dial pointer I'm aware of to be broadly marketed. Still, since it's one of the few fundamentally-new pointer types, specifying its behavior may be worthwhile.

@shepazu

This comment has been minimized.

Copy link
Member

commented Oct 31, 2016

I agree that it may be useful for us to define a set of events related to a dial input, and adding a "dial" pointerType seems reasonable to me.

Most of the specific functions are already defined using pre-existing events, though, and we should only define those things that are unique to the dial, or define a dial's characteristics in existing terms. Effectively, it's a mouse with rotation, much like a mouse with a mousewheel.

  • x, y: this would be the centerpoint of the dial.
  • size/radius/contact geometry would be available the same way it is with normal pointer events: event.target.style.width and event.target.style.height.
  • rotation should be defined as deltaX, deltaY, or deltaZ (depending on the axis mode), as defined in UI Events.
  • orientation is a possible property we could look at, though I don't think it's distinguishable from the rotation.
  • zoom (and pan) is something we've long had in SVG documents, and I think this is a property of the element or document more than an event; to the extent that it's expressed in events, I think it corresponds to deltaZ.
  • press and hold seems like it could be expressed as a longkeypress (though I don't see that in UI Events anymore)

Can you think of any specific aspect of a dial that can't be expressed in a predefined way?

Notably, you don't have to have a physical dial for these types of event properties to be used; in a AR/VR environment, it could be mapped to some specific gestures (e.g. extending the fingers and twisting the hand).

@dtapuska

This comment has been minimized.

Copy link

commented Oct 31, 2016

How would you support haptic feedback?

@shepazu

This comment has been minimized.

Copy link
Member

commented Oct 31, 2016

How would you support haptic feedback?

Maybe the Vibration API?

@dtapuska

This comment has been minimized.

Copy link

commented Nov 1, 2016

That is a terribly abused API. Chrome is trying to limit it's use. My main concern is what is the target latency of haptic feedback and whether a web browser can actually hit it. Or whether it is a described API of when haptic feedback is applied at certain steps.

@appsforartists

This comment has been minimized.

Copy link
Author

commented Nov 1, 2016

Based on the PointerEvent interface, I'd expect:

  • dial rotation to be reported as twist
  • contact size to be reported as width and height on the event
  • contact with the screen being equivalent to the primary button being pressed/a pen making screen contact (buttons: 1)
  • pressing the dial to be the equivalent of pressing a pen barrel button (buttons: 2)
@shepazu

This comment has been minimized.

Copy link
Member

commented Nov 2, 2016

@dtapuska wrote:

That is a terribly abused API.

How do you mean? Using it for haptic feedback is precisely what it's intended for, per the spec abstract: "Vibration is a form of tactile feedback."

You may have reason to believe the Vibration API is ineffective or suboptimal, but it's clearly intended for haptic feedback.

Chrome is trying to limit it's use. My main concern is what is the target latency of haptic
feedback and whether a web browser can actually hit it.

That seems like a reasonable concern. Also, how quickly can you turn off a vibration / haptic pattern once it's started, if the UI state has moved on (e.g. the user quickly skips to another option)?

Do you have another suggestion for haptic feedback? I guess you knew what answer your original question was likely to elicit, so maybe you have something else in mind?

Or whether it is a described API of when haptic feedback is applied at certain steps.

I'm not sure what you mean here. (Maybe the same start/stop issue I noted above?)

@patrickhlauke

This comment has been minimized.

@dtapuska

This comment has been minimized.

Copy link

commented Nov 2, 2016

It appears the haptic API is declarative...

A haptic event is defined when a rotationchanged event is broadcast.

You control it via controlling the resolution and setting. This gets around the latency problem by being declarative instead of event driven.

https://msdn.microsoft.com/library/windows/apps/windows.ui.input.radialcontroller.rotationresolutionindegrees
https://msdn.microsoft.com/library/windows/apps/windows.ui.input.radialcontroller.useautomatichapticfeedback

@mustaqahmed

This comment has been minimized.

Copy link
Contributor

commented Nov 4, 2016

PointerEvents can be tweaked to represent a "dial" (Windows Wheel) for sure. But before we commit to changes in this direction, we should discuss the pros & cons of other comparable alternatives. Here are my thoughts on one such alternative: WheelEvents could also be tweaked to represent the device too---is it really bad compared to PointerEvents?

Both PointerEvent and WheelEvent (each of them is a direct subclasses of MouseEvent) seem to have some unique winning points. I looked at three properties of a "dial", as follows. I found WheelEvents to be a better choice for two of them, YMMV.

Dial rotation

Dial rotation comes first because it is the most prominent aspect of the new hardware. The dial is symmetric around its axis, so the exact physical angle of the dial position seems immaterial to users, e.g. there is no left/right/up/down position. This is like a mouse-wheel, and unlike stylus devices (possibly with a "calligraphic" tip) for which PointerEvent.twist was added.

The website shows some nice widgets to mark dial position but the physical dial has no mark, to allow defining the "position" arbitrarily through software (which is a great thing IMO). Even if a software widget exactly simulates a physical marker position on the dial, the connection between the widget and the device breaks completely when the device goes off-screen. Consider this experiment: mark the dial (with a marker) at the widget-marked rotation angle, detach the device from the screen, bring it back at a different angle.

Clearly an absolute rotation angle ("twist") from the device would make no sense to the software. Only the rotation offset matters here. This is like WheelEvent.deltaZ as @shepazu mentioned above, and unlike PointerEvent.twist.

So dial rotation is more like turning a mouse wheel than twisting a pen, both physically & for the software interface.

Device diameter

WheelEvent doesn't have anything to offer here. PointerEvent wins, with width & height.

Pointer location

The use-cases shown in Microsoft website seem to suggest that the location of the dial has only a weak association to the content below it. If a dial is off the screen, it's location is irrelevant, unlike any other pointing device. When the dial is about to touch the screen, it doesn't seem to affect the content below it but instead "pops up" a new UI widget, and then controls the part of the widget around it. "Center of of the Widget upon contact" seems to be the only use of its location. Does it have any use case as a pointing device? I didn't find any.

A mouse wheel is not used as a pointing tool (even though it "resides" on a pointing device). We don't move the mouse when turning the wheel---either for the physical awkwardness of simultaneous movements, or to avoid scrolling wrong things accidentally. Dial seems exactly like this: meant to be used by our "non-dominant" hand which is not an expert for precise movement, at least for me.

Thoughts?

@lpd-au

This comment has been minimized.

Copy link

commented Nov 6, 2016

Pointer location

The use-cases shown in Microsoft website seem to suggest that the location of the dial has only a weak association to the content below it. If a dial is off the screen, it's location is irrelevant, unlike any other pointing device. When the dial is about to touch the screen, it doesn't seem to affect the content below it but instead "pops up" a new UI widget, and then controls the part of the widget around it. "Center of of the Widget upon contact" seems to be the only use of its location. Does it have any use case as a pointing device? I didn't find any.

A mouse wheel is not used as a pointing tool (even though it "resides" on a pointing device). We don't move the mouse when turning the wheel---either for the physical awkwardness of simultaneous movements, or to avoid scrolling wrong things accidentally. Dial seems exactly like this: meant to be used by our "non-dominant" hand which is not an expert for precise movement, at least for me.

The video on the docs page linked above does mention around 18:30 that the dial's menu could/should contain different items depending on where it's placed on the screen (link). It is also my understanding that only the black circle with the white menu icons is painted by the OS, other widgets such as colour sliders are painted by the app. Although it may be possible to create a higher level API that's more useful for simple scenarios, it doesn't seem difficult to imagine someone wanting to know the position of the dial to add elements surrounding it manually.

I agree with regards to the nature of pointers, however. When the dial is placed on the screen, it would be misleading to ever consider it a primary pointer, even when it's the first of its type. However, just checking if the pointer is primary in event handlers would effectively disable pointerover/enter and pointerout/leave events for secondary hover-capable pens, probably not a great outcome either. I suspect the end result would be surrounding alot of event handlers with if statements to check if the pointer type is not dial, assuming the adoption rate is high enough for devs to bother, which could quickly get ugly if alternative other-hand input devices ever spring up.

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Nov 7, 2016

Looking at https://www.youtube.com/watch?v=lwsu6lN5M8k, the position of the dial is relevant/important.

When the dial is placed on the screen, it would be misleading to ever consider it a primary pointer, even when it's the first of its type.

it should be primary if it's the first dial type pointer. primary doesn't mean "primary input mechanism". a pen, mouse, touch can also be primary at the same time.

@lpd-au

This comment has been minimized.

Copy link

commented Nov 8, 2016

it should be primary if it's the first dial type pointer. primary doesn't mean "primary input mechanism". a pen, mouse, touch can also be primary at the same time.

I understand, but this presents two potential negative implications:

  • The dial may produce compatibility mouse events.
  • Contrary to the note in the spec, "Authors who desire single-pointer interaction can not achieve this by ignoring non-primary pointers," because the dial is inherently not supposed to be used as a standalone single-pointer.

In my opinion there needs to be some distinction between preferred hand inputs (mouse, touch and pen) and non-preferred hand inputs (dial).

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Nov 8, 2016

the dial is inherently not supposed to be used as a standalone single-pointer.

I can't locate the video just now, but I have seen the demo of using the dial when editing music notation where the dial itself is used as an actual pointer to click/select part of the music sheet to highlight it for editing. So I don't think that statement is necessarily true.

@teddink @jacobrossi or any other MS folks... any thoughts on this?

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Nov 8, 2016

In my opinion there needs to be some distinction between preferred hand inputs (mouse, touch and pen) and non-preferred hand inputs (dial).

I would strongly advise against this sort of approach - related, trying to tackle exactly this sort of "one input is more important than others" philosophy over here w3c/csswg-drafts#690

@RByers

This comment has been minimized.

Copy link
Contributor

commented Jan 11, 2017

In the call today @teddink says there are no plans in the immediate term to add support for the dial to Edge.

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Apr 4, 2017

@patrickkettner any thoughts on this?

@patrickkettner

This comment has been minimized.

Copy link

commented Apr 14, 2017

@patrickhlauke we still don't have plans to add it in the shortterm

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Apr 14, 2017

@patrickkettner shame, there goes my excuse to pester you for a test device ;)

seriously though, looks like this issue should be closed for now then as it won't happen for v2 in any case?

@patrickkettner

This comment has been minimized.

Copy link

commented Apr 14, 2017

might be able to make that happen still. I see what I can dig up.

I don't think it makes sense for v2 since we are so close to having it be cr-able

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Apr 16, 2017

I don't think it makes sense for v2 since we are so close to having it be cr-able

agreed. closing this for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.