Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable direct pen and touch to have different touch-action behavior #203

Open
RByers opened this issue May 5, 2017 · 18 comments
Open

Enable direct pen and touch to have different touch-action behavior #203

RByers opened this issue May 5, 2017 · 18 comments
Labels
v3

Comments

@RByers
Copy link
Contributor

@RByers RByers commented May 5, 2017

Today a stylus in Chrome Android behaves as a direct manipulation input device (with the implications on touch-action discussed in #202). This works well in practice except for one special case:

In some situations like note taking applications, you might want the pen to draw by default, while still allowing a finger to scroll. That's not possible to express with touch-action today.

I propose we address this by defining a new pen-action property that is defined to apply ONLY to "direct manipulation pen devices", with (for now) only two possible values:

  • auto: use the value of touch-action. I believe this must be the default for web compat reasons (again see #202).
  • none: disable all pen-based panning (and zooming if it exists), just send events.

For implementation (and specification) complexity reasons I'd prefer to avoid adding other properties like pan-x etc. I think from a UX perspective you're always going to want the pen to behave either identically to the finger, or to draw.

@ChumpChief and @atanassov have told me they're interested in exploring this on Edge.

@NavidZ @mustaqahmed @patrickhlauke WDYT?

@patrickhlauke

This comment has been minimized.

Copy link
Member

@patrickhlauke patrickhlauke commented May 5, 2017

would we consider this for v2 still? concerned about this (and the 2 implementation requirement) potentially prolonguing the path to TR (which we've not embarked on yet).

with that out of the way, this sounds reasonable. it's another "why are we doing this in CSS if it's about behavior?" one, but that boat already sailed with touch-action and the unrelated pointer-events properties I guess...

@scottgonzalez

This comment has been minimized.

Copy link
Member

@scottgonzalez scottgonzalez commented May 5, 2017

Have you considered extending touch-action to allow values based on pointer type?

Perhaps something similar to linear-gradient:

touch-action: "auto", some-other-keyword("none", "pen");

Not sure what would be a good name for some-other-keyword, but the idea is that it would take the touch-action value and then a list of pointer types. Some other syntax could be used as well. I'm just a little nervous about adding a type-specific property when Pointer Events is all about type abstraction.

@RByers RByers added the enhancement label May 5, 2017
@RByers

This comment has been minimized.

Copy link
Contributor Author

@RByers RByers commented May 5, 2017

would we consider this for v2 still?

Nah, not worth delaying v2 for to me. I'd be fine considering it for level 3 (an extension for now).

Have you considered extending touch-action to allow values based on pointer type?

Yeah my concern with that is just around web compat. That rule will fail to parse in browsers that don't support it, so developers will end up having to specify two rules for at least the next 5 years anyway. I think it makes such compat bugs less likely if we just introduce a new property.

I'm just a little nervous about adding a type-specific property when Pointer Events is all about type abstraction.

That's a good point. We should totally think about how it would interact with a new/unexpected pointerType. In the drawing app scenario where you've got the pen doing inking but the finger scrolling, if some other direct manipulation pointer device was used (I have no idea what that would be since the esoteric device are all AFAIK indirect manipulation) should that device do inking or scrolling?

Another way we could address your concern is with a generic property that has a list of pointer types. Eg:

touch-action-pointers: "pen"

The default would be "pen","touch", and it would apply only to direct manipulation pointers of that type (so you couldn't, for example, use this to ask for a mouse device to start scrolling on drag). Any direct manipulation devices not specified here would behave as touch-action: auto.

@mustaqahmed

This comment has been minimized.

Copy link
Contributor

@mustaqahmed mustaqahmed commented May 5, 2017

I think touch-action-pointers is a cleaner solution.

I have different concern though: any of these ideas (touch-action-pointers or pen-action) adds another dimension to how pens behave, on top of InputDeviceCapabilities.pointerMovementScrolls. I have a feeling that one of them should be redundant when we already have pointerType.

For example, the outcome is not obvious when pen events have pointerMovementScrolls == false but the same events are supposed to pan-y. (This is true for pointerType="touch" even today, but perhaps nobody expects pointerMovementScrolls == false here.)

Am I the only one to find three levels of parameters (type + capabilities + action) too complicated? Any chance we may not need pointerMovementScrolls at all if we have touch-action-pointers?

@patrickhlauke

This comment has been minimized.

Copy link
Member

@patrickhlauke patrickhlauke commented May 5, 2017

Have you considered extending touch-action to allow values based on pointer type?

the danger i see here is that touch-action is becoming a bit of a clown-car of values, personally...

@RByers

This comment has been minimized.

Copy link
Contributor Author

@RByers RByers commented May 5, 2017

Am I the only one to find three levels of parameters (type + capabilities + action) too complicated? Any chance we may not need pointerMovementScrolls at all if we have touch-action-pointers?

It's definitely complicated but I think they represent different thing. touch-action is an API from the developer to the browser saying where they do and do not want scrolling. pointerMovementScrolls is an API from the browser to the developer saying which devices support touch-action at all. Maybe there's a better name for that? I'm also not convinced pointerMovementScrolls is that valuable - at the PE hackathon people were just concerned about how touch behaves differently than other pointers and not wanting developers to make assumptions about pointerType related to that.

@NekR

This comment has been minimized.

Copy link

@NekR NekR commented May 5, 2017

I actually thought about something similar as @scottgonzalez suggested, but kind of in revers:

pointer-action: touch(pan-y), pen(none);  

touch() and pen() functions here are corresponding to supported/possible pointerType. Why it might be good:

  • Future compatible: with any future type like dial
  • Backwards compatible: the new property. Doesn't break anything, can support touch and mouse
  • Allow rich customization for different types of input devices, not only touch or pen specific
  • Unknown function are just ignored
@patrickkettner

This comment has been minimized.

Copy link

@patrickkettner patrickkettner commented May 9, 2017

I love the concept of allowing for different touch-action like behaviors for each type of pointer, but am opposed to having new *-action values because of the unbounded nature of future pointer types.

This feels like the exact sort of reason @media blocks were invented. What about

@media(pointer-type: touch) {
  #carousel {
    pointer-action: pan-y
  }
}

@media(pointer-type: pen) {
  #carousel {
    pointer-action: none
  }
}
@NekR

This comment has been minimized.

Copy link

@NekR NekR commented May 9, 2017

@patrickkettner I thought about this, but we already have @media(pointer: coarse) which was made abstract on purpose, @media(pointer-type: touch) will destroy any purpose of @media(pointer: coarse).

Also problem here is to ably 2 different pointer-actions (one per pen, other per touch) at the same time. I'm not sure how media query would help with that. Do you mean that media query will be automatically applied when one type of a pointer is used? What about 2 at the same time?

@NekR

This comment has been minimized.

Copy link

@NekR NekR commented May 9, 2017

If something, I also feel that pointer-action: touch(pan-y), pen(none); syntax doesn't look quite right. Maybe it's possible with CSS grammar to have a better one?

@patrickkettner

This comment has been minimized.

Copy link

@patrickkettner patrickkettner commented May 9, 2017

I thought about this, but we already have @​media(pointer: coarse) which was made abstract on purpose

its solving a completely different problem. you could very much want a different *-action of input on something like a trackpad than you would a mouse (e.g. locking panning or scrolling) and both would be fine. Not to mention the idea that other potential pointer types like dials don't fit into either fine or coarse.

media(pointer-type: touch) will destroy any purpose of @​media(pointer: coarse).

firmly disagree. even if it did, a more useful interface shouldn't be stopped because a lesser one already exists.

Also problem here is to ably 2 different pointer-actions (one per pen, other per touch) at the same time.

thats not a problem? you can do that today

@media (min-width: 500px) {
  background: blue
}

@media (width: 500px) {
  background: red
}

both will apply at 500px wide, and if they interfere with one another source order precedence exists.

Do you mean that media query will be automatically applied when one type of a pointer is used?

yep, just like :hover and :selected pseudo selectors, they apply based on state of the element.

What about 2 at the same time?

see above

@NekR

This comment has been minimized.

Copy link

@NekR NekR commented May 9, 2017

its solving a completely different problem.

I didn't mean it solves this problem. I mean that pointer-type media query could be started used instead of pointer media query and the idea of pointer media query was to abstract types from concrete, like touch. But yes, pointer media query is unrelated to the problem in this issue. I'm just worried about its future.

thats not a problem? you can do that today

Sorry, I don't get. Having element background property set to red and blue doesn't make it having red and blue at the same time. Same with pointer-type. I doubt that someone will be scrolling and drawing at the same time though.

they apply based on state of the element.

On the whole document then because @media query are global (unless shadow dom?) to the document (just correction, not that I disagree here).

@patrickkettner

This comment has been minimized.

Copy link

@patrickkettner patrickkettner commented May 9, 2017

@patrickkettner

This comment has been minimized.

Copy link

@patrickkettner patrickkettner commented May 16, 2017

After speaking with Matt Rakow about this more, due to the number of potential edge cases with my proposal, I withdraw my complaints about pen-action. I am fine with it

@phistuck

This comment has been minimized.

Copy link

@phistuck phistuck commented Oct 28, 2017

Looks like EdgeHTML 16 shipped pen-action.
https://codepen.io/MicrosoftEdgeDocumentation/pen/jLGZZY

@patrickhlauke patrickhlauke added v3 and removed enhancement labels Feb 15, 2018
@patrickhlauke

This comment has been minimized.

Copy link
Member

@patrickhlauke patrickhlauke commented Jul 10, 2019

Discussed at today's PEWG meeting (https://www.w3.org/2019/07/10-pointerevents-minutes.html) ... there's a desire to look at a more input-agnostic model (or at least a more neutral "naming", so we don't end up with lots of different [pointertype]-action css properties every time a new pointer type gets introduced. this feels most in line with the agnostic/generic idea of PE. keeping this open for now though for further discussion.

@BoCupp-Microsoft

This comment has been minimized.

Copy link

@BoCupp-Microsoft BoCupp-Microsoft commented Sep 15, 2019

Here's a link to the pen-action explainer that was discussed and resulted in the comment Patrick wrote above.

@BoCupp-Microsoft

This comment has been minimized.

Copy link

@BoCupp-Microsoft BoCupp-Microsoft commented Sep 15, 2019

Regarding a more generic proposal in response to the WG feedback, @gked and I played with some syntax and arrived at the same proposal by @NekR above.

pointer-action: pen(none) touch(pan zoom);

We like the future-proofing of this proposal. Here are some additional specifics that I imagined were implied by it:

The type should parse correctly for any string that syntactically could be a valid future pointer-type. The property should also parse correctly for any string that syntactically could be a valid future action.

For example, pointer-action: gaze(pan); could be a valid future pointer-action value and should be allowed by user agents.

The user agent is not obligated to recognize and fulfill any particular action, however it is obligated to limit the actions possible to those that have been explicitly put into the list. For example, pointer-action: pen(none);, not only means that all pen input should generate pointer events, but also means that touch is unaffected as it isn't explicitly listed. It also means that if gaze(pan) is ever supported that the user agent should consider it still enabled.

If the author wants to ensure no actions are possible (the author always wants to see the events), pointer-action: all(none); can be specified.

The touch-action property would become deprecated and equivalent to pointer-action: all(touch-action values here); or pointer-action: touch(values here) pen(values here);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.