Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
107 lines (86 sloc) 5.86 KB

Ambient Pen Events API Explainer

Motivation:

Some digital pens and pencils can not only provide input through their interaction with a digitizer but can also pair with a device so that additional signals can be received when a button is pressed. In some cases, the signal may relate to the pen or pencil but come from another source, e.g. a charger may send a signal that the pen or pencil has been docked or undocked. Native applications can use these signals to customize their behavior, but no corresponding events are available to web applications.

Native applications use these signals in a variety of ways; here are some inspirational use cases:

  • Clicking the button on a Surface Pro pen can advance to the next slide in a slide show
  • Double tapping the side of an Apple Pencil can switch drawing tools
  • Removing a pen from the dock on a Surface Hub prompts the user to enter the whiteboard app

Providing these new pen event primitives would enable web applications to achieve parity with native applications.

Proposed Events

Event Type Interface Target Bubbles Cancelable Composed Default Action
penbuttonclick PenButtonEvent Window No No No None
penbuttondblclick PenButtonEvent Window No No No None
penbuttonpressandhold PenButtonEvent Window No No No None
pendockchange PenDockChangeEvent Window No No No None

PenButtonEvent Webidl

dictionary PenButtonEventInit {
    readonly attribute long pointerId = 0;
    readonly attribute long button = 0;
};

[
    Constructor(DOMString type, optional PenButtonEventInit eventInitDict),
    Exposed=Window
] 
interface PenButtonEvent : UIEvent {
    readonly attribute long pointerId;
    readonly attribute long button;
};

penbuttonclick, penbuttondblclick, penbuttonpressandhold Events

A PenButtonEvent named for the gesture the user has performed with the pen button must be fired to the active window when the pen button gesture is detected. Corresponding down/up events are not proposed because of hardware limitations, e.g. the Microsoft Surface Pen produces a signal describing the gesture, not the state transitions of the actual button.

The pointerId property identifies the pen which triggered the event. If the user agent supports providing a stable pointerId for PointerEvents generated by the pen, that pointerId must also be used for PenButtonEvents. For devices that don't support stable pointerIds, the pointerId must be set to 0.

The button property indicates which button has been pressed. The numeric value of the button indicating which button triggered the event is taken from the recommendations made in the existing PointerEvent specification.

PenDockChangeEvent Webidl

dictionary PenDockChangeEventInit {
    readonly attribute long pointerId = 0;
    readonly bool docked = false;
};

[
    Constructor(DOMString type, optional PenDockChangeEventInit eventInitDict),
    Exposed=Window
] 
interface PenDockChangeEvent : UIEvent {
    readonly attribute long pointerId;
    readonly bool docked;
};

pendockchange Event

A PenDockEvent must be fired to the active window when a pen is returned to, or removed from, a dock on hardware which supports detecting these transitions of the pen.

The pointerId property identifies the pen which triggered the event. If the user agent supports providing a stable pointerId for PointerEvents generated by the pen, that pointerId must also be used for PenDockChangeEvents. For devices that don't support stable pointerIds, the pointerId must be set to 0.

The docked property identifies whether the pen has transitioned to a docked state (the docked property is true) or an undocked state (the docked property is false).

System Interactions

Note on some operating systems, adding or removing the event listener may cause side-effects. For example, on Windows, the shell provides configurable behavior for launching frequently used inking applications when pen button interactions are detected. Applications may register with the system to provide alternative behaviors and suppress the default behavior from the OS. For web applications, it is expected that only active documents that have a registered event listener can override the system behavior.

Sample Code

PenButtonEvent Example

document.addEventListener('penbuttonclick', event => {
  if (event.button === 5) {
      // eraser button clicked; move to next slide in the slide show
  }
});
document.addEventListener('penbuttonpressandhold', event => {
  if (event.button === 5) {
      // eraser button held; move to previous slide in the slide show
  }
});

PenDockChangeEvent Example

document.addEventListener('pendockchange', event => {
  if (event.docked) {
    // the pen is docked, handle it, e.g. by transitioning out of drawing mode
  }
  else {
    // the pen has been undocked; drawing may be imminent; deploy color palettes!
  }
});

Open Questions:

  1. Microsoft Surface Hubs have more than one pen and more than one dock to hold those pens. Should a dock identifier be considered so that web apps could, for example, position drawing UI nearby the location of the dock which generated the event?
  2. Is an API needed to query the docked state of a pen, or is it sufficient to wait for the event to signal a transition?
  3. Should the API be more generic and not pen specific?
  4. Should penbuttonup/down events be defined and synthesized for devices that don't support that fidelity?
  5. Cross-platform compatibility: Does the Apple Pencil support detecting the double-tap gesture or is that only handled by the OS? Same question for the Chromebook Pen...
You can’t perform that action at this time.