mobile/userinput.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>User Interaction</title>
  </head>
  <body>
    <header>
      <h1>User Interaction</h1>
      <p>The possibility to use mobile devices on the go where the user's attention may be limited, coupled with the form factor of these devices that make touch the main means of interaction, mean that Web developers need to pay particular attention to interaction functionalities when they develop applications that are to run on mobile devices.</p>
      <p>Addressing the accessibility of these interactions requires a multi-layer approach. At the device level, accessibility support exposed by operating systems and assistive technologies have made a lot of progress in the past few years. At the browser level, user agents need to interact with assistive technologies to expose the semantics of the web content. At the application level, Web content developers need to follow a set of accessibility principles to help users with disabilities interact with their content. Interestingly, the mobile constraints mentioned above can make mobile users experience similar barriers to people with disabilities and similar solutions can be used to cater for both angles.</p>
    </header>
    <main>
      <section class="featureset well-deployed">
        <h2>Well-deployed technologies</h2>
        <div data-feature="Touch-based interactions">
          <p>An increasing share of mobile devices relies on touch-based interactions. While the traditional interactions recognized in the Web platform (keyboard, mouse input) can still be applied in this context, a more specific handling of touch-based input is a critical aspect of creating well-adapted user experiences, which <a data-featureid="touchevent">Touch Events in the DOM</a> (Document Object Model) enable.</p>
          <p>The <a data-featureid="pointer-events">Pointer Events</a> specification defines a single model for mouse, touch and pen events. It provides a complementary and more unified approach to Touch Events. It includes the <a data-featureid="css-touch-action">CSS property <code>touch-action</code></a> that allows to filter gesture events on elements and that is widely implemented across browsers.</p>
        </div>

        <p data-feature="Vibration">The <a data-featureid="vibration">Vibration API</a> lets mobile developers take advantage of haptic feedback to create new forms of interactions (e.g. in games).</p>

        <p data-feature="Notification">Mobile devices follow their users everywhere, and many mobile users rely on them to remind them or notify them of events, such as messages: the <a data-featureid="notifications">Web Notifications</a> specification enables that feature in the Web environment.</p>

        <div data-feature="Accessibility">
          <p>The <a data-featureid="uaag20">User Agent Accessibility Guidelines (UAAG) 2.0</a> note defines principles and guidelines for user agents to design an accessible user agent interface and communicate with assistive technologies. The supporting document <a data-featureid="uaag20-reference">UAAG 2.0 Reference</a> explains the intent and best practices of UAAG 2.0 success criteria, and lists numerous examples for each of them. Examples that are directly targeted at mobile devices are summarized in the <a href="https://www.w3.org/TR/UAAG20-Reference/mobile.html">Mobile Accessibility Examples from UAAG 2.0 Reference</a>.</p>
          <p>Following the <a data-featureid="wcag21">Web Content Accessibility Guidelines (WCAG) 2.1</a> will make content accessible to a wider range of people with disabilities. The 2.1 revision adds new success criteria and guidelines to version 2.0, including new criteria that have a specific resonance in mobile contexts, such as the <a href="https://www.w3.org/TR/WCAG21/#pointer-gestures">Pointer Gestures</a>, <a href="https://www.w3.org/TR/WCAG21/#target-size">Target Size</a> and <a href="https://www.w3.org/TR/WCAG21/#orientation">Orientation</a> criteria.</p>
          <p>Web content developers may benefit from authoring tools that follow the <a data-featureid="atag20">Authoring Tool Accessibility Guidelines (ATAG) 2.0</a> standard, which provides guidelines for designing Web content authoring tools that are both more accessible to authors with disabilities and that help design content that conforms to WCAG.</p>
          <p>The <a data-featureid="mobile-accessibility-mapping">Mobile Accessibility</a> note explains how WCAG and other accessibility guidelines can be applied to mobile Web applications, as well as to native applications and hybrid applciations using Web components inside native applications.</p>
          <p>The <a data-featureid="wai-aria11">Accessible Rich Internet Applications (WAI-ARIA) 1.1</a> standard provides an ontology of roles, states, and properties that define the semantics of user interface elements and that can be used to improve the accessibility and interoperability of Web content and applications. The <a data-featureid="core-aam11">Core Accessibility API Mappings 1.1</a> standard describes how user agents should expose these semantics to accessibility APIs &mdash; unfortunately, mobile platforms do not yet have fully comprehensive accessibility API mappings and these mappings are only meaningful on desktop platforms for now.</p>
        </div>
      </section>

      <section class="featureset in-progress">
        <h2>Technologies in progress</h2>

        <div data-feature="Game controllers">
          <p>The <a data-featureid="gamepad">Gamepad</a> specification defines a low-level interface that exposes gamepad devices attached to the browsing device, such as those paired with a smartphone via Bluetooth.</p>
        </div>

        <div data-feature="Smooth scrolling">
          <p>As more and more content gets rendered as long scrollable lists, more and more logic is attached to scrolling events, and the quality of the user experience of these actions is highly dependent on their performances. The <a data-featureid="cssom-view">CSSOM View Module</a> determines when scrolling events get fired, and let developers specify the type of scrolling behavior they want.</p>
          <p>The proposed work on <a data-featureid="css-scroll-snap">CSS Scroll Snap Points</a> adds greater ability to control the behavior of panning and scrolling by defining points to which an app view would snap when the user moves through the page.</p>
          <p>The <a data-featureid="css-will-change">CSS <code>will-change</code></a> property is also available to indicate to browsers that a given part of the page will be soon scrolled to and should be pre-rendered.</p>
        </div>

        <div data-feature="Notification">
          <p>The <a data-featureid="push">Push API</a> makes it possible for server-side notifications to alert the user, even if the browser is not running.</p>
          <p>The <a data-featureid="badging">Badging API</a> defines a more subtle notification mechanism than <a href="https://www.w3.org/TR/notifications/">Web Notifications</a>, allowing Web applications that have been installed on the device (e.g. through a manifest file) to set an application-wide badge, typically shown next to the application's icon on the home screen, to notify the user when the state of the application has changed and might require their attention (e.g. a new message has arrived).</p>
        </div>

        <p data-feature="Screen wake">Whether users are speaking commands to their apps or working with them through non-haptic interactions, they risk seeing the screens turned off automatically by their devices screensaver. An early proposal for a <a data-featureid="screen-wake-lock">Screen Wake Lock API</a> would let developers signal the needs to keep the screen up in these circumstances.</p>
      </section>

      <section class="featureset exploratory-work">
        <h2>Exploratory work</h2>

        <p data-feature="Speech-based interactions">Mobile devices, and mobile phones in particular, are also in many cases well-suited to be used through voice-interactions; the <a href="https://www.w3.org/community/speech-api/">Speech API Community Group</a> developed a JavaScript API to enable interactions with a Web page through spoken commands. <a data-featureid="speech-api/synthesis">Speech synthesis</a> is well supported across browsers. Support for <a data-featureid="speech-api/recognition">speech recognition</a> is still underway.</p>

        <p data-feature="Input method">The <a data-featureid="ime-api">Input Method Editor (IME) API</a> provides Web applications with scripted access to an IME (input-method editor) associated with a hosting user agent. Editorial support is required for this specification to move forward.</p>

        <div data-feature="Touch-based interactions">
          <p>The proposal for an <a data-featureid="inputdevice">Input Device capabilities API</a> would provide information about whether a given “mouse” event comes from a touch-capable device.</p>
        </div>

        <div data-feature="Game controllers">
          <p>Gamepads exist in a variety of flavours, from <em>usual</em> console gamepads to custom devices such as guitars, pedals, dance pads, scratching gamepads, magic wands, or VR/AR controllers. Each of these devices has its own inputs that need to be mapped into native input APIs for them to appear to Web applications, and outputs (LEDs, vibration, etc.) that are mostly unavailable on the Web. <a data-featureid="webhid">WebHID</a> proposes to expose the <abbr title="Human Interface Device">HID</abbr> protocol that most of these devices use under the hoods to Web applications, allowing them to support the long tail of HID devices through JavaScript-based logic when the browser lacks support. Looking forward, this proposal could also ease integration between smartphones and all sorts of tangible user interfaces.</p>
        </div>

        <div data-feature="Responsiveness">
          <p>Applications that need to run long tasks (e.g. when pages are loaded) need to make a tradeoff between loading pages quickly (or reducing task execution duration) and responding to input quickly. The <a data-featureid="is-input-pending">Early detection of input events</a> specification proposes an <code>isInputPending</code> method that long running scripts can call synchronously, without losing time yiedling to other scripts and events processing, to detect whether there are pending input events that their execution might delay from firing.</p>
        </div>

        <div data-feature="Input latency">
          <p>Today, all DOM events need to go through the main thread, which is the only one who has access to DOM elements. The <a data-featureid="input-for-workers">Input for Workers and Worklets</a> specification proposes an event delegation scheme to workers that assumes no DOM access. This mechanism would enable latency sensitive event dependent logic, which would no longer be blocked by the main thread. Use cases include drawing on an <a data-featureid="offscreencanvas" data-linkonly><code>OffscreenCanvas</code></a>, cloud scenarios (e.g. cloud gaming) where input events need to be forwarded to a server with minimal latency for processing, interactive animations in conjunction with <a data-featureid="animation-worklet" data-linkonly>CSS Animation Worklet</a> where animation is driven by user input, and interactive audio in conjunction with <a data-featureid="webaudio/worklet" data-linkonly><code>AudioWorklet</code></a>.</p>
        </div>
      </section>

      <section>
        <h2>Features not covered by ongoing work</h2>
        <dl>
          <dt>Gesture events</dt>
          <dd>As mentioned above, touch-based interaction is common on mobile devices and available to Web applications through <a data-featureid="touchevent">Touch events</a>. <strong>Gesture-based interaction</strong>, which includes pinching, rotating and swiping, is also a common interaction paradigm on mobile devices. Web developers may derive gesture events from touch events to some extent, but may have to develop multiple versions for different browsers. Native support for these interactions would reduce fragmentation and improve performance. Early discussions to define <a href="https://github.com/JuntaoPeng/GestureEvents/blob/master/GestureEvents.md#gesture-events">Gesture events</a> have started in the <a href="https://www.w3.org/community/mwma/">Merging of Web and Mobile APP Community Group</a>.</dd>
        </dl>
      </section>

      <section>
        <h2>Discontinued features</h2>
        <dl>
          <dt>Intent-based events</dt>
          <dd>As the Web reaches new devices, and as devices gain new user interactions mechanisms, it seems useful to allow Web developers to react to a more abstract set of user interactions: instead of having to work in terms of “click”, “key press”, or “touch event”, being able to react to an “undo” command, or a “next page” command independently of how the user instructed it to the device. The <a data-featureid="indie-ui-events">IndieUI Events</a> specification was an attempt to address this need. The work has been discontinued for now, due to lack of support from would-be implementers.</dd>
        </dl>
      </section>
    </main>
    <script src="../js/generate.js"></script>
  </body>
</html>