On-screen gamepads

[nondebug]

It is common for mobile games to implement on-screen touch controls instead of or in addition to gamepad support. When mobile games use on-screen controls they can vibrate the device using Vibration API. However, Vibration API is not consistently supported across browsers (it's only fully implemented and working in Chromium browsers) and has a high potential for abuse.

In w3c/vibration#33 Gamepad API is proposed as a potential avenue for supporting vibration effects for on-screen gamepads. Requiring apps to use Gamepad API to vibrate the device would bypass some of the limitations of Vibration API and would make it simpler for mobile games to support both "real" gamepads and on-screen gamepads through the same code path.

On some operating systems there's system-level support for on-screen gamepads, for instance on iOS GCVirtualController can be used to configure the layout of on-screen gamepad touch inputs. The operating system creates a virtual gamepad that applications can enumerate as if it were a real (hardware) gamepad. Android does not have any system-level support for on-screen gamepads but there are apps that provide similar functionality.

[nondebug]

I see two general approaches to supporting on-screen gamepads: allow sites to configure the system's built-in on-screen gamepad overlay, or create a virtual gamepad and feed it with inputs provided by the site.

Configuring the system's built-in on-screen gamepad would enable better parity with platform-specific apps but would complicate the implementation on OSes where there is no built-in on-screen gamepad or the OS does not support the same level of configuration.

The virtual gamepad approach gives sites more flexibility in arranging the on-screen controls and more consistency across platforms and browsers. It would also potentially enable other use cases like polyfill support for hardware gamepads that are currently unsupported by the browser.

With either approach, the site would gain the ability to vibrate the device by playing a vibration effect on the Gamepad object representing the on-screen gamepad. Currently, Gamepad API uses the gamepad user gesture as a consent signal and does not expose any gamepad information or capabilities until the user has interacted with a gamepad. This would be insufficient in the virtual gamepad approach since the site would have the ability to simulate a user gesture without user interaction. For user consent we would likely require some additional user prompt before connecting the virtual gamepad. Even if we are configuring an overlay we would still need user consent before enabling the overlay since it's drawn on top of the page content.

[marcoscaceres]

For consent, I wonder if something like .requestFullscreen(options) could work?