VirtualKeyboard API

[bramus]

Request for position on an emerging web specification

Information about the spec

• Spec Title: VirtualKeyboard API
• Spec URL: https://www.w3.org/TR/virtual-keyboard/
• GitHub repository: https://github.com/w3c/virtual-keyboard/
• Issue Tracker: https://github.com/w3c/virtual-keyboard/issues
• Explainer: https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/VirtualKeyboardAPI/explainer.md

Design reviews and vendor positions

• TAG Design Review: VirtualKeyboard API - show/hide policy w3ctag/design-reviews#498 (comment)
• Mozilla standards-positions issue: Virtual Keyboard Control mozilla/standards-positions#531

Bugs tracking this feature

• WebKit Bugzilla: https://bugs.webkit.org/show_bug.cgi?id=230225
• Radar: rdar://83154723

Anything else we need to know

Implemented in Chromium: https://web.dev/virtualkeyboard/

[othermaciej]

Seems odd that there's no Mozilla standards-positions issue on this.

[bramus]

Nothing official on the Standards Position website, but there is this issue that has no response yet: mozilla/standards-positions#531

[litherum]

I have concerns about navigator.virtualKeyboard.show() for 2 reasons:

1. User confusion. If the keyboard randomly appears, that would/could be very disconcerting for a user. Even if this was guarded by user gesture, there are plenty of use gestures which have nothing to do with text entry.
2. Currently when we show the keyboard, we have a reason for doing so, and a context that the keyboard is meant for. We can show different keyboards depending on which input element triggered the keyboard to appear. We can scroll the triggering element into view. If script has the ability to show the keyboard for any reason, we lose this context.

[tomayac]

FWIW, the inability to control how the virtual keyboard appears was brought up as a top pain point by Telegram in a technical post on WebKit issues:

2. Text Field.

Safari automatically scrolls the page after tapping a text field, in order to center it – completely ignoring the interface layout. The devs have no way to tune or adjust it.

[othermaciej]

If scroll-to-center behavior is bad for their app, there's probably a much simpler feature that could meet that specific need - this API seems too complex to be the obvious way to address this problem.

[snianu]

1. User confusion. If the keyboard randomly appears, that would/could be very disconcerting for a user. Even if this was guarded by user gesture, there are plenty of use gestures which have nothing to do with text entry.
2. Currently when we show the keyboard, we have a reason for doing so, and a context that the keyboard is meant for. We can show different keyboards depending on which input element triggered the keyboard to appear. We can scroll the triggering element into view. If script has the ability to show the keyboard for any reason, we lose this context.

VK would only appear if the focus is inside an editable element and there is a sticky user activation. This API lets the web author control the show/hide behavior of the VK only when the virtualKeyboardPolicy is in manual mode. Default behavior is auto. The discussion in the TAG review thread has more details about the behavior of this API. Based on feedback from @rniwa & @whsieh, I thought we had positive support from Apple?

[whsieh]

My understanding is that we will only allow web authors to programmatically show the keyboard using the Virtual Keyboard API in scenarios where authors are already capable of programmatically showing the keyboard by focusing an editable element.

(At one point, I recall giving feedback during TPAC that show() and possibly also hide() should return a promise, that rejects if the UA prefers to not show the keyboard, for the reasons Myles noted earlier).

[liamdebeasi]

This API is something that would be very valuable for developers using Ionic. Our use case is ensuring that inputs that scroll on non-body elements are not covered by the keyboard.

The default scroll-to-center behavior in Safari is not ideal because it causes other content to be shifted offscreen. We wrote a "scroll assist" utility to prevent the browser from shifting content off screen while ensuring that the input is not covered by the keyboard: https://github.com/ionic-team/ionic-framework/blob/main/core/src/utils/input-shims/hacks/scroll-assist.ts.

Below is an example of the default behavior as well as the custom Ionic behavior:

default Safari behavior	custom Ionic behavior
default-safari.mp4	custom-ionic.mp4

With the default behavior, observe that the header and part of the content are shifted offscreen. With the custom behavior, observe that neither the header nor the content is shifted offscreen.

The downside of this custom approach is we need to estimate the height of the keyboard. This can cause inputs to still be hidden by particularly tall keyboards. It can also cause inputs to be scrolled more than they need to be.

Being able to know a) the geometry of the keyboard and b) when the geometry of the keyboard changes would significantly improve the user experience of apps built with Ionic. We do not have a big need for programmatically showing/hiding the keyboard at the moment, but I do see how that functionality could be considered confusing by some.

edit: We do also make use of the Virtual Viewport API to get a better idea of the keyboard geometry. Our main problem with that is the event fires after the keyboard is fully open. As a result, we have to wait ~500ms before we can scroll the input into view. This gives the impression of a slow/unresponsive application.

[brunostasse]

If scroll-to-center behavior is bad for their app, there's probably a much simpler feature that could meet that specific need - this API seems too complex to be the obvious way to address this problem.

Apart from preventing the scroll-to-center and the layout viewport shrinking behaviours, which is useful in itself as shown by @liamdebeasi example, this API also provides information about the size of the virtual keyboard. It is provided both through the JS API, and through CSS environment variables (keyboard-inset-*).

Such information is useful to properly scroll into view in-flow inputs as suggested above, but also for element which are supposed to be fixed right above the VK. This is very common in chat interfaces, where the text input is usually at the bottom of the viewport by default, and moves up with the keyboard as it opens. It is also common for note taking apps, where you often have a toolbar positioned right above the keyboard. Without this information, there is currently no way to produce these kind of interfaces.

As pointed out above, it appears that in Chrome's implementation the ongeometrychange event might be fired too late (I haven't tested myself). It would indeed be much better for the event allowing to detect change in the geometry to be sent as the VK starts appearing, and not after it has fully appeared. This is what the specification seems to be recommending, so this might just be an implementation issue.

The event is dispatched when the intersection of the VK and the document's viewport changes, e.g., in response to the VK being shown or hidden or the browser window being repositioned.

Virtual Keyboard API specification

[jamie-pate]

This api would also be useful for webassembly projects that afaik currently have no supported way to control the virtual keyboard?

[HolgerJeromin]

I want to prevent my own keyboard when there is a native keyboard.
So having navigator.virtualKeyboard.boundingRect and the geometrychange event would be great for some use cases even without having show and hide.