Virtual Keyboard API - boundaries of docked overlay keyboard

[kenchris]

Hello TAG!

I'm requesting a TAG review of VirtualKeyboard API .

The Virtual Keyboard (VK) is the on-screen keyboard used for input in scenarios where a hardware keyboard may not be available. Unlike a hardware keyboard, a VK can adapt its shape to optimize for the expected type of input. Authors have control over the displayed shape of the VK through the inputmode attribute, but have limited control over when the VK is shown or hidden.
We propose a new VirtualKeyboard interface that has APIs to provide authors with more control over when the VK is shown or hidden and also exposes the size of the VK so the page can reflow its content if part of it is occluded by the VK.

• Explainer (minimally containing user needs and example code): https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/VirtualKeyboardAPI/explainer.md
• Security and Privacy self-review²: https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/VirtualKeyboardPolicy/security-privacy.md
• GitHub repo (if you prefer feedback filed there): https://github.com/MicrosoftEdge/MSEdgeExplainers/issues
• Primary contacts (and their relationship to the specification):
  • Anupam Snigdha (@snianu), Microsoft
  • Bo Cupp(@BoCupp-Microsoft), Microsoft
  • Grisha Lyukshin(@gked), Microsoft
• Organization/project driving the design: Microsoft
• External status/issue trackers for this feature (publicly visible, e.g. Chrome Status):
  https://www.chromestatus.com/feature/5680057076940800

Further details:

• [X ] I have reviewed the TAG's API Design Principles
• The group where the incubation/design work on this is being done (or is intended to be done in the future): Editing Task Force
• The group where standardization of this work is intended to be done ("unknown" if not known):
  Editing Task Force
• Existing major pieces of multi-stakeholder review or discussion of this design: inputPanelPolicy as a way to control software keyboard w3c/editing#225
• Major unresolved issues with or opposition to this design: N/A
• This work is being funded by: Microsoft

We'd prefer the TAG provide feedback as (please delete all but the desired option):

🐛 open issues in our GitHub repo for each point of feedback

[kenchris]

I am positive over the new overlaysContent + inset idea (I have suggested similar myself before).

As using an inset doesn't require JavaScript, I am wondering whether there will be another way to set this in addition to a JavaScript only API?

"overlaygeometrychange" sounds quite generic, and as there can be other overlays (other application windows, modal dialogs, file pickers etc) that might be a bit confusing. Are the values flipped (or should they be) in a right-to-left layout?

The boundingRect is currently on the event itself, but it might make sense to additionally also have it on the virtualKeyboard object like the generic sensors do. Or maybe only have it there (@anssiko)

[BoCupp-Microsoft]

Are the values flipped (or should they be) in a right-to-left layout?

No they are not flipped. They are in client coordinates. I don't think they should be flipped as the location of the keyboard is not writing mode dependent.

[snianu]

As using an inset doesn't require JavaScript, I am wondering whether there will be another way to set this in addition to a JavaScript only API?

Do you mean the virtualKeyboard geometry? You do have to register for VK updates so there has to be JS involved anyways, but if there is a way to do this without JS, then I would love to hear it.

"overlaygeometrychange" sounds quite generic, and as there can be other overlays

We are open to suggestions on naming this event.

The boundingRect is currently on the event itself, but it might make sense to additionally also have it on the virtualKeyboard object like the generic sensors

This sounds good to me. I'll open an issue to make this change.