Windows 11 modern keyboard: support modern input method editor interface

[josephsl]

Hi,

This issue can be taken as either a feature request or a bug report as it is sort of a border case:

Background:

As part of #11562 and parts of #10093, NVDA introduced support for modern IME (input methd editor) interface in Windows 10. Windows 11 tweaks the IME interface with updated look and UIA events set. As a result, NVDA does not support Windows 11 IME interface properly unless add-ons are installed.

Is your feature request related to a problem? Please describe.

NVDA does not natively support Windows 11 IME. It does support the new interface if add-ons such as Windows App Essentials are installed. Specifically:

• UIA tree hierarchy is different between Windows 10 and 11 interfaces.
• A focus event is raised whenever candidates are focused.

These changes affect not only IME interface, but also hardware keyboard input suggestions.

Describe the solution you'd like

Let NVDA natively support Windows 11 IME interface. In other words, bring parts of Windows App Essentials and similar add-ons to NVDA itself.

Describe alternatives you've considered

Leave it as is.

Additional context

IME mechanism is employed in hardware keyboard input suggestions, and for whoever is writing a pull request, do not break support for Windows 10.

Thanks.

[josephsl]

Hi,

March 2024 update: hardware keyboard input suggestion announcement is #16283 as UIA Automation Id's are different between UI elements. A pull request to address this issue is in the planning and experiemntation phases a.k.a. tested in Windows App Essentials.

Thanks.

[josephsl]

Hi,

Another March 2024 update: as a starting point, announcement of visible IME candidate items is part fo Windows App Essentials 24.04, and a pull request to let NVDA handle focus entered event in Windows 11 IME window iwll be submitted later in northern spring.

Thanks.