WebView2 focus trap on UWP XBOX when using XY directional navigation

[LDS2020]

What happened?

While testing the WebView2 WinUI2 Sample on Xbox, along with an app I have in the MS Store, I find that I am unable to disengage from the WV2 control when using XY directional navigation. I am not sure if this is an implementation issue in the Sample or a bug in WebView2 for Xbox. I find it hard to believe this issue didn't get picked up in testing, so I am starting here.

• The existing UWP sample works fine on Xbox but it uses Mouse or Pointer mode, which allows the users to move from the Webview to other controls on the page. An Xbox app, however, would normally use XY navigation which allows full access to every button on the Gamepad and is the #1 best practice for Xbox UI.

• When you switch to XY navigation, the user gets stuck in a a focus trap on the WebView2 control and cannot disengage by pressing the B button which is how most controls (like the address bar in this sample or the original WebView1) work. The only way to exit is click the Xbox button, Quit the app and start again.

Importance
Blocking. Although it is theoretically possible to switch Xbox apps over to Mouse mode, it would require major changes for developers and a foreign user experience for the users.

Environment

Runtime Channel
Stable release (WebView2 Runtime)

Runtime Version
Pre-installed on Xbox

WebView2 Version
1.0.2210.55

Framework
WinUI2/UWP
Microsoft.UI.Xaml 2.8.6

Operating System
Xbox

OS Version
10.0.25398.2921

Repro steps

• See this modified repo of the WebView2 WinUI2 sample app, where I have:

• Switched to XY Navigation Mode by changing PointerMode to WhenRequested in App.xaml.cs

• Turned on Reveal Focus and written debug lines, so you can see which control has focus

• Disabled scaling to avoid the bug documented in WebView2 issue 4133 by @remokeitel

• Set IsFocusEngagementEnabled=true, RequiresPointer=true and set IsFocusEngaged=true for the WebView2 control

Expected behavior

The user should be able to navigate to the WebView2 on the page, select A to engage and interact with it using the temporary Mouse pointer, and select B to disengage so they can select another control, as described in this quote from here:

When the IsFocusEngagementEnabled property is set to true, it marks the control as requiring focus engagement. This means that the user must press the A/Select button to "engage" the control and interact with it. When they are finished, they can press the B/Back button to disengage the control and navigate away from it.

Instead, the user is stuck on the WebView control, as shown in this screenshot, and cannot get to any of the other controls on the page or use any of the buttons on the gamepad:

Related Issue
A separate but related issue is that WebView2 for WinUi2 has not yet implemented Key events like OnPreviewKeyDown, as has been done on other platforms, so it is not possible to intercept the B button inside the Webview2 to code a workaround.

AB#48450583

[jennifer-jacobs]

Hi @LDS2020,

I can repro this as well and sorry you are hitting this. Most of our testing was done in scenarios where the app was entirely a WebView and not as much mixed XAML and WebView content. This has been added to our backlog!

[jennifer-jacobs]

In order to better investigate this, do you have a sample WV1 app that shows this behavior working?

Do you own the content that is being shown? If so, as a workaround you could intercept the B message using a key listener on the page and send a message back to XAML and manually set focus back to the correct element.

[LDS2020]

@jennifer-jacobs, thanks for confirming that you can reproduce the error and I will follow this issue for future updates.

• to see how it works with WV1, check out my app CleanOut for Xbox or, if you need source code, let me know if you want me to create a version of the sample app using WV1?
• forgetting WV1 though, WV2 on Xbox should behave just like the TextBox AddressBar does on the Browser.xaml page. Press A to interact with it and B to escape, and you can tab over it without engaging if you want to.
• as I stated in my Related Issues comment above, you can't code a workaround to intercept the B button because the key listeners have not been implemented in the WinUI2 version of WV2, like they have on WPF. You can wire up the event handlers at either the WV2 control or Page level but they don't do anything when the WV2 is FocusEngaged. The WV2 control intercepts everything. Let me know if I am wrong about that?

[remokeitel]

Hi @LDS2020,

you can use (as a workaround) the Gamepad API to get pressed buttons:

webView2.CoreWebView2.WebMessageReceived += (coreWebView2, args) =>
{
    var key = Enum.Parse<VirtualKey>(args.TryGetWebMessageAsString());
    switch (key)
    {
        case VirtualKey.GamepadB:
            //...
            break;
        //...
    } 
};

Add the following JavaScript code to the WebView2 HTML:

let interval;
window.addEventListener(
    "gamepadconnected",
    (e) => {
        interval = setInterval(() => {
            for (const gamepad of navigator.getGamepads()) {
                if (gamepad != null) {
                    if (gamepad.buttons[0].pressed)
                        window.chrome.webview.postMessage("GamepadA");
                    else if (gamepad.buttons[1].pressed)
                        window.chrome.webview.postMessage("GamepadB");
                    else if (gamepad.buttons[2].pressed)
                        window.chrome.webview.postMessage("GamepadX");
                    else if (gamepad.buttons[3].pressed)
                        window.chrome.webview.postMessage("GamepadY");
                    else if (gamepad.buttons[4].pressed)
                        window.chrome.webview.postMessage("GamepadLeftShoulder");
                    else if (gamepad.buttons[5].pressed)
                        window.chrome.webview.postMessage("GamepadRightShoulder");
                    else if (gamepad.buttons[6].pressed)
                        window.chrome.webview.postMessage("GamepadLeftTrigger");
                    else if (gamepad.buttons[7].pressed)
                        window.chrome.webview.postMessage("GamepadRightTrigger");
                    else if (gamepad.buttons[8].pressed)
                        window.chrome.webview.postMessage("GamepadView");
                    else if (gamepad.buttons[9].pressed)
                        window.chrome.webview.postMessage("GamepadMenu");
                    else if (gamepad.buttons[10].pressed)
                        window.chrome.webview.postMessage("GamepadLeftThumbstickButton");
                    else if (gamepad.buttons[11].pressed)
                        window.chrome.webview.postMessage("GamepadRightThumbstickButton");
                    else if (gamepad.buttons[12].pressed)
                        window.chrome.webview.postMessage("GamepadDPadUp");
                    else if (gamepad.buttons[13].pressed)
                        window.chrome.webview.postMessage("GamepadDPadDown");
                    else if (gamepad.buttons[14].pressed)
                        window.chrome.webview.postMessage("GamepadDPadLeft");
                    else if (gamepad.buttons[15].pressed)
                        window.chrome.webview.postMessage("GamepadDPadRight");
                    //else if (gamepad.buttons[16].pressed)
                    //    window.chrome.webview.postMessage("GoHome");
                }
            }
        }, 100);
    },
    false
);
window.addEventListener(
    "gamepaddisconnected",
    (e) => {
        if (navigator.getGamepads()[0] == null &&
            navigator.getGamepads()[1] == null &&
            navigator.getGamepads()[2] == null &&
            navigator.getGamepads()[3] == null)
            clearInterval(interval);
    },
    false
);

But I hope in the near future we will get functional key event listeners for the WebView2 control!

[LDS2020]

Thanks @remokeitel! I will give that a try as a workaround.

@jennifer-jacobs I noticed Victor tagged this issue as Tracked, but not as Bug yet, even though you were able to repro the issue. Does this mean you have not determined if it is an implementation issue or a WV2 defect yet?

[jennifer-jacobs]

@LDS2020 We haven't determined the right fix for it yet (whether it's an implementation or WV2 bug). We've added it to our backlog to continue looking into it and I've marked it as a bug. Thanks for calling that out!

[LDS2020]

@jennifer-jacobs - what is the latest on this issue? It has been a year since I reported it, so please tell me honestly if you don't think it will ever be fixed.

I am still using Webview1 as I was not able to inject the JS for the gamepad like remokeitel suggested, and would really like to upgrade to Webview2.

I look forward to hearing from you.

[LDS2020]

@victorhuangwq Hi Victor - any thoughts on this?

[LDS2020]

@victorhuangwq - can you please let me know the status of this bug and what is the ETA for a fix? Or please be transparent and let me know if you are no longer making changes to WebView2 on Xbox, and I will give up on it.

I am surprised that MS thought it was important to make this announcement but did not test one of the most basic features, which is standard Xbox controller button commands:
https://blogs.windows.com/msedgedev/2023/11/01/webview2-for-xbox-announcement/#:~:text=Support%20for%20Microsoft%20Edge%20WebView2,in%20addition%20to%20playing%20games.