What is Trusted Immersive UI?

[NellWaliczek]

[Disclaimer: This issue is one of several being filed to capture discussions that began either on #638, on #689, or at the most recent F2F]

The concept of “Trusted UI” is what allows User Agents to display a UI to end users on which sensitive information can be displayed and interacted with such that a website cannot snoop on it and cannot spoof it. Some features which use Trusted UI are user consent prompts, URL bars, navigation controls, favorite/bookmarks, and many more.

In 2D browsers, Trusted UI is presented either exclusively around the outside of a web page’s visual container or overlapping with it partially. In the context of an immersive experience, the definition of a “Trusted Immersive UI” is a bit more complex due to the fact there is no “outside” of immersive content; all pixels the user sees are rendered by the immersive content.

Various User Agents are exploring variations of Trusted Immersive UI, and as such WebXR should probably have a stance on the minimum requirements of these UIs to ensure they can meet an agreed upon security bar. For example, one already agreed upon aspect of WebXR is that there must always be a hardware button or dedicated gesture that users can rely on to bring them out of an immersive browsing session.

This GitHub issue is to track the discussion of what this bar should be. These are some of the ideas I recall folks mentioning:

• On devices with motion controllers with buttons, a dedicated button press/hold could be used to pull up the Trusted UI. When a trusted UI popup is needed, interacting with the same dedicated hardware button before displaying the content could be required. (e.g. “This browser needs your attention. Please press the Application button.”). Alternatively, it could go straight into presenting the request but require the dedicated hardware button to make a selection (e.g. “This site is requesting your permission to blahdiblah. Press the Application button to see your choices”)
• Displaying a sigil/totem that the User Agent knows about but the website does not. This approach would probably require disabling MR video and photo capture while the sigil is displayed.
• Regardless of how the UI is presented, frame data will need to be modified in some way (significantly throttled, quantized, rounded, etc) when Trusted Immersive UI is displayed to ensure the site cannot sniff password or URL entry.

I’m sure I’ve forgotten some and there are likely others not mentioned yet. So please please chime in with any information you can share about the approach your products are taking, and from there we’ll work on distilling the the information and establishing a consistent baseline.

[bricetebbs]

For reference there is a little more info on this in this comment on #424

[joshmarinacci]

This seems like a reasonable definition of a Trusted UI. I think it can be up to the UA to decide exactly what qualifies as a Trusted UI (since this will vary from platform to platform), as long as the concept of a Trusted UI is there.

[NellWaliczek]

There's a lot packed into that one comment, so let me see if I can tease it apart to make sure we're focusing on one issue at a time...

John enumerates ways a UA might provide a Trusted UI, which is inclusive of, but not limited to Trusted Immersive UI. My goal with this issue is to define common characteristics of Trusted Immersive UI. I think I've divided his original comment correctly, but please double check me.

Trusted Immersive UI

• On an HMD with a handheld input device, a user agent might reserve certain inputs (e.g. a home button) as an affordance for the user to confirm that a user interface can be trusted;
• On a standalone HMD without a handheld input device, a user agent might require that the immersive session be suspended in order to present a trusted interface;
• On a desktop or mobile device without an HMD, similar to HTML5 fullscreen the user agent may have no alternative but to suspend/resume the session in order to present a trusted interface (one analysis for HTML5 fullscreen is here);

Trusted UI

• On a tethered HMD, a user agent might preserve the session, but may or may not require the user to remove the headset in order to present a trusted interface on the desktop;
• On other form factors such as CAVE systems there may be additional input constraints or requirements for the persistence of the session that limit the ability for the presentation of a trusted interface (or the ability to gain consent) in other ways. In the extreme, such systems may not physically be capable of supporting bespoke feature requests.

His comments go on to discuss the differences in flows should a Trusted Immersive UI not be present, but that's a topic for #719.

[NellWaliczek]

Just to make sure we're all on the same page, this issue is NOT intended to result in prescriptive or normative text describing what Trusted Immersive UI should look like. I'm trying to capture what the principles of a trusted immersive UI are.

For example, while it wouldn't be considered a Trusted Immersive UI, in the spec we already call out that there needs to be a dedicated mechanism to exit an immersive experience that cannot be overridden by the page. That's the principle, but then it's left up to the UA/hardware to define the exact interaction.

I filed this issue as a location for us to identify and discuss similar statements, but about Trusted Immersive UI. For example (and I'm sure this will need iteration) "Trusted Immersive UI must not rely on a shared secret that can be observed by a mixed reality capture".

[avadacatavra]

A first pass of properties of a trusted immersive UI:

• non-spoofable
• indicates where the request/content displayed originates from
• doesn't rely on a shared secret that can be observed by a mixed reality capture (e.g. a gesture that can be seen by the camera)
• consistent between immersive experiences in the same UA

I'm sure we'll think of more/refinements but wanted to get a starting point

[johnpallett]

Some suggested additional properties:

• Easy to intentionally grant consent (e.g. the UI should be easily discovered)
• Hard to unintentionally grant user consent (e.g. the UI should prevent clickjacking)
• Clear methods for the user to revoke consent, and (if necessary) verify the current state of consent
• Avoid spamming/overloading the user with prompts

[johnpallett]

For context: What might be a useful concept from the Permissions API:

"This is intentionally vague about the details of the permission UI and how the UA infers user intent. UAs should be able to explore lots of UI within this framework."

[avadacatavra]

@johnpallett great additions!

I think that being clear about guidelines/properties and leaving it up to UAs to implement appropriately is the best way to go

[klausw]

@johnpallett wrote:

• Hard to unintentionally grant user consent (e.g. the UI should prevent clickjacking)

Side note, timing-based clickjacking is potentially more open to abuse if the input device uses an analog trigger that transitions to a "clicked" state at a certain threshold. Apps would be able to read the current trigger axis value through the gamepad API and could try to use that to pop up a trusted UI prompt a fraction of a second before the trigger reaches the threshold.

(Apparently there have been clickjacking exploits for 2D permission APIs based on asking users to double-click a location, where the second click goes to a permission dialog.)

[Manishearth]

will be fixed by #875