Is XRDevice still necessary?

[toji]

It was suggested at the Seattle F2F that XRDevice is possibly not a useful interface any longer, and we should evaluate whether or not it should stay in the spec. (Thanks, Nell. 😉) It's a valid question, albeit one that I'm somewhat opinionated about, so I'm opening this issue to lake a look at the reasons why or why not we may want to keep the interface. Whatever we decide, we should strive to resolve this quickly as changes to the fundamental API structure like this really ought to be drawing to a close if we intend to wrap up the core API any time soon.

To recap the argument for dropping the interface: We previously made the decision to only return a single XRDevice even when multiple physical devices may be connected to the system. This means that we could probably make the device implicit and still achieve the same thing with less API surface in some cases. Less API surface being a generally accepted Good Thing:tm: it's easy to see why the idea is attractive.

Presumably if we dropped the XRDevice interface, the WebXR initialization code path would look something like:

navigator.xr.supportsSession({ immersive: true }).then(() => {
  // Show "Enter XR" button
}).catch((err) => {
  // Presumably you'd end up here if no device is available.
});

function onEnterXR() {
  navigator.xr.requestSession({ immersive: true }).then((session) => {
    let gl = webglCanvas.createContext('webgl', { compatibleXRSession: session });
    // Remainder of session setup as normal.
  });
}

It's my opinion, though, that the XRDevice interface still has value largely because of WebGL context compatibility:

• If WebGL context compatibility is pushed to a per-session basis then it becomes impossible to reliably preload WebGL context before entering an immersive session. You can create the context beforehand, but can't make it compatible because you have no session to do so with. Upon creating the session if you set the session compatibility afterwards you run the risk of losing the context, which may result in what appeared to be an otherwise loaded scene reloading all it's resources upon clicking "Enter XR".
• Similarly, session-based context compatibility prevents some magic window gallery uses, where multiple output canvases all show imagery from the same WebGL context. This is because there's no guarantee that setting context compatibility with second magic window session won't break compatibility with the first. (Logically it would be fine in most cases, but we'd have to twist ourselves into knots in the spec language to explain why and how.)

Furthermore, if we ever DO want to extend the API to allow for more nuance in how physical devices are enumerated we'll want a XRDevice-like concept regardless.

That's the state of things as I see it currently, would love to hear reasons from other contributors for or against keeping this particular level of abstraction.

[ddorwin]

Assorted thoughts on this topic:

• The main purpose of breaking out XRDevice and XRSession in WebXR was to avoid prompting or spinning up hardware before it was needed. However, we also have supportsSession() for this.
  • Do we really need an additional XRDevice abstraction?
  • Are there things other than session creation that we might want to add to XRDevice that wouldn't make (more) sense on XRSession.
• XRDevice does not actually (or consistently) represent a device - even a virtualized one.
• Really, the current requestSession() is requesting a device with specific properties.
  • On desktop, immersive and non-immersive sessions are literally different physical devices, possibly with different graphics adapters.
  • In the Unified Rendering Path discussion (Unified Render Paths: Poseless XRSessions #367), we're talking about whether XRDevice.requestSession(immersive:false) should return a session if there are no device sensors. However, I don't think current implementations even allow requestDevice() to succeed in this case, which is a bit weird since capabilities are currently checked at the session level.
• Creating an entirely new session to go in and out of immersive mode is a bit weird, especially when the page is already being viewed in headset.
  • Someone at the f2f suggested that what we currently think of as separate sessions is really just moving up/down the level of immersiveness.
  • When in headset, the application has a session on a display that has immersive capabilities, so it should just be able to request presentation on that session.
  • However, this doesn’t map well to desktop where a) the author may wish to continue rendering inline and b) there is actually a different device.
• Graphics context and/or graphics adapters
  • My understanding is that there are some oddities with how contexts may be lost that aren’t obvious from the current API.
  • Not pretending the sessions come from a single device may help.
• Multiple headsets: It’s hard to imagine how we would go from the current design to actually supporting multiple headsets with different XRDevices. If that’s not possible, calling anything an XRDevice is odd.
  • One option might be to add an immersiveDeviceIndex that would allow the app to request 1st (default), 2nd, 3rd, etc. device. We could wait until this is needed, but it is an option that doesn’t require a separate device abstraction.
• Changes to session creation (Add an explainer for session creation and configuration #376) may give us additional options.

One possible solution might be to really differentiate inline and immersive, possibly using different methods, possibly on different objects. It might make sense to tie the immersive session creation to the inline session to allow optimization, such as reusing the session in-headset and trying to use the same graphics adapter.

For example, something like this simplified pseudocode:

inlineSession = navigator.xr.requestInlineSession(mayRequestImmersive: true);
// mayRequestImmersive is a hint to the UA that it should pick the graphics adapter connected to the external headset if possible.

// LATER:
inlineSession.requestImmersiveSession();
// On a standalone device, the session internals could be reused.
// On desktop, a new session is created on the same adapter if possible.

We could replace the last call with requestImmersiveDevice() if we think there is value in a device abstraction, though we'd probably also need to do the same for the inline device/session.

One thing that is not clear in this scenario is how we’d ask about session support for immersive sessions. Is it possible that immersive sessions are supported or support things that inline sessions do not? Maybe this is addressed by allowing applications to query different combinations via the session creation and configuration mechanisms (#376).

[ddorwin]

The observations in the second bulleted list in #367 (comment) identify some other interesting issues related to XRSession vs. XRDevice.

• Detecting when actual XR hardware is [un]available. It's not clear what the use case is here, but we can probably handle this in requestSession()/XRSession properties.
• There is mention of checking for hardware when an XRDevice is requested. I'm not sure requestDevice() should really be doing that. It seems that requestSession() can and should handle this.
• WebGL context compatibility is based on the XRDevice. If XRDevice is removed, we'd need a new mechanism and/or to use a different object for this.
  • The session may or may not be appropriate. This may relate to whether the sessions are on the same adapter.
  • The comment mentions potential issues with requesting/handling different types of hardware at the session level. We should probably think this through for all scenarios (no sensors, inline, immersive on same display, immersive on different hardware and/or graphics adapter).

[blairmacintyre]

I'm curious what people's thoughts are on this now.

My biggest concern is how we deal with multiple devices / modes, especially where they might overlap in functionality, without some notion of device.

[toji]

I've posted a pull request for this issue: #405

[toji]

The answer is now definitively "No". Closing. ;)