TAG Feedback: Open question for WebXR in OffscreenCanvas

[travisleithead]

Following up on a few extra comments in TAG review w3ctag/design-reviews#185

When running in "window", the display's framerate is known (though this is not perfect, as "window" might be displaying on any one of N monitors with different framerates, but let's gloss over that...)

When WebXR is being generated in a Worker, there's no great way to know what framerate to attach the worker to for frame generation. We think using rAF is the right approach, but how is the rate communicated/changed/statically or dynamically bound to the Worker whose primary purpose is XR-related?

(Loosly) related issue: #225

[toji]

This was discussed briefly at a WebXR call several weeks back and I've neglected to update this issue till just now.

We see this as unambiguous, because WebXR's design dictates that there will always be an output surface for each session that is either explicitly designated for display in a headset or explicitly linked to a canvas element via an outputContext. The number of possible scenarios is easy to enumerate:

1. Exclusive session, no outputContext (Displayed in headset, no mirror): XRSession.requestAnimationFrame runs at the refresh rate of the headset
2. Exclusive session with outputContext (Displayed in headset with in-page mirror):
   XRSession.requestAnimationFrame runs at the refresh rate of the headset, mirror canvas element is updated automatically as the UA sees fit, but probably either at the headset framerate for simplicity of at the output canvas' window framerate for efficiency. (Could also skip every other mirrored frame or other similar behavior if needed.)
3. Non-exclusive session, which must have an outputContext (Magic Window):
   XRSession.requestAnimationFrame runs at the refresh rate of the 'window' the outputContext's canvas it's attached to. Note that this is not the WebGL canvas, and as such whether we are using a OffscreenCanvas or not is not relevant. Because of user privacy concerns the session won't be allowed to process frames if the canvas is not part of a visible document tree, so there will never be a question about which window should be used. If WebXR is running in a worker we'll either disallow Magic Window style content or require the outputContext be transferred to the worker from the DOM.

Note that these scenarios only hold true for WebXR content. We do not intend to produce a general worker rAF mechanism as part of this spec.