Local Presentation API to enable child-window presentation documents

[mfoltzgoogle]

Currently the Presentation API can only present fully fledged documents which are loaded in a separate (and isolated) browsing context. There may be use cases where it is not possible or desirable to render the intended content in a separate UA or a separate context.

1. The controlling UA has access to resources that can't easily be accessed elsewhere (because of login, VPN, etc.)
2. The controlling UA has some hardware capabilties (like a powerful GPU) required to render the content
3. The controlling UA has a known set of fonts or other assets that might not be available elsewhere
4. Messaging between the controller and presentation is too slow or awkward for the application (also thinking of gaming)

There may be other use cases as well.

In my thinking there could be two avenues to explore:

• A spec change to require 1-UA mode for rendering the presentation (helps with 1, 2, 3)
• A new API to present an offscreen frame or canvas that is part of the controlling document in 1-UA mode (helps with all of the above)

[tidoust]

For reference, see related discussion at TPAC

[mfoltzgoogle]

Updates

There is a third requirement which is related to the above. We've found that some controlling pages have authentication state in its cookie jar that can't be shared with the presentation (for security reasons, or because it owned by other origins). This makes it difficult for the presentation to obtain the necessary resources.

• The first approach (forced 1-UA) could also allow sharing of part of the cookie jar and other local storage for certain origins.
• The second approach (offscreen frame) would also address this requirement since the presentation is part of the same document.

In either scenario, reconnection to the presentation doesn't make sense or seems problematic from a security point of view.

Note that Media Capture from Element, along with WebRTC, allows a limited form of the second approach.

I'll be able to share some feedback on this from developers and possibly concrete proposals at TPAC.

[mfoltzgoogle]

This is an important item from our perspective, as it addresses specific challenges we have had with adoption of the Presentation API.

[mfoltzgoogle]

Wrote up a brief explainer for F2F discussion.

https://mfoltzgoogle.github.io/remote-window-api/explainer.md

[mfoltzgoogle]

From https://www.w3.org/2017/11/06-webscreens-minutes.html#x18:

ACTION: @mfoltzgoogle to refine the Remote Window API proposal and see if people prefer this over current 1-UA mode, evaluate similar Android API

ACTION: @anssiko to add this v2 feature provisionally to the Charter 2018

[anssiko]

ACTION: @anssiko to add this v2 feature provisionally to the Charter 2018

Since the draft Charter 2018 links to examples of features that may be integrated [into Presentation API Level 2] -- including this issue -- we agreed at the closing Revisit Charter 2018 F2F session that we do not need to explicitly enumerate v2 features in the charter.

[anssiko]

The plan is to recharter CG to incubate https://mfoltzgoogle.github.io/remote-window-api/explainer.md further.

[anssiko]

ACTION: @anssiko to amend the CG charter accordingly.

[mfoltzgoogle]

The updated link for the explainer is here (I renamed the repository to better reflect the proposed API).

https://github.com/mfoltzgoogle/local-presentation-api/blob/gh-pages/explainer.md

[anssiko]

As a reminder, the CfC to incubate the "local presentation mode" in the Second Screen CG is open until 20 June 2018. See the CfC sent to the public-webscreens on 21 May 2018.

To support this CG charter amendment, simply add your 👍to webscreens/cg-charter#18

[anssiko]

@mfoltzgoogle volunteered to update the status in this issue briefly here.

Per our resolution, I started a CfC to drop "Local Presentation Mode" from the CG Charter scope, see webscreens/cg-charter#26

[mfoltzgoogle]

Summary of developments since this issue was first filed:

The primary use case that was in mind when the explainer was written was to allow presentation applications to make use of multiple displays. These applications wish to present slides or other content on the secondary display and controls and speaker notes on the primary display. The Presentation API doesn't allow the two documents to share state which was a blocker.

If the secondary display is a wired attached display, then the Multi-Screen Window Placement API handles this use case by giving the application the ability to place a new window on that secondary display. The Fullscreen Companion Window makes the flow even simpler than what was proposed here (not requiring additional browser dialogs).

If the secondary display is a wireless display, the application can open the presentation content in a new tab, control it by cross-window messaging, and then invoke the Tab Self-Mirroring API to project it to the secondary display.

As the core use cases are being handled by ongoing work in the community and working groups, we don't plan on pursuing this feature any further. I plan on closing this out once the CG charter is updated to reflect that it's no longer a work item for the group.