The Web in WebXR (with COOP / COEP) + Declarative Interaction

[KooIaIa]

The Web in AR / VR

HTML became the foundation for the WorldWideWeb in 1990. You're reading this right now on the Web in HTML. The goal of this proposal is to see if we can agree that the Web can be loaded security and safely in WebXR so discussion can move towards clear next steps and implementations.

Previous Proposals

Layering HTML with composition into 3D web content and simple detached 3D CSS are the current proposed framings for loading HTML in XR. I'm not proposing to change that - I only want to address any security concerns around them. Discussion on them can be found here and work here. The key points I want to reiterate are that same-origin content and declarative interaction like ray-casts and touch-events can make the Web in WebXR secure - possibly even more secure than today's model of WebXR.

Security

Security is the # 1 reason I am aware of for HTML in WebXR not being possible.

My proposal is to follow the same security model of Shared Array Buffers with COOP / COEP. If html content can't be trusted to access shared memory, then it can't load in WebXR. This is a simple starting point like the Web 1.0 days before iFrames and Advertising.

Key Question: Is COOP + COEP and declarative interaction paradigms enough to ensure WebXR HTML content is same-origin and secure to load in WebXR?

Accessibility

HTML is the best language today for creating content that can be made accessible and read by anyone. VR and WebXR faces massive accessibility challenges and today's 3D engines have struggled to address these challenges. If HTML is able to be used as a format for spatial information then it could open VR up to decades of thoughtful standards and a11y innovation. No data model today comes close to HTML, and currently, it's just limited to small 2D screens. If HTML could be used in AR / VR freely - it could be loaded boundlessly across our entire world. Try to imagine how HTML would look describing your current surroundings and how revolutionary it would be for creating accessible real-life and virtual environments.

Spatial Computing
Looking forward 10 years from now, as XR devices become more powerful, spatial computing could allow the Web to be integrated alongside any native application. For a visual example, see this demonstration of how WebXR looks when loaded in today's software. HTML / the Web could help not just make information in WebXR accessible, but also the entire ecosystem of VR / AR software.

Improved Security

Declarative interactions with HTML can improve the ability for web authors to create simple secure content without the need to continuously track your hands or body for interactivity. If the right human-friendly declarative systems are offered out of the box then developers will be more inclined to use them than writing their own. This is an area of VR software design we want to see continuously advancing, improving, and changing as new ergonomic solutions for AR / VR interaction are found.

Why now?

VR / AR displays are now finally becoming capable of displaying text comfortably this year and next thanks to new technology like Pancake lenses. Previously, VR browser makers needed to display HTML on giant specialized virtual panels to make them legible. Going forward, the ability for readable text in AR / VR is only going to get better. In the context of Spatial Computing, HTML is a major technology for software development - the most popular IDEs today are created in HTML. HTML could help advance the state of VR / AR devices to become full-fledged computers and web authoring tools.

No?

Could we work together to create a single issue that can track all the challenges that need to be solved? No matter how complex it gets, the Web is important to all of us - we all want to be able to use it in VR / AR.

[KooIaIa]

/agenda

[KooIaIa]

Thank you everyone for the fun insightful meeting. I'm glad this proposal makes sense as a good security option. Looking forward to more future thoughts and discussion about the Web in the Web and WebXR!