forked from immersive-web/webxr
-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.bs
1179 lines (813 loc) · 69.7 KB
/
index.bs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<pre class="metadata">
Shortname: webxr
Title: WebXR Device API
Group: immersivewebwg
Status: w3c/ED
ED: https://immersive-web.github.io/webxr/
Repository: immersive-web/webxr
Level: 1
Mailing List Archives: https://lists.w3.org/Archives/Public/public-immersive-web/
!Participate: <a href="https://github.com/immersive-web/webxr/issues/new">File an issue</a> (<a href="https://github.com/immersive-web/webxr/issues">open issues</a>)
!Participate: <a href="https://lists.w3.org/Archives/Public/public-immersive-web/">Mailing list archive</a>
!Participate: <a href="irc://irc.w3.org:6665/">W3C's #immersive-web IRC</a>
Editor: Brandon Jones, Google http://google.com/, bajones@google.com
Editor: Nell Waliczek, Amazon [Microsoft until 2018] https://amazon.com/, nhw@amazon.com
Abstract: This specification describes support for accessing virtual reality (VR) and augmented reality (AR) devices, including sensors and head-mounted displays, on the Web.
Ignored Vars: layer
Warning: custom
Custom Warning Title: Unstable API
Custom Warning Text:
<b>The version of the WebXR Device API represented in this document is incomplete and may change at any time.</b>
<p>While this specification is under development some concepts may be represented better by the <a href="https://github.com/w3c/webvr/blob/master/explainer.md">WebXR Device API Explainer</a>.</p>
</pre>
<pre class="anchors">
urlPrefix: http://www.w3.org/TR/hr-time/
type: typedef; text: DOMHighResTimeStamp
type: dfn; text: timestamp origin
urlPrefix: https://html.spec.whatwg.org/multipage/system-state.html
type: interface; text: Navigator
urlPrefix: https://wiki.whatwg.org/wiki/OffscreenCanvas
type: typedef; text: OffscreenCanvas
type: dfn; text: offscreen canvas
urlPrefix: https://www.w3.org/TR/html51/webappapis.html
type: dfn; text: window.requestAnimationFrame
type: typedef; text: EventHandler; url:#typedefdef-eventhandler
urlPrefix: https://www.w3.org/TR/html5/semantics-scripting.html
type: interface; text: HTMLCanvasElement
urlPrefix: https://www.w3.org/TR/html5/
type: interface; text: Document
urlPrefix: https://www.khronos.org/registry/webgl/specs/latest/1.0/
type: typedef; text: uniformMatrix4fv
type: interface; text: WebGLFramebuffer
type: interface; text: WebGLRenderingContext
type: interface; text: WebGLRenderingContextBase
type: dictionary; text: WebGLContextAttributes
type: dfn; text: Create the WebGL context; url:#2.1
type: dfn; text: WebGL viewport; url:#5.14.4
type: dfn; text: WebGL context lost flag; url:#webgl-context-lost-flag
type: dfn; text: handle the context loss; url:#CONTEXT_LOST
type: dfn; text: Restore the context; url: #restore-the-drawing-buffer
urlPrefix: https://www.khronos.org/registry/webgl/specs/latest/2.0/
type: interface; text: WebGL2RenderingContext
urlPrefix: https://drafts.fxtf.org/geometry/
type: interface; text: DOMMatrix
type: interface; text: DOMPointReadOnly
urlPrefix: https://w3c.github.io/orientation-sensor/; spec: ORIENTATION-SENSOR
type: interface; text: AbsoluteOrientationSensor
type: interface; text: RelativeOrientationSensor
spec: ECMAScript; urlPrefix: https://tc39.github.io/ecma262/#
type: interface
text: Promise; url:sec-promise-objects
spec: WebIDL; urlPrefix: https://www.w3.org/TR/WebIDL-1/#
type: dfn
text: invoke the Web IDL callback function; url:es-invoking-callback-functions
spec: Feature Policy; urlPrefix: https://wicg.github.io/feature-policy/#
type: dfn
text: default allowlist
text: feature
text: feature name
</pre>
<style>
/* Re-add this once there are actually stable sections of the spec */
/*.unstable::before {
content: "This section is not stable.";
float: right;
color: red;
}*/
.unstable {
background-image: url("data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' width='300' height='290'><text transform='rotate(-45)' text-anchor='middle' font-family='sans-serif' font-weight='bold' font-size='70' y='210' opacity='.1'>Unstable</text></svg>");
background-repeat: repeat
}
.unstable.example:not(.no-marker)::before {
content: "Example " counter(example) " (Unstable)";
float: none;
}
</style>
<section class="unstable">
Introduction {#intro}
=============
<p class=non-normative>
Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications requires high-precision, low-latency interfaces to deliver an acceptable experience. Other interfaces, such as the {{RelativeOrientationSensor}} and {{AbsoluteOrientationSensor}}, can be repurposed to surface input from these devices to polyfill the WebXR Device API. The WebXR Device API provides purpose-built interfaces to VR/AR hardware to allow developers to build compelling, comfortable immersive experiences.
</p>
Terminology {#terminology}
===========
This document uses the acronym <b>XR</b> throughout to refer to the spectrum of hardware, applications, and techniques used for Virtual Reality, Augmented Reality, and other related technologies. Examples include, but are not limited to:
* Head mounted displays, whether they are opaque, transparent, or utilize video passthrough
* Mobile devices with positional tracking
* Fixed displays with head tracking capabilities
The important commonality between them being that they offer some degree of spatial tracking with which to simulate a view of virtual content.
Terms like "XR Device", "XR Application", etc. are generally understood to apply to any of the above. Portions of this document that only apply to a subset of these devices will indicate so as appropriate.
The terms [=3DoF=] and [=6DoF=] are used throughout this document to describe the tracking capabilities of XR devices.
- A <dfn>3DoF</dfn> device, short for "Three Degrees of Freedom", is one that can only track rotational movement. This is common in devices which rely exclusively on accelerometer and gyroscope readings to provide tracking. [=3DoF=] devices do not respond translational movements from the user, though they may employ algorithms to estimate translational changes based on modeling of the neck or arms.
- A <dfn>6DoF</dfn> device, short for "Six Degrees of Freedom", is one that can track both rotation and translation, enabling for precise 1:1 tracking in space. This typically requires some level of understanding of the user's environment. That environmental understanding may be achived via inside-out tracking, where sensors on the tracked device itself (such as cameras or depth sensors) are used to determine the device's position, or outside-in tracking, where external devices placed in the user's environment (like a camera or light emmiting device) provides a stable point of reference against which the XR device can determine it's position.
Security, Privacy, and Comfort Considerations {#security}
=============================================
The WebXR Device API provides powerful new features which bring with them several unique privacy, security, and comfort risks that user agents must take steps to mitigate.
Gaze Tracking {#gazetracking-security}
-------------
While the API does not yet expose eye tracking capabilites a lot can be inferred about where the user is looking by tracking the orientation of their head. This is especially true of XR devices that have limited input capabilities, such as Google Cardboard, which frequently require users to control a "gaze cursor" with their head orientation. This means that it may be possible for a malicious page to infer what a user is typing on a virtual keyboard or how they are interacting with a virtual UI based solely on monitoring their head movements. For example: if not prevented from doing so a page could estimate what URL a user is entering into the user agent's URL bar.
To prevent this risk the user agent MUST [=blur all sessions=] when the users is interacting with sensitive, trusted UI such as URL bars or system dialogs. Additionally, to prevent a malicious page from being able to monitor input on a other pages the user agent MUST [=blur all sessions=] on non-focused pages.
Trusted Environment {#trustedenvironment-security}
-------------------
If the virtual environment does not consistently track the user's head motion with low latency and at a high frame rate the user may become disoriented or physically ill. Since it is impossible to force pages to produce consistently performant and correct content the user agent MUST provide a tracked, trusted environment and an [=XR Compositor=] which runs asynchronously from page content. The compositor is responsible for compositing the trusted and untrusted content. If content is not performant, does not submit frames, or terminates unexpectedly the user agent should be able to continue presenting a responsive, trusted UI.
Additionally, page content has the ability to make users uncomfortable in ways not related to performance. Badly applied tracking, strobing colors, and content intended to offend, frighten, or intimidate are examples of content which may cause the user to want to quickly exit the XR experience. Removing the XR device in these cases may not always be a fast or practical option. To accomodate this the user agent SHOULD provide users with an action, such as pressing a reserved hardware button or performing a gesture, that escapes out of WebXR content and displays the user agent's trusted UI.
When navigating between pages in XR the user agent should display trusted UI elements informing the user of the security information of the site they are navigating to which is normally presented by the 2D UI, such as the URL and encryption status.
Context Isolation {#contextisolation-security}
-----------------
The trusted UI must be drawn by an independent rendering context whose state is isolated from any rendering contexts used by the page. (For example, any WebGL rendering contexts.) This is to prevent the page from corrupting the state of the trusted UI's context, which may prevent it from properly rendering a tracked environment. It also prevents the possibility of the page being able to capture imagery from the trusted UI, which could lead to private information being leaked.
Also, to prevent CORS-related vulnerabilities each page will see a new instance of objects returned by the API, such as {{XRSession}}. Attributes such as the {{XRWebGLLayer/context}} set by one page must not be able to be read by another. Similarly, methods invoked on the API MUST NOT cause an observable state change on other pages. For example: No method will be exposed that enables a system-level orientation reset, as this could be called repeatedly by a malicious page to prevent other pages from tracking properly. The user agent MUST, however, respect system-level orientation resets triggered by a user gesture or system menu.
Fingerprinting {#fingerprinting-security}
--------------
Given that the API describes hardware available to the user and its capabilities it will inevitably provide additional surface area for fingerprinting. While it's impossible to completely avoid this, steps can be taken to mitigate the issue. This spec limits reporting of available hardware to only a single device at a time, which prevents using the rare cases of multiple headsets being connected as a fingerprinting signal. Also, the devices that are reported have no string identifiers and expose very little information about the devices capabilities until an XRSession is created, which may only be triggered via user activation in the most sensitive case.
Issue: Discuss use of sensor activity as a possible fingerprinting vector.
Initialization {#initialization}
===================
XR {#xr-interface}
----
<pre class="idl">
[SecureContext, Exposed=Window] interface XR : EventTarget {
// Methods
Promise<void> supportsSession(optional XRSessionCreationOptions options);
Promise<XRSession> requestSession(optional XRSessionCreationOptions parameters);
// Events
attribute EventHandler ondevicechange;
};
[SecureContext]
partial interface Navigator {
[SameObject] readonly attribute XR xr;
};
</pre>
The <dfn attribute for="Navigator">xr</dfn> object is the entry point to the API, used to query for XR features available to the user agent. It has a <dfn>list of XR devices</dfn>, which MUST be initially empty, and an <dfn>XR device</dfn> which MUST be initially <code>null</code> and represents the active device from the [=list of XR devices=] that API calls will interact with.
The user agent MUST be able to <dfn>enumerate XR devices</dfn> attached to the system, at which time each available device is placed in the [=list of XR devices=]. Subsequent algorithms requesting enumeration MAY reuse the cached [=list of XR devices=]. Enumerating the devices [=should not initialize device tracking=]. After the first enumeration the user agent SHOULD begin monitoring device connection and disconnection, adding connected devices to the [=list of XR devices=] and removing disconnected devices.
<div class="algorithm" data-algorithm="xr-device-selection">
Each time the [=list of XR devices=] changes the user agent should <dfn>select an XR device</dfn> by running the following steps:
1. Let |oldDevice| be the current [=XR device=].
1. If the [=list of XR devices=] is empty, set the [=XR device=] to <code>null</code>.
1. If the [=list of XR devices=] contains one device set the [=XR device=] to that device.
1. If there are any active {{XRSession}}s and |oldDevice| is in the [=list of XR devices=], set the [=XR device=] to |oldDevice|.
1. Else set the [=XR device=] to a device of the user agent's choosing.
1. If this is the first time devices have been enumerated or |oldDevice| equals [=XR device=], abort these steps.
1. [=Shut down the session|Shut down=] any active {{XRSession}}s.
1. Set the [=XR compatible=] boolean of all {{WebGLRenderingContextBase}} instances to <code>false</code>.
1. Queue a task that fires a simple event named {{devicechange}} on the {{XR}} object.
</div>
NOTE: The user agent is allowed to use any criteria it wishes to [=select an XR device=] when the [=list of XR devices=] contains multiple devices. For example, the user agent may always select the first item in the list, or provide settings UI that allows users to manage device priority. Ideally the algorithm used to select the default device is stable and will result in the same device being selected across multiple browsing sessions.
<div class="algorithm" data-algorithm="ensure-device-selected">
Any time an [=XR device=] is needed by an algorithm it can <dfn>ensure an XR device is selected</dfn> by running the following steps:
1. If [=XR device=] is not <code>null</code>, abort these steps.
1. [=Enumerate XR devices=].
1. [=Select an XR device=].
</div>
The <dfn attribute for="XR">ondevicechange</dfn> attribute is an [=Event handler IDL attribute=] for the {{devicechange}} event type.
Each [=XR device=] has a <dfn>supports immersive</dfn> value, which is a boolean which MUST be set to <code>true</code> if the device can support [=immersive sessions=] and <code>false</code> if it cannot.
<div class="algorithm" data-algorithm="supports-session-mode">
When the <dfn method for="XR">supportsSession(|options|)</dfn> method is invoked, it MUST return [=a new Promise=] |promise| and run the following steps [=in parallel=]:
1. [=Ensure an XR device is selected=].
1. If [=XR device=] is <code>null</code>, [=reject=] |promise| with a {{NotSupportedError}} and abort these stepss.
1. If the |options| are not [=supported by the XR device=], [=reject=] |promise| with a {{NotSupportedError}} and abort these steps.
1. Else [=/resolve=] |promise|.
</div>
Calling {{XR/supportsSession()}} MUST NOT trigger device-selection UI as this would cause many sites to display XR-specific dialogs early in the document lifecycle without user activation.
The {{XR}} object has an <dfn>active immersive session</dfn>, which MUST be initially <code>null</code>, and a <dfn>list of non-immersive sessions</dfn>, which MUST be initially empty.
<div class="algorithm" data-algorithm="request-session">
When the <dfn method for="XR">requestSession(|options|)</dfn> method is invoked, the user agent MUST return [=a new Promise=] |promise| and run the following steps [=in parallel=]:
1. [=Ensure an XR device is selected=].
1. If [=XR device=] is <code>null</code>, [=reject=] |promise| with <code>null</code>.
1. If the |options| are not [=supported by the XR device=], [=reject=] |promise| with a {{NotSupportedError}} and abort these steps.
1. Let |immersive| be the {{XRSessionCreationOptions/immersive}} attribute of the |options| argument.
1. If |immersive| is <code>true</code> and the [=active immersive session=] is not <code>null</code>, [=reject=] |promise| with an {{InvalidStateError}} and abort these steps.
1. If |immersive| is <code>true</code> and the algorithm is not [=triggered by user activation=], [=reject=] |promise| with a {{SecurityError}} and abort these steps.
1. Let |session| be a new {{XRSession}}.
1. [=Initialize the session=] |session| with the [=session description=] given by |options|.
1. If |immersive| is <code>true</code> set the [=active immersive session=] to |session|.
1. Else append |session| to the [=list of non-immersive sessions=].
1. [=/Resolve=] |promise| with |session|.
</div>
<div class="example">
The following code attempts to retrieve an immersive {{XRSession}}.
<pre highlight="js">
let xrSession;
xrDevice.requestSession({ immersive: true }).then((session) => {
xrSession = session;
});
</pre>
</div>
Session {#session}
=======
XRSessionCreationOptions {#xrsessioncreationoptions-interface}
-------------------------
<pre class="idl">
dictionary XRSessionCreationOptions {
boolean immersive = false;
XRPresentationContext outputContext;
};
</pre>
The {{XRSessionCreationOptions}} dictionary provides a <dfn>session description</dfn>, indicating the desired properties of a session to be returned from {{requestSession()}}.
<div class="algorithm" data-algorithm="supported-by-device">
To determine if an {{XRSessionCreationOptions}} |options| is <dfn>supported by the XR device</dfn> run the following steps:
1. Let |immersive| be |options|.{{XRSessionCreationOptions/immersive}}.
1. If |immersive| is <code>true</code> and the [=XR device=]'s [=supports immersive=] boolean is <code>false</code> return <code>false</code>.
1. If |immersive| is <code>false</code> and |options|.{{XRSessionCreationOptions/outputContext}} is <code>null</code>, return <code>false</code>
</div>
A session is considered to be an <dfn>immersive session</dfn> if it's output is displayed to the user in a way that makes the user feel the content is present in the same space with them, shown at the proper scale. Sessions are considered <dfn>non-immersive</dfn> (sometimes referred to as inline) if their output is displayed as an element in an HTML document.
NOTE: Content shown as part of an HTML document is always considered [=non-immersive=] even if headtracking is taken into account, the content is displayed in stereo, or the document is displayed in a headset.
Issue: Document restrictions and capabilities of an [=immersive session=]
XRSession {#xrsession-interface}
---------
<pre class="idl">
enum XREnvironmentBlendMode {
"opaque",
"additive",
"alpha-blend",
};
[SecureContext, Exposed=Window] interface XRSession : EventTarget {
// Attributes
readonly attribute boolean immersive;
readonly attribute XRPresentationContext outputContext;
readonly attribute XREnvironmentBlendMode environmentBlendMode;
attribute double depthNear;
attribute double depthFar;
attribute XRLayer baseLayer;
// Methods
Promise<XRFrameOfReference> requestFrameOfReference(XRFrameOfReferenceType type, optional XRFrameOfReferenceOptions options);
FrozenArray<XRInputSource> getInputSources();
long requestAnimationFrame(XRFrameRequestCallback callback);
void cancelAnimationFrame(long handle);
Promise<void> end();
// Events
attribute EventHandler onblur;
attribute EventHandler onfocus;
attribute EventHandler onresetpose;
attribute EventHandler onend;
attribute EventHandler onselect;
attribute EventHandler oninputsourceschange;
attribute EventHandler onselectstart;
attribute EventHandler onselectend;
};
</pre>
Any interaction with XR hardware is done via an {{XRSession}} object, which can only be retrieved by calling {{requestSession()}} on the {{XR}} object. Once a session has been successfully acquired it can be used to [=poll the device pose=], query information about the user's environment and, present imagery to the user.
The user agent, when possible, <dfn>SHOULD NOT initialize device tracking</dfn> or rendering capabilities until an {{XRSession}} has been acquired. This is to prevent unwanted side effects of engaging the XR systems when they're not actively being used, such as increased battery usage or related utility applications from appearing when first navigating to a page that only wants to test for the presence of XR hardware in order to advertise XR features. Not all XR platforms offer ways to detect the hardware's presence without initializing tracking, however, so this is only a strong recommendation.
<div class="algorithm" data-algorithm="initialize-session">
When an {{XRSession}} is created, the user agent MUST <dfn>initialize the session</dfn> by running the following steps:
1. Let |session| be the newly created {{XRSession}} object.
1. Let |options| be the {{XRSessionCreationOptions}} passed to {{requestSession()}}.
1. Initialize |session|'s {{XRSession/immersive}} to |options| {{XRSessionCreationOptions/immersive}} value.
1. Initialize |session|'s {{XRSession/outputContext}} to |options| {{XRSessionCreationOptions/outputContext}} value.
1. Initialize |session|'s {{XRSession/depthNear}} to <code>0.1</code>.
1. Initialize |session|'s {{XRSession/depthFar}} to <code>1000.0</code>.
1. Initialize |session|'s {{XRSession/baseLayer}} to <code>null</code>.
1. If no other features of the user agent have done so already, perform the necessary platform-specific steps to initialize the device's tracking and rendering capabilities.
</div>
A number of diffrent circumstances may <dfn>shut down the session</dfn>, which is permanent and irreversable. Once a session has been shut down the only way to access the [=XR device=]'s tracking or rendering capabilities again is to request a new session. Each {{XRSession}} has an <dfn>ended</dfn> boolean, initially set to <code>false</code>, that indicates if it has been shut down.
<div class="algorithm" data-algorithm="shut-down-session">
When an {{XRSession}} is shut down the following steps are run:
1. Let |session| be the target {{XRSession}} object.
1. Set |session|'s [=ended=] value to <code>true</code>.
1. If the [=active immersive session=] is equal to |session|, set the [=active immersive session=] to <code>null</code>.
1. Remove |session| from the [=list of non-immersive sessions=].
1. If no other features of the user agent are actively using them, perform the necessary platform-specific steps to shut down the device's tracking and rendering capabilities.
</div>
<div class="algorithm" data-algorithm="end-session">
The <dfn method for="XRSession">end()</dfn> method provides a way to manually shut down a session. When invoked, it MUST return [=a new Promise=] |promise| and run the following steps [=in parallel=]:
1. [=Shut down the session|Shut down=] the target {{XRSession}} object.
1. [=/Resolve=] |promise|.
</div>
<div class="algorithm" data-algorithm="request-frame-of-reference">
When the <dfn method for="XRSession">requestFrameOfReference(|type|, |options|)</dfn> method is invoked, the user agent MUST return [=a new Promise=] |promise| and run the following steps [=in parallel=]:
1. Let |session| be the target {{XRSession}} object.
1. Let |frameOfRef| be a new {{XRFrameOfReference}}.
1. [=Initialize the frame of reference=] |frameOfRef| with |session|, |type|, and |options|.
1. [=/Resolve=] |promise| with |frameOfRef|.
</div>
<div class="algorithm" data-algorithm="get-input-sources">
When the <dfn method for="XRSession">getInputSources()</dfn> method is invoked, the user agent MUST run the following steps:
1. Return the current [=list of active input sources=].
</div>
Each {{XRSession}} has a <dfn>environment blending mode</dfn> value, which is a enum which MUST be set to whichever of the following values best matches the behavior of imagery rendered by the session in relation to the user's surrounding environment.
- A blend mode of <dfn enum-value for="XREnvironmentBlendMode">opaque</dfn> indicates that the user's surrounding environment is not visible at all. Alpha values in the {{XRSession/baseLayer}} will be ignored, with the compositor treating all alpha values as 1.0.
- A blend mode of <dfn enum-value for="XREnvironmentBlendMode">additive</dfn> indicates that the user's surrounding environment is visible and the {{XRSession/baseLayer}} will be shown additively against it. Alpha values in the {{XRSession/baseLayer}} will be ignored, with the compositor treating all alpha values as 1.0. When this blend mode is in use black pixels will appear fully transparent, and there is no way to make a pixel appear fully opaque.
- A blend mode of <dfn enum-value for="XREnvironmentBlendMode">alpha-blend</dfn> indicates that the user's surrounding environment is visible and the {{XRSession/baseLayer}} will be blended with it according to the alpha values of each pixel. Pixels with an alpha value of 1.0 will be fully opaque and pixels with an alpha value of 0.0 will be fully transparent.
The <dfn attribute for="XRSession">environmentBlendMode</dfn> attribute returns the {{XRSession}}'s [=environment blending mode=]
NOTE: Most Virtual Reality devices exhibit {{XREnvironmentBlendMode/opaque}} blending behavior. Augmented Reality devices that use transparent optical elements frequently exhibit {{XREnvironmentBlendMode/additive}} blending behavior, and Augmented Reality devices that use passthrough cameras frequently exhibit {{XREnvironmentBlendMode/alpha-blend}} blending behavior.
The <dfn attribute for="XRSession">onblur</dfn> attribute is an [=Event handler IDL attribute=] for the {{blur}} event type.
The <dfn attribute for="XRSession">onfocus</dfn> attribute is an [=Event handler IDL attribute=] for the {{focus}} event type.
The <dfn attribute for="XRSession">onresetpose</dfn> attribute is an [=Event handler IDL attribute=] for the {{resetpose}} event type.
The <dfn attribute for="XRSession">onend</dfn> attribute is an [=Event handler IDL attribute=] for the {{end}} event type.
The <dfn attribute for="XRSession">oninputsourceschange</dfn> attribute is an [=Event handler IDL attribute=] for the {{inputsourceschange}} event type.
The <dfn attribute for="XRSession">onselectstart</dfn> attribute is an [=Event handler IDL attribute=] for the {{selectstart}} event type.
The <dfn attribute for="XRSession">onselectend</dfn> attribute is an [=Event handler IDL attribute=] for the {{selectend}} event type.
The <dfn attribute for="XRSession">onselect</dfn> attribute is an [=Event handler IDL attribute=] for the {{XRSession/select}} event type.
Issue: Example of acquiring a session here.
Issue: Document what happens when we end the session.
Issue: Document effects when we <dfn lt="blur all sessions">blur the session</dfn>.
Issue: Document how to <dfn>poll the device pose</dfn>.
Issue: Document how the <dfn>list of active input sources</dfn> is maintained.
Animation Frames {#animation-frames}
----------------
<pre class="idl">
callback XRFrameRequestCallback = void (DOMHighResTimeStamp time, XRFrame frame);
</pre>
Each {{XRFrameRequestCallback}} object has a <dfn for="XRFrameRequestCallback">cancelled</dfn> boolean initially set to <code>false</code>.
Each {{XRSession}} has a <dfn>list of animation frame callbacks</dfn>, which is initially empty, and an <dfn>animation frame callback identifier</dfn>, which is a number initially be zero. Each {{XRSession}} also has a <dfn>processing frame</dfn> boolean, which is initially be set to <code>false</code>.
<div class="algorithm" data-algorithm="request-animation-frame">
When the <dfn method for="XRSession">requestAnimationFrame(|callback|)</dfn> method is invoked, the user agent MUST run the following steps:
1. Let |session| be the target {{XRSession}} object.
1. Increment |session|'s [=animation frame callback identifier=] by one.
1. Append |callback| to |session|'s [=list of animation frame callbacks=], associated with |session|'s [=animation frame callback identifier=]’s current value.
1. Return |session|'s [=animation frame callback identifier=]’s current value.
</div>
<div class="algorithm" data-algorithm="cancel-animation-frame">
When the <dfn method for="XRSession">cancelAnimationFrame(|handle|)</dfn> method is invoked, the user agent MUST run the following steps:
1. Let |session| be the target {{XRSession}} object.
1. Find the entry in |session|'s [=list of animation frame callbacks=] that is associated with the value |handle|.
1. If there is such an entry, set it's [=cancelled=] boolean to <code>true</code> and remove it from |session|'s [=list of animation frame callbacks=].
</div>
<div class="algorithm" data-algorithm="run-animation-frames">
When the user agent is to run the animation frame callbacks for an {{XRSession}} |session| with a timestamp |now| and an {{XRFrame}} |frame|, it MUST run the following steps:
1. Let |callbacks| be a list of the entries in |session|'s [=list of animation frame callback=], in the order in which they were added to the list.
1. Set |session|'s [=list of animation frame callbacks=] to the empty list.
1. Set |session|'s [=processing frame=] to <code>true</code>.
1. For each entry in |callbacks|, in order:
1. If the entry's [=cancelled=] boolean is <code>true</code>, continue to the next entry.
1. [=Invoke the Web IDL callback function=], passing |now| and |frame| as the arguments
1. If an exception is thrown, [=report the exception=].
1. Set |session|'s [=processing frame=] to <code>false</code>.
</div>
The XR Compositor {#compositor}
-----------------
Issue: This needs to be broken up a bit more and more clearly decribe things such as the frame lifecycle.
The user agent MUST maintain an <dfn>XR Compositor</dfn> which handles presentation to the [=XR device=] and frame timing. The compositor MUST use an independent rendering context whose state is isolated from that of any WebGL contexts used as {{XRWebGLLayer}} sources to prevent the page from corrupting the compositor state or reading back content from other pages. the compositor MUST also run in separate thread or processes to decouple performance of the page from the ability to present new imagery to the user at the appropriate framerate.
The [=XR Compositor=] has a list of layer images, which is initially empty.
<!--There are no direct interfaces to the compositor, but applications may submit bitmaps to be composited via the layer system and observe the frame timing via calls to {{XRSession/requestAnimationFrame()}}. The compositor consists of two different loops, assumed to be running in separate threads or processes. The <dfn>Frame Loop</dfn>, which drives the page script, and the <dfn>Render Loop</dfn>, which continuously presents imagery provided by the Frame Loop to the XR device. The render loop maintains its own copy of the session's layer list. Communication between the two loops is synchronized with a lock that limits access to the render loop's layer list.
Both loops are started when a session is successfully created. The compositor's render loop goes through the following steps:
1. The layer lock is acquired.
1. The render loop's layer list images are composited and presented to the device.
1. The layer lock is released.
1. Notify the frame loop that a frame has been completed.
1. return to step 1.
The render loop MUST throttle its throughput to the refresh rate of the XR device. The exact point in the loop that is most effective to block at may differ between platforms, so no perscription is made for when that should happen.
Upon session creation, the following steps are taken to start the frame loop:
1. A new promise is created and set as the session's current frame promise. The current frame promise is returned any time XRCanvasLayer/commit() is called.
1. The {{sessionchange}} event is fired.
1. The promise returned from {{requestSession()}} is resolved.
Then, the frame loop performs the following steps while the session is active:
1. The render loop's layer lock is acquired.
1. Any dirty layers in the session's layer list are copied to the render loop's layer list.
1. The render loop's layer lock is released.
1. Wait for the render loop to signal that a frame has been completed.
1. The session's current frame promise is set as the the previous frame promise.
1. A new promise is created and set as the session's current frame promise.
1. The previous frame promise is resolved.
1. Once the promise has been resolved, return to step 1.-->
Frame Loop {#frame}
==========
XRFrame {#xrpresentationframe-interface}
-------------------
<pre class="idl">
[SecureContext, Exposed=Window] interface XRFrame {
readonly attribute XRSession session;
readonly attribute FrozenArray<XRView> views;
XRDevicePose? getDevicePose(XRCoordinateSystem coordinateSystem);
XRInputPose? getInputPose(XRInputSource inputSource, XRCoordinateSystem coordinateSystem);
};
</pre>
An {{XRFrame}} provides all the values needed to render a single frame of an XR scene to the [=XR device=]'s display. Applications can only aquire an {{XRFrame}} by calling {{XRSession/requestAnimationFrame()}} on an {{XRSession}} with an {{XRFrameRequestCallback}}. When the callback is called it will be passed an {{XRFrame}}.
The <dfn attribute for="XRFrame">session</dfn> attribute returns the {{XRSession}} that produced the {{XRFrame}}.
<dfn attribute for="XRFrame">views</dfn>
<dfn method for="XRFrame">getDevicePose(coordinateSystem)</dfn>
<dfn method for="XRFrame">getInputPose(inputSource, coordinateSystem)</dfn>
Coordinate Systems {#coordinatesystems}
==================
XRCoordinateSystem {#xrcoordinatesystem-interface}
------------------
<pre class="idl">
[SecureContext, Exposed=Window] interface XRCoordinateSystem : EventTarget {
Float32Array? getTransformTo(XRCoordinateSystem other);
};
</pre>
<div class="algorithm" data-algorithm="get-transform-to">
When the <dfn method for="XRCoordinateSystem">getTransformTo(|other|)</dfn> method is invoked, the user agent MUST run the following steps:
1. Let |current| be the target {{XRCoordinateSystem}} object.
1. If a known transform exists from the coordinate system described by |current| to the coordinate system described by |other|, return it as a [=matrix=].
1. Else return <code>null</code>
</div>
XRFrameOfReference {#xrframeofreference-interface}
------------------
<pre class="idl">
enum XRFrameOfReferenceType {
"head-model",
"eye-level",
"stage",
};
dictionary XRFrameOfReferenceOptions {
boolean disableStageEmulation = false;
double stageEmulationHeight = 0.0;
};
[SecureContext, Exposed=Window] interface XRFrameOfReference : XRCoordinateSystem {
readonly attribute XRStageBounds? bounds;
readonly attribute double emulatedHeight;
attribute EventHandler onboundschange;
};
</pre>
An {{XRFrameOfReference}} describes an {{XRCoordinateSystem}} that is generally expected to remain static for the duration of the {{XRSession}}, with the most common exception being mid-session reconfiguration by the user. An {{XRFrameOfReference}} is created by calling {{XRSession/requestFrameOfReference()}}. Every {{XRFrameOfReference}} describes a coordinate system where the Y axis MUST be aligned with gravity, with positive Y being "Up". Negative Z is considered "Forward", and positive X is considered "Right".
Each {{XRFrameOfReference}} has as <dfn for="XRFrameOfReference">session</dfn>, which is the {{XRSession}} that created it, and a <dfn>frame of reference type</dfn>, which is one of the {{XRFrameOfReferenceType}} values, set to the type passed {{XRSession/requestFrameOfReference()}}. The behavior of the {{XRFrameOfReference}} differs depending on its [=frame of reference type=] as follows:
<b>eye-level</b>
An {{XRFrameOfReference}} with a [=frame of reference type=] of <dfn enum-value for="XRFrameOfReferenceType">"eye-level"</dfn> describes a coordinate system
with an origin that corresponds to the first device pose acquired by the {{XRSession}} after the <code>"head-model"</code> frame of reference is created, with the device yaw (rotation around the Y axis) at that point defining the coordinate system's forward direction.
<b>head-model</b>
An {{XRFrameOfReference}} with a [=frame of reference type=] of <dfn enum-value for="XRFrameOfReferenceType">"head-model"</dfn> describes a coordinate system identical to an {{XRFrameOfReferenceType/eye-level}} frame of reference, but where the device is always located at the origin. Any translation is provided by a software estimate of the devices position based solely on the device orientation and the assumption that the device is being worn as a headset. The translation estimate SHOULD be based off of modeling how the human head moves on average when an individual is only moving their neck.
If the device is capable of reporting accurate translation, it MUST be discarded when using this [=frame of reference type=] in favor of an orientation-based estimate. If the translation reported by the device is already based on head modeling the device-provided translation MAY be preserved.
NOTE: A <code>"head-model"</code> [=frame of reference type=] is useful when displaying content that cannot be viewed from arbitrary positions, such as panoramic images or videos.
<b>stage</b>
An {{XRFrameOfReference}} with a [=frame of reference type=] of <dfn enum-value for="XRFrameOfReferenceType">"stage"</dfn> describes a space known as a <dfn for="XRStageBounds">stage</dfn>. A [=stage=] is a bounded, floor-relative play space that the user can be expected to safely be able to move within. Other XR platforms sometimes refer to this concept as "room scale" or "standing space".
Note: A [=stage=] is not intended to describe multi-room spaces, areas with uneven floor levels, or very large open areas. Future iterations of this specification may add more detailed support for tracking in those scenarios.
The <dfn lt="stage origin">origin of the stage</dfn> MUST be at floor level, such that Y equals 0 at the floor, is negative below the floor level, and is positive above the floor. The origin on the X and Z axes is determined in a platform-specific manner, but in general if the user is in an enclosed space it's ideal if the X and Z axes originate at or near the center of the room.
The [=stage bounds=] are described by a {{XRFrameOfReference}}'s <dfn attribute for="XRFrameOfReference">bounds</dfn> attribute. If the frame of reference is not a [=stage=] or the [=stage bounds=] cannot be determined the {{bounds}} attribute MUST be <code>null</code>.
Note: When the {{bounds}} for a [=stage=] are <code>null</code> the user should not be required to physically move around their environment in order to interact with content. The device may still support [=6DoF=] tracking, but it can't be assumed that the user will know where they are relative to their environment and as such content that encourages movement beyond leaning is discouraged.
The <dfn attribute for="XRFrameOfReference">onboundschange</dfn> attribute is an [=Event handler IDL attribute=] for the {{boundschange}} event type.
The <dfn attribute for="XRFrameOfReference">emulatedHeight</dfn> attribute is initially set to <code>0.0</code>. This value indicates the offset in meters along the Y axis that will be applied to poses acquired with this frame of reference.
<div class="algorithm" data-algorithm="initialize-frame-of-reference">
When an {{XRFrameOfReference}} is created, the user agent MUST <dfn>initialize the frame of reference</dfn> by running the following steps:
1. Let |frameOfRef| be the newly created {{XRFrameOfReference}} object.
1. Let |session| be the {{XRSession}} object that requested |frameOfRef|'s creation.
1. Let |type| be the {{XRFrameOfReferenceType}} passed to {{XRSession/requestFrameOfReference()}}.
1. Let |options| be the {{XRFrameOfReferenceOptions}} passed to {{XRSession/requestFrameOfReference()}}.
1. Initialize |frameOfRef|'s [=XRFrameOfReference/session=] to |session|.
1. Initialize |frameOfRef|'s [=frame of reference type=] to |type|.
1. If |type| is not {{XRFrameOfReferenceType/"stage"}}, abort these steps.
1. // Initialize stage-specific properties
</div>
XRStageBounds {#xrstagebounds-interface}
-------------
<pre class="idl">
[SecureContext, Exposed=Window] interface XRStageBounds {
readonly attribute FrozenArray<DOMPointReadOnly> geometry;
};
</pre>
The {{XRStageBounds}} interface describes a border around a [=stage=], known as the <dfn>stage bounds</dfn> which the user can be expected to safely be able to move within.
The polygonal boundary is given by the <dfn attribute for="XRStageBounds">geometry</dfn> array of {{DOMPointReadOnly}}s, which represents a loop of points at the edges of the safe space. The points describe offsets from the [=stage origin=] in meters, and MUST be given in a clockwise order as viewed from above, looking towards the negative end of the Y axis. The {{DOMPointReadOnly/y}} value of each point MUST be <code>0</code> and the {{DOMPointReadOnly/w}} value of each point MUST be <code>1</code>. The bounds can be considered to originate at the floor and extend infinitely high. The shape it describes MAY not be convex. The values reported are relative to the [=stage origin=], but MAY not contain it.
Note: Content should not require the user to move beyond the [=stage bounds=]; however, if their physical surroundings allow for it, it is possible for the user to ignore the bounds resulting in position values outside of the polygon they describe. This is not an error condition and should be handled gracefully by page content.
Note: Content generally should not provide a visualization of the [=stage bounds=], as it's the user agent's responsibility to ensure that safety critical information is provided to the user.
Views {#views}
=====
XRView {#xrview-interface}
------
<pre class="idl">
enum XREye {
"left",
"right"
};
[SecureContext, Exposed=Window] interface XRView {
readonly attribute XREye eye;
readonly attribute Float32Array projectionMatrix;
};
</pre>
An {{XRView}} describes a single <dfn>view</dfn> into an XR scene. Each [=view=] corresponds to a display or portion of a display used by an XR device to present imagery to the user. They are used to retrieve all the information necessary to render content that is well aligned to the [=view=]'s physical output properties, including the field of view, eye offset, and other optical properties. [=Views=] may cover overlapping regions of the user's vision. No guarantee is made about the number of [=views=] any XR device uses or their order, nor is the number of [=views=] required to be constant for the duration of an {{XRSession}}.
NOTE: Many HMDs will request that content render two [=views=], one for the left eye and one for the right, while most magic window devices will only request one [=view=], but applications should never assume a specific view comfiguration. For example: A magic window device may request two views if it is capable of stereo output, but may revert to requesting a single view for performance reasons if the stereo output mode is turned off. Similarly, HMDs may request more than two views to facilitate a wide field of view or displays of different pixel density.
The <dfn attribute for="XRView">eye</dfn> attribute describes which eye this view is expected to be shown to. This attribute's primary purpose is to ensure that prerendered stereo content can present the correct portion of the content to the correct eye. If the view does not have an intrinsicly associated eye (the display is monoscopic, for example) this attribute MUST be set to {{XREye/"left"}}.
The <dfn attribute for="XRView">projectionMatrix</dfn> attribute provides a [=matrix=] describing the projection to be used for the view's rendering. It is <b>strongly recommended</b> that applications use this matrix without modification. Failure to use the provided projection matrices when rendering may cause the presented frame to be distorted or badly aligned, resulting in varying degrees of user discomfort.
XRViewport {#xrviewport-interface}
------
<pre class="idl">
[SecureContext, Exposed=Window] interface XRViewport {
readonly attribute long x;
readonly attribute long y;
readonly attribute long width;
readonly attribute long height;
};
</pre>
An {{XRViewport}} object describes a viewport, or rectangular region, of a graphics surface. The <dfn attribute for="XRViewport">x</dfn> and <dfn attribute for="XRViewport">y</dfn> attributes define an offset from the surface origin and the <dfn attribute for="XRViewport">width</dfn> and <dfn attribute for="XRViewport">height</dfn> attributes define the rectangular dimensions of the viewport.
The exact interpretation of the viewport values depends on the conventions of the graphics API the viewport is associated with:
- When used with a {{XRWebGLLayer}} the {{XRViewport/x}} and {{XRViewport/y}} attributes specify the lower left corner of the viewport rectangle, in pixels, with the viewport rectangle extending {{XRViewport/width}} pixels to the right of {{XRViewport/x}} and {{XRViewport/height}} pixels above {{XRViewport/y}}. The values can be passed to the [=WebGL viewport=] function directly.
<div class="example">
The following code sets the [=WebGL viewport=] for an {{XRWebGLLayer}}'s framebuffer using an {{XRViewport}} retrieved from an {{XRView}}.
<pre highlight="js">
xrSession.requestAnimationFrame((time, xrFrame) => {
let xrView = xrFrame.views[0];
let xrViewport = xrWebGLLayer.getViewport(xrView);
gl.bindFramebuffer(xrWebGLLayer.framebuffer);
gl.viewport(xrViewport.x, xrViewport.y, xrViewport.width, xrViewport.height);
// WebGL draw calls will now be rendered into the appropriate viewport.
});
</pre>
</div>
Pose {#pose}
====
Matrices {#matrices}
--------
WebXR provides various transforms in the form of <dfn lt="matrix">matrices</dfn>. WebXR matrices are always 4x4 and given as 16 element {{Float32Array}}s in column major order. They may be passed directly to WebGL's {{uniformMatrix4fv}} function, used to create an equivalent {{DOMMatrix}}, or used with a variety of third party math libraries.
Translations specified by WebXR matrices are always given in meters.
XRDevicePose {#xrdevicepose-interface}
-------------
<pre class="idl">
[SecureContext, Exposed=Window] interface XRDevicePose {
readonly attribute Float32Array poseModelMatrix;
Float32Array getViewMatrix(XRView view);
};
</pre>
An {{XRDevicePose}} describes the position and orientation of an [=XR device=] relative to the {{XRCoordinateSystem}} it was queried with. It also describes the view and projection matrices that should be used by the application to render a frame of an XR scene.
The <dfn attribute for="XRDevicePose">poseModelMatrix</dfn> is a [=matrix=] describing the transform from the origin of the {{XRCoordinateSystem}} to the [=XR device=].
NOTE: The {{XRDevicePose/poseModelMatrix}} can be used to position graphical representations of the [=XR device=] for spectator views of the scene or multi-user interaction.
The <dfn method for="XRDevicePose">getViewMatrix(view)</dfn> method returns a [=matrix=] describing the view transform to be used when rendering the passed {{XRView}}. The matrices represent the inverse of the model matrix of the associated viewpoint.
Input {#input}
=====
XRInputSource {#xrinputsource-interface}
-------------
<pre class="idl">
enum XRHandedness {
"",
"left",
"right"
};
enum XRTargetRayMode {
"gaze",
"tracked-pointer",
"screen"
};
[SecureContext, Exposed=Window]
interface XRInputSource {
readonly attribute XRHandedness handedness;
readonly attribute XRTargetRayMode targetRayMode;
};
</pre>
Each {{XRInputSource}} SHOULD define a <dfn>primary action</dfn>. The [=primary action=] is a platform-specific action that, when engaged, produces {{selectstart}}, {{selectend}}, and {{XRSession/select}} events. Examples of possible [=primary action=]s are pressing a trigger, touchpad, or button, speaking a command, or making a hand gesture. If the platform guidelines define a recommended primary input then it should be used as the [=primary action=], otherwise the user agent is free to select one.
The <dfn attribute for="XRInputSource">handedness</dfn> attribute describes which hand the input source is associated with, if any. Input sources with no natural handedness (such as headset-mounted controls or standard gamepads) or for which the handedness is not currently known MUST set this attribute to the empty string.
The <dfn attribute for="XRInputSource">targetRayMode</dfn> attribute describes the method used to produce the target ray, and indicates how the application should present the target ray to the user if desired.
- <dfn enum-value for="XRTargetRayMode">gaze</dfn> indicates the target ray will originate at the user's head and follow the direction they are looking (this is commonly referred to as a "gaze input" device).
- <dfn enum-value for="XRTargetRayMode">tracked-pointer</dfn> indicates that the target ray originates from either a handheld device or other hand-tracking mechanism and represents that the user is using their hands or the held device for pointing.
- <dfn enum-value for="XRTargetRayMode">screen</dfn> indicates that the input source was an interaction with the canvas element associated with a non-immersive session's output context, such as a mouse click or touch event.
Note: Some input sources, like an {{XRInputSource}} with {{targetRayMode}} set to {{screen}}, will only be added to the session's [=list of active input sources=] immediately before the {{selectstart}} event, and removed from the session's [=list of active input sources=] immediately after the {{selectend}} event.
XRRay {#xrray-interface}
-----
<pre class="idl">
[SecureContext, Exposed=Window]
interface XRRay {
readonly attribute DOMPointReadOnly origin;
readonly attribute DOMPointReadOnly direction;
readonly attribute Float32Array transformMatrix;
};
</pre>
An {{XRRay}} describes a geometric ray. The <dfn attribute for="XRRay">origin</dfn> attribute defines the 3-dimensional point in space that the ray originates from. The {{XRRay/origin}}'s {{DOMPointReadOnly/w}} attribute MUST be <code>1.0</code>. The <dfn attribute for="XRRay">direction</dfn> attribute defines the ray's 3-dimensional directional vector. The {{direction}}'s {{DOMPointReadOnly/w}} attribute MUST be <code>0.0</code> and the vector MUST be normalized to have a length of <code>1.0</code>.
The <dfn attribute for="XRRay">transformMatrix</dfn> is a [=matrix=] which represents the transform from a ray originating at <code>[0, 0, 0]</code> and extending down the negative Z axis to the ray described by the {{XRRay}}'s {{XRRay/origin}} and {{direction}}.
NOTE: The {{XRRay}}'s {{transformMatrix}} can be used to easily position graphical representations of the ray when rendering.
XRInputPose {#xrinputpose-interface}
-------------
<pre class="idl">
[SecureContext, Exposed=Window]
interface XRInputPose {
readonly attribute boolean emulatedPosition;
readonly attribute XRRay targetRay;
readonly attribute Float32Array? gripMatrix;
};
</pre>
<dfn attribute for="XRInputPose">targetRay</dfn>
<dfn attribute for="XRInputPose">gripMatrix</dfn>
The <dfn attribute for="XRInputPose">emulatedPosition</dfn> attribute indicates the accuracy of the {{XRRay/origin}} of the {{XRInputPose/targetRay}} and translation component of the {{XRInputPose/gripMatrix}}. {{XRInputPose/emulatedPosition}} MUST be set to <code>true</code> if positional values are software estimations, such as those provided by a neck or arm model. {{XRInputPose/emulatedPosition}} MUST be set to <code>false</code> if the positional values are based on sensor readings.
Layers {#layers}
======
XRLayer {#xrlayer-interface}
-------
<pre class="idl">
[SecureContext, Exposed=Window] interface XRLayer {};
</pre>
An {{XRLayer}} defines a source of bitmap images and a description of how the image is to be rendered to the [=XR device=]. Initially only one type of layer, the {{XRWebGLLayer}}, is defined but future revisions of the spec may extend the available layer types.
XRWebGLLayer {#xrwebgllayer-interface}
-------
Each {{XRSession}} MUST identify a <dfn>native WebGL framebuffer resolution</dfn>, which is the pixel resolution of a WebGL framebuffer required to match the physical pixel resolution of the [=XR device=].
<div class="algorithm" data-algorithm="native-webgl-framebuffer-resolution">
The [=native WebGL framebuffer resolution=] is detemined by running the following steps:
1. Let |session| be the target {{XRSession}}.
1. If |session|'s {{XRSession/immersive}} value is <code>true</code>, set the [=native WebGL framebuffer resolution=] to the resolution required to have a 1:1 ratio between the pixels of a framebuffer large enough to contain all of the session's {{XRView}}s and the physical screen pixels in the area of the display under the highest magnification and abort these steps. If no method exists to determine the native resolution as described, the [=recommended WebGL framebuffer resolution=] MAY be used.
1. If |session|'s {{XRSession/immersive}} value is <code>false</code>, set the [=native WebGL framebuffer resolution=] to the size of the |session|'s {{XRSession/outputContext}}'s {{XRPresentationContext/canvas}} in physical display pixels and reevaluate these steps every time the size of the canvas changes.
</div>
Additionally, the {{XRSession}} MUST identify a <dfn>recommended WebGL framebuffer resolution</dfn>, which represents a best estimate of the WebGL framebuffer resolution large enough to contain all of the session's {{XRView}}s that provides an average application a good balance between performance and quality. It MAY be smaller than, larger than, or equal to the [=native WebGL framebuffer resolution=].
NOTE: The user agent is free to use and method of it's choosing to estimate the [=recommended WebGL framebuffer resolution=]. If there are platform-specific methods for querying a recommended size it is recommended that they be used, but not required.
<pre class="idl">
typedef (WebGLRenderingContext or
WebGL2RenderingContext) XRWebGLRenderingContext;
dictionary XRWebGLLayerInit {
boolean antialias = true;
boolean depth = true;
boolean stencil = false;
boolean alpha = true;
boolean multiview = false;
double framebufferScaleFactor = 1.0;
};
[SecureContext, Exposed=Window, Constructor(XRSession session,
XRWebGLRenderingContext context,
optional XRWebGLLayerInit layerInit)]
interface XRWebGLLayer : XRLayer {
// Attributes
readonly attribute XRWebGLRenderingContext context;
readonly attribute boolean antialias;
readonly attribute boolean depth;
readonly attribute boolean stencil;
readonly attribute boolean alpha;
readonly attribute boolean multiview;
readonly attribute WebGLFramebuffer framebuffer;
readonly attribute unsigned long framebufferWidth;
readonly attribute unsigned long framebufferHeight;
// Methods
XRViewport? getViewport(XRView view);
void requestViewportScaling(double viewportScaleFactor);
// Static Methods
static double getNativeFramebufferScaleFactor(XRSession session);
};
</pre>
<div class="algorithm" data-algorithm="construct-webgl-layer">
The <dfn constructor for="XRWebGLLayer">XRWebGLLayer(|session|, |context|, |layerInit|)</dfn> constructor MUST perform the following steps when invoked:
1. Let |layer| be a new {{XRWebGLLayer}}
1. If |session|'s [=ended=] value is <code>true</code>, throw an {{InvalidStateError}} and abort these steps.
1. If |context| is lost, throw an {{InvalidStateError}} and abort these steps.
1. If |context|'s [=XR compatible=] boolean is false, throw an {{InvalidStateError}} and abort these steps.
1. Initialize |layer|'s {{XRWebGLLayer/context}} to |context|.
1. Initialize |layer|'s {{XRWebGLLayer/antialias}} to |layerInit|'s {{XRWebGLLayerInit/antialias}} value.
1. Initialize |layer|'s {{XRWebGLLayer/depth}} to |layerInit|'s {{XRWebGLLayerInit/depth}} value.
1. Initialize |layer|'s {{XRWebGLLayer/stencil}} to |layerInit|'s {{XRWebGLLayerInit/stencil}} value.
1. Initialize |layer|'s {{XRWebGLLayer/alpha}} to |layerInit|'s {{XRWebGLLayerInit/alpha}} value.
1. If |context| [=supports multiview=] and |layerInit|'s {{XRWebGLLayerInit/multiview}} value is <code>true</code>, initialize |layer|'s {{XRWebGLLayer/multiview}} to <code>true</code>.
1. Else initialize |layer|'s {{XRWebGLLayer/multiview}} to <code>false</code>.
1. Initialize |layer|'s {{XRWebGLLayer/framebuffer}} to a new [=opaque=] {{WebGLFramebuffer}} created with |context|.
1. Initialize the |layer|'s [=swap chain=].
1. If |layer|'s [=swap chain=] was unable to be created for any reason, throw an {{OperationError}} and abort these steps.
1. Return |layer|.
</div>
The <dfn attribute for="XRWebGLLayer">framebufferWidth</dfn> and <dfn attribute for="XRWebGLLayer">framebufferHeight</dfn> attributes return the width and height of the [=swap chain=]'s backbuffer, respectively.
{{getViewport()}} queries the {{XRViewport}} the given {{XRView}} should use when rendering to the layer.
<div class="algorithm" data-algorithm="get-viewport">
The <dfn method for="XRWebGLLayer">getViewport(|view|)</dfn> method, when invoked, MUST run the following steps:
1. Let |view| be the target {{XRView}}.
1. If |layer| was created with a different {{XRSession}} than the one that produced |view| return <code>null</code>
1. ???
</div>
<dfn method for="XRWebGLLayer">requestViewportScaling(viewportScaleFactor)</dfn>
<div class="algorithm" data-algorithm="get-native-framebuffer-scale-factor">
The <dfn method for="XRWebGLLayer">getNativeFramebufferScaleFactor(|session|)</dfn> method, when invoked, MUST run the following steps:
1. Let |session| be the target {{XRSession}}.
1. If |session|'s [=ended=] value is <code>true</code>, return <code>0.0</code> and abort these steps.
1. Return the value that the |session|'s [=recommended WebGL framebuffer resolution=] must be multiplied by to yield the |session|'s [=native WebGL framebuffer resolution=].
</div>
Issue: Document what it means when a context <dfn>supports multiview</dfn>.
Issue: Document what an <dfn>opaque</dfn> framebuffer is.
Issue: Document the creation of a <dfn>swap chain</dfn>.
Issue: Need an example snippet of setting up and using an {{XRWebGLLayer}}.
WebGL Context Compatiblity {#contextcompatibility}
--------------------------
<pre class="idl">
partial dictionary WebGLContextAttributes {
boolean xrCompatible = null;
};
partial interface mixin WebGLRenderingContextBase {
Promise<void> makeXRCompatible();
};
</pre>
When a user agent implements this specification it MUST set a <dfn>XR compatible</dfn> boolean, initially set to <code>false</code>, on every {{WebGLRenderingContextBase}}.
In order for a WebGL context to be used as a source for XR imagery it must be created on a <dfn>compatible graphics adapter</dfn> for the [=XR device=]. What is considered a [=compatible graphics adapter=] is platform dependent, but is understood to mean that the graphics adapter can supply imagery to the [=XR device=] without undue latency. If a WebGL context was not already created on the [=compatible graphics adapter=], it typically must be re-created on the adapter in question before it can be used with an {{XRWebGLLayer}}. Once the [=XR compatible=] boolean is set, the context can be used with layers for any {{XRSession}} requested from that [=XR device=].
Note: On an XR platform with a single GPU, it can safely be assumed that the GPU is compatible with the [=XR device=]s advertised by the platform, and thus any hardware accelerated WebGL contexts are compatible as well. On PCs with both an integrated and discreet GPU the discreet GPU is often considered the [=compatible graphics adapter=] since it generally a higher performance chip. On desktop PCs with multiple graphics adapters installed, the one with the [=XR device=] physically connected to it is likely to be considered the [=compatible graphics adapter=].
The [=XR compatible=] boolean can be set either at context creation time or after context creation, potentially incurring a context loss. To set the [=XR compatible=] boolean at context creation time, the {{xrCompatible}} context creation attibute must be set to <code>true</code> when requesting a WebGL context.
<div class="algorithm" data-algorithm="create-with-compatible-xr-device">
When the {{HTMLCanvasElement}}'s getContext() method is invoked with a {{WebGLContextAttributes}} dictionary with {{xrCompatible}} set to <code>true</code>, run the following steps:
1. Let |attributes| be the {{WebGLContextAttributes}} passed to the function.
1. [=Create the WebGL context=] as usual, ensuring it is created on a [=compatible graphics adapter=] for the [=XR device=].
1. Let |context| be the newly created WebGL context.
1. Set |context|'s [=XR compatible=] boolean to true.
1. Return |context|.
</div>
<div class="example">
The following code creates a WebGL context that is compatible with an [=XR device=] and then uses it to create an {{XRWebGLLayer}}.
<pre highlight="js">
function onXRSessionStarted(xrSession) {
let glCanvas = document.createElement("canvas");
let gl = glCanvas.getContext("webgl", { xrCompatible: true });
loadWebGLResources();
xrSession.baseLayer = new XRWebGLLayer(xrSession, gl);
}
</pre>
</div>
To set the [=XR compatible=] boolean after the context has been created, the {{makeXRCompatible()}} method is used.