Update navigator interface and event targets #116

toji · 2016-10-14T17:42:29Z

THIS IS A BACKWARDS-COMPATIBILITY BREAKING CHANGE

TL;DR: navigator.getVRDisplays() is now navigator.vr.getDisplays(), connect and disconnect events now fire from the navigator.vr, and all other events fire from the VRDisplay. This is to enable use in workers or other non-main-thread contexts.

This updates the VR interfaces to remove dependencies on the window object and enable use of the API in contexts other than the main page thread, such as workers. (Note that the WorkerNavigator interface and worker exposure meta tags will be added in a follow-up PR.)

Because workers don't have access to the window object the API cannot depend on events that are fired from the window object if we want it to be useable in a worker or similar context. To facilitate this most events will be fired from the VRDisplay that the event is associated with, with the exception of events that describe display lifetime, such as vrdisplayconnect and vrdisplaydisconnect, which will now be fired off of the vr "namespace" object.

Also added a vrdisplaynavigate event that fires from the vr object which is functionally similar to vrdisplayactivate but is specifically for page-to-page navigation within VR. With the new changes it's not practical to have vrdisplayactivate handle navigation events since the event is intended to be fired early in the page lifetime and the page may not have had a chance to call getDisplays and install the appropriate listeners.

DigiTec · 2016-10-14T17:57:38Z

index.bs

+
+<pre class="idl">
+[NoInterfaceObject]
+interface VRDisplayEventHandlers {
  attribute EventHandler onvrdisplayactivate;


If we are making the breaking change, we should remove on* events and have people use addEventListener. on* should be used only for legacy cases that require its semantics.

Good point. I'll update this in a bit.

DigiTec · 2016-10-14T17:59:51Z

index.bs

 };
+
+VR implements EventTarget;


You can't implements EventTarget, this breaks the ability to do event listener logging by swapping out the EventTarget.prototype values since EventTarget as an implements is not part of your prototype chain.

Interesting, didn't know that. This syntax was picked up from the WebBluetooth spec, so they've probably got it wrong. Fix incoming.

Oh yeah, a lot of people get this wrong. In general, the pro of EventTarget is that it goes into your prototype chain and becomes a single source for the functions add/remove/dispatch, but the con is that it goes into your prototype chain so you have to design it in.

If you do implements it, then you get your own "copy" of those functions and people have to crawl the entire type system if they want to do any sort of event listener logging via script. This is how IE 11 was implemented and then I fixed this in Edge.

DigiTec · 2016-10-14T18:00:07Z

index.bs

-  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getVRDisplays();
-  readonly attribute FrozenArray&lt;VRDisplay&gt; activeVRDisplays;
-  readonly attribute boolean vrEnabled;
+interface VR {


interface VR : EventTarget

DigiTec · 2016-10-14T18:01:33Z

index.bs

-  readonly attribute FrozenArray&lt;VRDisplay&gt; activeVRDisplays;
-  readonly attribute boolean vrEnabled;
+interface VR {
+  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getDisplays();


Consider: getDevices, but only if we have a good reason for it. Same for active* if we change the getter here.

Not sure what you mean here: We did use the terminology "devices" previously, but moved away from that with WebVR 1.0. I don't see any reason to stop using the term "Displays" at this point.

DigiTec · 2016-10-14T18:02:53Z

index.bs

@@ -168,6 +168,9 @@ interface VRDisplay : EventTarget {
   */
  void submitFrame();
 };
+
+VRDisplay implements EventTarget;
+VRDisplay implements VRDisplayEventHandlers;


Is this the direction everyone is heading? We preferred partials over implements UNLESS multiple things would use the implements. At this time, only VR would use one set of handlers and only VRDisplay would use the other set. So why implements instead of partial or simply adidng it on the interface directly in this case since we are in a single spec? Curious what others think here.

Again, this was picked up from WebBluetooth. Not sure what the reasoning was there. Worth asking them.

DigiTec · 2016-10-14T18:10:11Z

index.bs

+A user agent MAY dispatch this event type to indicate that a {{VRDisplay}} has been disconnected.
+
+<dfn event for="VREventHandlers" id="vreventhandlers-vrdisplaynavigate-event">onvrdisplaynavigate</dfn>
+A user agent MAY dispatch this event type to indicate that the current page has been navigated to from a page that was actively presenting VR content to the {{VRDisplay}}. The current page can call {{requestPresent()}} in response to this event in order to stay in VR presentation mode.


We need to know when to fire this event since this will be one of the indications that we should stay in VR presentation mode.

Right, we definitely need to flesh out the order of operations on all the events really. I just wanted to make sure this was declared to head off questions about "wait, if activate is on the display then how does navigation work?"

DigiTec · 2016-10-14T18:16:07Z

index.bs

+User agents implementing this specification MUST provide the following new DOM events. The corresponding events must be of type {{VRDisplayEvent}} and must fire on the {{vr}} object. Registration for and firing of the events must follow the usual behavior of DOM4 Events.
+
+<dfn event for="VREventHandlers" id="vreventhandlers-vrdisplayconnect-event">onvrdisplayconnect</dfn>
+A user agent MAY dispatch this event type to indicate that a {{VRDisplay}} has been connected.


We need to define more what this means. We've been having issues with the expectations around this event. For instance, a Gear VR is technically always connected, but there is also the USB drop in connection and the mounted indicators. We don't want people to wait to hear a vrdisplayconnect event before they requestPresent or otherwise try to use VR.

Now that we have Rift, Gear and DayDream, let's define what we think this means for all 3. Rift is kind of easy since maybe it means you've either plugged it in via USB or that you've told the browser you want to be in VR available mode (this is the toggle I've been proposing for the UI so someone could say, please don't enable VR experiences right now I don't want a page to automatically use my Rift).

Completely agreed, but maybe as a different PR? (Since we're not changing the definition here, just where it fires from)

DigiTec · 2016-10-14T18:18:27Z

index.bs

@@ -50,7 +50,7 @@ This section describes the interfaces and functionality added to the DOM to supp
 The {{VRDisplay}} interface forms the base of all VR devices supported by this API. It includes generic information such as device IDs and descriptions.

 <pre class="idl">
-interface VRDisplay : EventTarget {
+interface VRDisplay {


This will be reverted since we do want to extend and not mixin the EventTarget

DigiTec · 2016-10-14T18:20:29Z

index.bs

@@ -689,7 +718,7 @@ partial interface HTMLIFrameElement {
 };
 </pre>

-The {{allowvr}} attribute is a boolean attribute. When specified, it indicates that {{Document}} objects in the iframe element's browsing context are to be allowed to access VR devices (if it's not blocked for other reasons, e.g. there is another ancestor iframe without this attribute set). {{Document}} objects in an iframe element without this attribute should reject calls to {{getVRDisplays()}} and should not fire any {{VRDisplayEvent}}.
+The {{allowvr}} attribute is a boolean attribute. When specified, it indicates that {{Document}} objects in the iframe element's browsing context are to be allowed to access VR devices (if it's not blocked for other reasons, e.g. there is another ancestor iframe without this attribute set). {{Document}} objects in an iframe element without this attribute should reject calls to {{getDisplays()}} and should not fire any {{VRDisplayEvent}}.


Is 'reject the call' the right lingo for a Promise returning API? I'm actually not sure, but I want to learn ;-) I would expect it to be 'reject the Promise returned by the call to getDisplays()'

That does sounds better, agreed.

AlbertoElias · 2016-10-14T23:05:07Z

index.bs

+<dfn event for="VRDisplay" id="vrdisplayblur-event">vrdisplayblur</dfn>
+A user agent MAY dispatch this event type to indicate that presentation to the display by the page is paused by the user agent, OS, or VR hardware. While a {{VRDisplay}} is blurred it does not lose it's presenting status ({{isPresenting}} continues to report true) but {{getFrameData()}} returns false without updating the provided {{VRFrameData}} and {{getPose()}} returns null. This is to prevent tracking while the user interacts with potentially sensitive UI. For example: A user agent SHOULD blur the presenting application when the user is typing a URL into the browser with a virtual keyboard, otherwise the presenting page may be able to guess the URL the user is entering by tracking their head motions.
+
+<dfn event for="VRDisplay" id="vrdisplayblur-event">vrdisplayfocus</dfn>


Small typo on the id, should be vrdisplayfocus

cvan · 2016-10-17T17:59:31Z

here's some proposed pseudo-code I've written for the common use cases of the APIs. thoughts?

Detection

// Exposes whether a VR display is available and accessible (e.g., permissions) on the user's system.

let vrButton = document.querySelector('#vr-button');

vrButton.dataset.available = false;

// This would replace `navigator.vrEnabled`.
navigator.vr.getAvailability().then(isAvailable => {
  vrButton.dataset.available = isAvailable;
});

navigator.vr.addEventListener('availabilitychanged', e => {
  vrButton.dataset.available = e.value;
});

Querying for displays

// This would replace `navigator.getVRDisplays()`.
navigator.vr.requestDisplays().then(displays => {
  console.log(displays);
  // >> [VRDisplay, VRDisplay]

  console.log(displays[0]);
  // >> VRDisplay {displayId: 1, displayName: "Oculus Rift", isConnected: true…}

  console.log(displays[1]);
  // >> VRDisplay {displayId: 2, displayName: "HTC Vive", isConnected: true…}
}).catch(err => {
  console.error(err);
});

navigator.vr.requestDisplays({
  filters: [{displayName: 'Oculus Rift'}]
}).then(displays => {
  console.log(displays);
  // >> [VRDisplay]

  console.log(displays[0]);
  // >> VRDisplay {displayId: 1, displayName: "Oculus Rift", isConnected: true…}
});

navigator.vr.requestDisplays({
  filters: [{displayNamePrefix: 'Oculus'}]
}).then(displays => {
  console.log(displays);
  // >> [VRDisplay]

  console.log(displays[0]);
  // >> VRDisplay {displayId: 1, displayName: "Oculus Rift", isConnected: true…}
});

navigator.vr.requestDisplays({
  filters: [
    {displayName: 'Oculus Rift'},
    {displayName: 'HTC Vive'}
  ]
}).then(displays => {
  console.log(displays);
  // >> [VRDisplay]

  console.log(displays[0]);
  // >> VRDisplay {displayId: 1, displayName: "Oculus Rift", isConnected: true…}

  console.log(display[1]);
  // >> VRDisplay {displayId: 2, displayName: "HTC Vive", isConnected: true…}
});

Presenting to displays

// Present to a connected HTC Vive.
navigator.vr.requestDisplays({
  filters: [{displayNamePrefix: 'HTC Vive'}]
}).then(displays => {
  if (!displays.length) {
    return;
  }

  return displays[0].requestPresent([
    {source: canvas}
  ]).then(
    render(display)
  );
});


// Very theoretical use case.
// Present to multiple connected headsets
// if we have both an Oculus Rift and HTC Vive.
navigator.vr.requestDisplays({
  filters: [
    {displayNamePrefix: 'Oculus Rift'},
    {displayNamePrefix: 'HTC Vive'}
  ]
}).then(displays => {
  return Promise.all(displays.map(displays => {
    return display.requestPresent([
      {source: canvas}
    ]).then(
      render(display)
    );
  });
}).catch(err => {
  console.error(err);
});


function render (display) {
  var onVRFrame = () => {
    display.requestAnimationFrame(onVRFrame);
    display.getFrameData(frameData);
    // …
    display.submitFrame();
  };
  return () => {
    display.requestAnimationFrame(onVRFrame);
  };
}

Link traversal

If display(s) is presenting, after the page is unloaded, the browser sets navigator.vr.referringDisplay (or navigator.vr.referringDisplays).

The next page can look at navigator.vr.referringDisplay and automatically present (without any user-gesture requirement):

if (navigator.vr.referringDisplay) {
  presentOrRender(navigator.vr.referringDisplay);
} else {
  navigator.vr.requestDisplays().then(displays => {
    if (!displays.length) {
      return Promise.reject();
    }

    return presentOrRender(displays[0]);
  });
}

function presentOrRender (display) {
  if (display.isPresenting) {
    return render(display);
  }
  return display.requestPresent([
    {source: canvas}
  ]).then(
    render(display)
  );
}

AlbertoElias · 2016-10-18T10:39:43Z

What would the benefit be for the API itself to support filtering in #getDisplays, wouldn't that encourage making content only for certain devices?

toji · 2016-10-18T17:26:06Z

I'm not particularly in favor of filtering based on the name. For one, that encourages needlessly exclusionary pages and two it relies on browser all exposing the name the same way AND never changing it. (Even the underlying SDK could change how it's reported and possibly break things.) True, you can already do this if you really want by just looping through the detected headsets and selecting one with name == "...", but that's an antipattern that I'd rather not see baked into the API.

I can see an argument for filtering on capabilities. Querying only 6DoF devices or only those with motion controls seems valuable. But again, it encourages exclusionary uses when I'd rather see developers focusing on progressive enhancements 99% of the time. (If you build a Tilt Brush clone, sure, require motion controllers, but don't block me from your 360 photo viewer because all I have is cardboard.)

Given the shape of the API filtering could be added in later without much trouble, so I'd vote to hold off on that till v2.

I'm curious about your intentions for getAvailability. Seems like it's got the potential to allow feature testing without spinning up the entire API, but that sort of depends on the API having an option for that. OpenVR has vr::VR_IsHmdPresent() which would work well here, but it can give false positives. On the Oculus side you could call ovr_Initialize without ovr_Create, but that seems less lightweight and I'm not sure when that fails. Certainly I would only expect the Oculus runtime to be installed on machines that are very likely to have a Rift attached, but again there's a possibility for false positives. As such, we'd have to be very careful about how we specced such a function to indicate that returning true only indicates that the system MAY have an HMD available.

cvan · 2016-10-19T01:22:16Z

thanks, good feedback. I agree the filtering is probably out of scope. It just seems like a common thing to want to get a Vive if it's present. what are your thoughts on the navigator.vr.referringDisplay? and I know we just went through a rename, but do you think it's a good idea to keep calling these displays? the API description and spec mentions displays and sensors. it seems like devices is a better name, IMO - at least that's my current thinking. thoughts?

toji · 2016-10-19T05:40:09Z

Hm... As I recall it was Mozilla that wanted the "display" name in the first place. ;)

I'm not really invested in either name, but I'm not excited to churn verbiage simply for the sake of churn. I also think there's something to be said for making it explicit that this API is intended to deal with devices that display content. That nicely separates it from things like controllers, which are arguable a "VRDevice" but definitely not a "VRDisplay".

NellWaliczek · 2016-10-27T04:36:17Z

index.bs

-partial interface Window {
-  attribute EventHandler onvrdisplayconnect;
-  attribute EventHandler onvrdisplaydisconnect;
-  attribute EventHandler onvrdisplayactivate;


Perhaps this is a silly question... but where did this portion of the idl get moved to? I thought the events were going to be added to VRDisplay and while I see the comments moved above, but I don't see the EventHandlers themselves. Am I misunderstanding how this works?

If we are making the breaking change, we should remove on* events and have people use addEventListener. on* should be used only for legacy cases that require its semantics.
#116 (comment)

Justin suggested that we remove them, which seemed reasonable to me. I'd like to make the required events more visible and easy to scan, but I've also found that there's no real convention for how events are presented in similar specs. Suggestions welcome!

As at @domenic suggests, the way to do this is to add "attribute EventHandlers". The HTML 5 spec does this for events like onload, onplaying and many others.

cvan · 2016-10-16T00:53:48Z

index.bs

@@ -213,6 +213,24 @@ Returns an array with the {{VRLayer}} currently being presented. MUST return an
 <dfn method for="VRDisplay">submitFrame()</dfn>
 Captures the current state of the {{VRLayer}} currently being presented and displays it on the {{VRDisplay}}. It is assumed that the frame was rendered using the {{VRPose}} and matrices provided by the last call to {{getFrameData()}}. If {{getFrameData()}} was not called prior to calling {{submitFrame()}} the user agent MAY warn the user of potentially malformed visuals or prevent the frame from being shown at all.

+### Events ### {#vrdisplay-events}
+
+User agents implementing this specification MUST provide the following new DOM events. The corresponding events must be of type {{VRDisplayEvent}} and must fire on a {{VRDisplay}} object. Registration for and firing of the events must follow the usual behavior of DOM4 Events.


can just say The UA MUST provide…

cvan · 2016-10-16T00:55:06Z

index.bs

-  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getVRDisplays();
-  readonly attribute FrozenArray&lt;VRDisplay&gt; activeVRDisplays;
-  readonly attribute boolean vrEnabled;
+interface VR : EventTarget{


don't you want to update index.idl too?

or we could just remove the file. thoughts?

space before {

Let's remove the IDL.

sgtm! opened #140

cvan · 2016-10-16T00:58:06Z

index.bs

+interface VR : EventTarget{
+  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getDisplays();
+  readonly attribute FrozenArray&lt;VRDisplay&gt; activeDisplays;
+  readonly attribute boolean isEnabled;


I feel like this should be a Promise that resolves a boolean of true or false. and possibly let's call it like navigator.vr.getAvailability().

cvan · 2016-10-16T01:00:18Z

index.bs

-  readonly attribute boolean vrEnabled;
+interface VR : EventTarget{
+  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getDisplays();
+  readonly attribute FrozenArray&lt;VRDisplay&gt; activeDisplays;


I've been thinking about this for a while. I'm not sure how useful this attribute is, especially considering the liveness of the VRDisplay instances (they have isConnected and isPresenting attributes).

how about instead having navigator.vr.referringDisplays(), which would resolve the displays that are actively being presented to and set upon page navigation, and the next page could look at those to know which to present to (though in most cases, only one)?

cvan · 2016-10-27T06:12:18Z

index.bs

+
+User agents implementing this specification MUST provide the following new DOM events. The corresponding events must be of type {{VRDisplayEvent}} and must fire on a {{VRDisplay}} object. Registration for and firing of the events must follow the usual behavior of DOM4 Events.
+
+<dfn event for="VRDisplay" id="vrdisplayactivate-event">vrdisplayactivate</dfn>


how do you feel about dropping the vrdisplay prefixes?

cvan · 2016-10-31T21:32:07Z

index.bs

 Return a Promise which resolves to a list of available {{VRDisplay}}s. The Promise MUST be rejected if the {{Document}} object is inside an iframe that does not have the {{allowvr}} attribute set.

-<dfn attribute for="Navigator" id="navigator-activevrdisplays-attribute">activeVRDisplays</dfn>
-{{activeVRDisplays}} includes every {{VRDisplay}} that is currently presenting.
+<dfn attribute for="VR" id="vr-activedisplays-attribute">activeDisplays</dfn>


do we need this? why can't we just use getDisplays() and look at displays[0].isPresenting?

cvan · 2016-10-31T21:32:26Z

index.bs

+<dfn attribute for="VR" id="vr-activedisplays-attribute">activeDisplays</dfn>
+{{activeDisplays}} includes every {{VRDisplay}} that is currently presenting.
+
+<dfn attribute for="VR" id="vr-isenabled-attribute">isEnabled</dfn>


is there a reason this isn't a Promise? why not getAvailability()?

cvan · 2016-10-31T21:34:31Z

index.bs


-<dfn attribute for="Navigator" id="navigator-vrenabled-attribute">vrEnabled</dfn>
-The {{vrEnabled}} attribute's getter must return true if the context object is allowed to use the feature indicated by attribute name {{allowvr}} and VR is supported, and false otherwise.
+<dfn event for="VR" id="vrdisplayconnect-event">vrdisplayconnect</dfn>


do we need to prefix these events with vrdisplay? FWIW, the events in the Web Bluetooth spec aren't prefixed with anything.

and we should probably use the past tense, IMO.

Good call on dropping the prefix.

As for the tense, I recall that we've discussed that before and found that the web at large is inconsistent. I would LOVE to be pointed at some proper codified guidance that says one or the other. I have no personal preference, as long as the spec is at least internally consistent.

yeah, I forget what our original rationale was. perhaps some DOM person (@annevk?) could point us to some reference/spec?

there's already a bunch of inconsistencies across APIs (an obvious one that comes to mind is load vs. DOMContentLoaded).

https://w3ctag.github.io/design-principles/#casing-rules by @domenic covers some but not tense. I think past tense is generally not used for events. We use past sense for promise-state attributes.

Looking through https://html.spec.whatwg.org/#events-2 it looks like present tense is correct (with some legacy exceptions as noted). I'll add that to the design principles list.

cvan · 2016-10-31T21:35:48Z

index.bs

+<dfn event for="VR" id="vrdisplaydisconnect-event">vrdisplaydisconnect</dfn>
+A user agent MAY dispatch this event type to indicate that a {{VRDisplay}} has been disconnected.
+
+<dfn event for="VR" id="vrdisplaynavigate-event">vrdisplaynavigate</dfn>


when would this fire. IMO, we should use a separate method such as navigator.getReferringDisplays() (or similarly named)? (see navigator.bluetooth.referringDisplay, for inspiration.)

domenic · 2016-11-01T14:40:48Z

If we are making the breaking change, we should remove on* events and have people use addEventListener. on* should be used only for legacy cases that require its semantics.

No!! Event handler IDL attributes are here to stay and should be on all new interfaces as well as old ones. w3ctag/design-principles#29

We preferred partials over implements UNLESS multiple things would use the implements.

Agreed.

RafaelCintron · 2016-11-03T01:20:53Z

index.bs

+A user agent MAY dispatch this event type to indicate that a {{VRDisplay}} has been disconnected.
+
+<dfn event for="VR" id="vr-navigate-event">navigate</dfn>
+A user agent MAY dispatch this event type to indicate that the current page has been navigated to from a page that was actively presenting VR content to the {{VRDisplay}}. The current page can call {{requestPresent()}} in response to this event in order to stay in VR presentation mode.


"... has been navigated to from a page ...." The words to and from being next to each other confuses me. I presume what was meant to go here is " ... has been navigated from a page ... "

good call on the phrasing.

borrowing some verbiage from the WHATWG's HTML Standard, perhaps we could change the text to read like this:

… the current browsing context is navigated from the initial document to another …

It would be good to get @annevk's opinion on anything that ties into navigation. From a brief skim this seems redundant with the Window's load event, but I'm sure I'm missing something.

it would be good to also get folks' opinions on the proposed interfaces (inspired by the Web Bluetooth API interfaces) outlined here (instead of, or in supplement, to the events): https://blog.mozvr.com/connecting-virtual-worlds-hyperlinks-in-webvr/#howitworks

"the current browsing context is navigated from the initial document to another" would be way too unspecific. You basically need to add a hook in HTML's processing model somewhere that defines exactly when this event is dispatched, provided it is indeed needed.

That is, you need to write a patch for the HTML Standard to enable this kind of thing.

RafaelCintron · 2016-11-03T01:24:17Z

index.bs

-partial interface Window {
-  attribute EventHandler onvrdisplayconnect;
-  attribute EventHandler onvrdisplaydisconnect;
-  attribute EventHandler onvrdisplayactivate;


As at @domenic suggests, the way to do this is to add "attribute EventHandlers". The HTML 5 spec does this for events like onload, onplaying and many others.

MortimerGoro · 2016-11-15T15:53:58Z

I suggest adding a new vrdisplay event type, called something like changeor parameterschange or configchange. This event would fire when users reconfigure some HMD params such as: offset between eyes, chaperone size or other configurable params.

What do you think?

This updates the VR interfaces to remove dependencies on the `window` object and enable use of the API in contexts other than the main page thread, such as workers. (Note that the WorkerNavigator interface and worker exposure meta tags will be added in a follow-up PR)

toji · 2017-01-20T21:18:14Z

Just rebased this patch. I think at this point it's non-controversial, though we'll probably need to make a few follow up tweaks. I do want to get this merged soon because we will be discussing several other significant changes to be built on top of it. I'll leave it up for the day, but barring any significant concerns being raised I plan on merging this tonight.

cvan · 2017-01-20T23:30:33Z

@toji I did a bunch of testing, and I have some proposed changes to this interface. I'll show you a mock interface I made for navigator.vr that essentially polyfills the existing WebVR 1.1 API. Noticed a few issues, inconsistencies with the proposed changes. I think my proposed changes handle everything right now, without making any more significant changes.

toji · 2017-01-21T00:38:27Z

That would be great, @cvan! Please do send them over. I've definitely got some changes to propose as well beyond what this patch does, but my proposals are more disruptive so I wanted to bring them up separately.

RafaelCintron · 2017-01-23T19:32:50Z

@toji , this change LGTM. Separately, we are discussing whether the current set of events should be collapsed as part of #111 but I think we should deal with that as part of a separate PR.

cvan · 2017-01-25T19:11:10Z

In my extension that polyfills navigator.vr I got the events nicely consolidated (especially with Chromium, since it fires more).

It's probably a bit hard to follow without all the context. We've talked about this a lot in various issues, including #30, this PR, and at the W3C Workshop and in meetings.

Let me know if this helps convey what I was thinking. Pay attention to navigator.getAvailability() (which would respect the Feature-Policy - the alternative to <iframe allowvr>: <iframe enable="vr">) and navigator.vr.addEventListener('display…', function (evt) { }) and navigator.vr.referringDisplay.

I researched Web Bluetooth, WebUSB, Presentation API, Web Audio, Web MIDI, among others, and this felt the most right to me. Open to suggestions. This will probably be easier to go through over our meeting next week, but I'd love to get people thinking about this now.

toji · 2017-01-26T17:03:38Z

@cvan: I'm assuming the changes that you talked about are what you showed in the IDL you posted to #179? As I mentioned in that issue, I think the concepts you're presenting have merit but should probably be broken out into individual issues/proposals/pull requests. In that light, it seems like there's value to merging this PR as-is, even if we want to make some adjustments on top of it, simply so that the delta of future updates is smaller and more manageable?

cvan

looking really good. I'm super pumped to get this in. see my comments :)

cvan · 2017-01-27T01:35:40Z

index.bs

@@ -169,6 +169,12 @@ interface VRDisplay : EventTarget {
   * created without preserveDrawingBuffer set to true will be cleared.
   */
  void submitFrame();
+
+  attribute EventHandler onactivate;


can we just call this mount and unmount? the verbiage is confusing (see #163).

if there's an event for this, IMO there should be a boolean (isMounted) à la isConnected and isPresenting. for example, during page navigation, I wouldn't expect activate to fire, but I may want to know (and can't assume) whether a headset is (still) worn.

cvan · 2017-01-27T01:42:09Z

index.bs

+  attribute EventHandler onactivate;
+  attribute EventHandler ondeactivate;
+  attribute EventHandler onblur;
+  attribute EventHandler onfocus;


Given we have blur and focus, it's odd there's no visibilitychange (especially since we have a presentchange).

FWIW, because the original blur and focus events weren't meaningful enough, browsers introduced the Page Visibility API, which gives you document.visibilityState, document.hidden, document.onvisibilitychange.

cvan · 2017-01-27T01:42:27Z

index.bs

+  attribute EventHandler ondeactivate;
+  attribute EventHandler onblur;
+  attribute EventHandler onfocus;
+  attribute EventHandler onpresentchange;


It seems a bit odd that we have presentchange as opposed to presentbegin and presentend (or presententer and presentexit)? (Also, see comment above.)

This was modeled after fullscreenchange and pointerlockchange, which seemed to me like similar state transitions. I also see that the Page Visibility API include visibilitychange, so I'd say this verbiage is consistent with the rest of the web platform.

I see different tenses and inconsistent patterns of usage and naming (and method signatures/args) with event names, booleans vs. state enums, and property types even with the newer APIs. The Presentation API, Web MIDI, Web USB, and Web Bluetooth in particular all handle input/output a little differently.

Would like to get @annevk, et. al.'s opinions on these, so we can change the API interfaces now to be as closest to the latest best practice today.

It would be a little easier to discuss design in dedicated threads, rather than subthreads of a PR.

What matters here are the use cases and what the typical programming experience would be like. Having said that, exposing state transitions through promises is the latest trend and if that makes sense for your API that'd the best to align on.

cvan · 2017-01-27T01:42:54Z

index.bs

+A user agent MAY dispatch this event type to indicate that something has occured which suggests the {{VRDisplay}} should exit presentation. For example, if the {{VRDisplay}} is capable of detecting when the user has taken it off, this event SHOULD fire when they do so with the reason "unmounted".
+
+<dfn event for="VRDisplay" id="vrdisplay-onblur-event">onblur</dfn>
+A user agent MAY dispatch this event type to indicate that presentation to the {{VRDisplay}} by the page is paused by the user agent, OS, or VR hardware. While a {{VRDisplay}} is blurred it does not lose it's presenting status ({{isPresenting}} continues to report true) but {{getFrameData()}} returns false without updating the provided {{VRFrameData}} and {{getPose()}} returns null. This is to prevent tracking while the user interacts with potentially sensitive UI. For example: A user agent SHOULD blur the presenting application when the user is typing a URL into the browser with a virtual keyboard, otherwise the presenting page may be able to guess the URL the user is entering by tracking their head motions.


will there be a reason in the event detail?

Added a "blur" reason and text indicating it must be used here.

cvan · 2017-01-27T01:42:57Z

index.bs

+A user agent MAY dispatch this event type to indicate that presentation to the {{VRDisplay}} by the page has resumed after being blurred.
+
+<dfn event for="VRDisplay" id="vrdisplay-onpresentchange-event">onpresentchange</dfn>
+A user agent MUST dispatch this event type to indicate that the {{VRDisplay}} has begun or ended VR presentation. This event should not fire on subsequent calls to {{requestPresent()}} after the {{VRDisplay}} has already begun VR presentation.


will there be a reason in the event detail?

Added "presentstart" and "presentend" reasons and text indicating they must be used here.

cvan · 2017-01-27T01:42:59Z

index.bs

+A user agent MAY dispatch this event type to indicate that presentation to the {{VRDisplay}} by the page is paused by the user agent, OS, or VR hardware. While a {{VRDisplay}} is blurred it does not lose it's presenting status ({{isPresenting}} continues to report true) but {{getFrameData()}} returns false without updating the provided {{VRFrameData}} and {{getPose()}} returns null. This is to prevent tracking while the user interacts with potentially sensitive UI. For example: A user agent SHOULD blur the presenting application when the user is typing a URL into the browser with a virtual keyboard, otherwise the presenting page may be able to guess the URL the user is entering by tracking their head motions.
+
+<dfn event for="VRDisplay" id="vrdisplay-onfocus-event">onfocus</dfn>
+A user agent MAY dispatch this event type to indicate that presentation to the {{VRDisplay}} by the page has resumed after being blurred.


will there be a reason in the event detail?

Added a "focus" reason and text indicating it must be used here.

cvan · 2017-01-27T03:13:27Z

index.bs

+interface VR : EventTarget {
+  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getDisplays();
+  readonly attribute FrozenArray&lt;VRDisplay&gt; activeDisplays;
+


I was suggesting adding navigator.vr.getAvailability (as a replacement for the navigator.vrEnabled property, which should obviously be removed).

This would respect the Feature Policy (which supports <iframe enable="vr" src="https://example.com/scene.html"></iframe> – see issue #86).

Besides the aforementioned <iframe> case, there are cases (e.g., browser extension) when a UA may not want to (or cannot attempt to) expose VR capabilities. (A VR display could still be connected, but the page may not be privileged to use it.)

Some similar methods in other web APIs:

new PresentationRequest([urls]).getAvailability().then(…)

navigator.bluetooth.getAvailability().then(…)

navigator.usb.requestDevice({filters: [], optionalServices: [], acceptAllDevices: true}).then(…)

Alternatively, instead of a Promise<boolean> getAvailability(), we could have the following:

readonly attribute VRAvailability availability; attribute EventHandler onavailabilitychange; enum VRAvailability { 'available', 'unavailable', 'unknown' };

SGTM, thanks for the research and suggestion! I'll make the update soon.

cvan · 2017-01-27T03:17:07Z

index.bs

-  readonly attribute FrozenArray&lt;VRDisplay&gt; activeVRDisplays;
+interface VR : EventTarget {
+  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getDisplays();
+  readonly attribute FrozenArray&lt;VRDisplay&gt; activeDisplays;


This doesn't make sense to me. I'm unclear of the use case, and it's ambiguous.

See my proposed navigator.vr.referringDisplay(s) idea: https://gist.github.com/cvan/9c0a2e6e1a6839816128aeb461eb3184#file-sample-usage-js-L21-L32

À la navigator.languages), I suggest having introducing a navigator.vr.preferredDisplays, an array of display name strings for user-preferred VR headsets.

If we think of VR devices/displays à la languages/locales, you can imagine a user flow something like this:

When a user first plugs in a headset, the UA remembers that headset name in some internal persisted list of "known VR displays" in that user's UA profile.

A user opens the UA's "Preferences" to add/remove/rearrange the preferred headsets to use when display WebVR content.

When a requests any page, the browser now starts sending a Accept-VR-Display request header (e.g., Accept-VR-Display: HTC Vive, Oculus Rift) – à la the Accept-Language request header [e.g., Accept-Language: en-US, fr; q=0.7]).

If a page's response header is Content-VR-Display: Gear VR, HTC Vive (or a <meta> tag: <meta http-equiv="content-vr-display" content="available HTC Vive, Oculus Rift"> – [or CSP-Style: <meta http-equiv="Content-VR-Policy" content="default-display 'HTC Vive'; supported-displays 'HTC Vive', 'Oculus Rift'; prefetch-navigation-to 'https://*.neocities.org'">]), then the UA knows which displays are supported by this WebVR page (or scenes within the page, if we're going to get complicated).

The UA now knows that when the page loads, the user may want to enter VR by clicking on some in-content button that calls <VRDisplay>.requestPresent(…) (where <VRDisplay> would be a VR Display that exists in navigator.vr.preferredDisplays and is also supported by the content creator of the WebVR scene).

Here's what I mocked up in Firefox's Preferences, for example:

If while a WebVR source is presenting to a headset and a window navigation occurs, the UA can ought to *fire the navigate event on navigator.vr in addition to pushing the display to navigator.vr.referringDisplays (i.e., attribute FrozenArray<VRDisplay> referringDisplays).

A developer could then check that and immediately request presentation back to the display the user was using before/during the page navigation.

I think allowing sites to say which headsets they support would be as dangerous as letting them say which browsers or OS they support and increase fragmentation

I was giving some thought to this as well, since it's going to be very difficult to get pages to always properly handle multiple headsets. (Even my WebVR.info samples don't, though there's comments in the source alluding to the fact that you should.) I think it's going to be very very very very common to have sited just use display[0].

It seems like the dialog that you mocked up above could be pretty effective if the only thing it did was control the order that headsets are exposed in through getDisplays(). Want to use your Vive instead of your Rift? Make it always show up as display[0]. Sites that do the dumb thing will almost always pick it up, sites that are better behaved will allow you to pick anyway.

(A page can still break it by actively filtering out displays unless displayName.includes('Rift'), but at that point they're just being actively user hostile and I don't think we can help them much.)

cvan

I'm quite happy with these changes. just had a few questions about clarification and the naming of the events (no on…) vs. event target methods (yes on…). very close! great work :)

P.S. I have quite a few other ideas - some of which I commented on/filed now, while the rest of the ideas I've mentally tabled for now (scribbled down in a Gist to articulate later).

cvan · 2017-01-27T23:57:12Z

index.bs

+  attribute EventHandler ondeactivate;
+  attribute EventHandler onblur;
+  attribute EventHandler onfocus;
+  attribute EventHandler onpresentchange;


I see different tenses and inconsistent patterns of usage and naming (and method signatures/args) with event names, booleans vs. state enums, and property types even with the newer APIs. The Presentation API, Web MIDI, Web USB, and Web Bluetooth in particular all handle input/output a little differently.

Would like to get @annevk, et. al.'s opinions on these, so we can change the API interfaces now to be as closest to the latest best practice today.

cvan · 2017-01-27T23:58:45Z

index.bs

+
+The UA MUST provide the following new events. The corresponding events must be of type {{VRDisplayEvent}} and must fire on a {{VRDisplay}} object. Registration for and firing of the events must follow the usual behavior of DOM4 Events.
+
+<dfn event for="VRDisplay" id="vrdisplay-onactivate-event">onactivate</dfn>


these should not have the on; event names never have prefix

since OpenVR uses the activate/deactivate terminology, you can ignore my other comment about mounted/unmounted.

cvan · 2017-01-27T23:59:44Z

index.bs

+A user agent MAY dispatch this event type to indicate that something has occured which suggests the {{VRDisplay}} should be presented to. For example, if the {{VRDisplay}} is capable of detecting when the user has put it on, this event SHOULD fire when they do so with the {{VRDisplayEvent/reason}} "mounted".
+
+<dfn event for="VRDisplay" id="vrdisplay-ondeactivate-event">ondeactivate</dfn>
+A user agent MAY dispatch this event type to indicate that something has occured which suggests the {{VRDisplay}} should exit presentation. For example, if the {{VRDisplay}} is capable of detecting when the user has taken it off, this event SHOULD fire when they do so with the {{VRDisplayEvent/reason}} "unmounted".


typo: "occured" should be "occurred"

cvan · 2017-01-28T00:06:55Z

index.bs

+
+<dfn event for="VRDisplay" id="vrdisplay-onactivate-event">onactivate</dfn>
+A user agent MAY dispatch this event type to indicate that something has occured which suggests the {{VRDisplay}} should be presented to. For example, if the {{VRDisplay}} is capable of detecting when the user has put it on, this event SHOULD fire when they do so with the {{VRDisplayEvent/reason}} "mounted".
+


can we add a update or statechange event that fires when a property on the VRDisplay object changes? see this OpenVR event.

since there's no way to observe an object (anymore), this can be problematic when folks want to know when the property's values have changed but no event otherwise fired.

also see the TrackedDevicePropertiesChanged OpenVR event.

cvan · 2017-01-28T00:09:58Z

index.bs

+
+  attribute EventHandler ondisplayconnect;
+  attribute EventHandler ondisplaydisconnect;
+  attribute EventHandler onnavigate;


I would love to improve the whole Web platform and just properly support a navigate event in the form of @jakearchibald's latest Navigation Transitions proposal. I understand the desire to not cause cross-spec dependencies and being able to nicely confine the WebVR API surface area to this navigator.vr namespace.

How does this event integrate with HTML? Is it properly hooked?

cvan · 2017-01-28T01:17:52Z

index.bs

-  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getVRDisplays();
-  readonly attribute FrozenArray&lt;VRDisplay&gt; activeVRDisplays;
+interface VR : EventTarget {
+  Promise&lt;boolean&gt; getAvailability();


I think we should use SecureContext:

[SecureContext] Promise<boolean> getAvailability();

@cvan Out of curiosity, what is the rational for marking getAvailability with [SecureContext]?

ah, I thought SecureContext is required for Feature Policy stuff. but, it appears – as I should have known – that SecureContext literally means for https: window contexts.

cvan · 2017-01-28T01:18:10Z

index.bs

+  Promise&lt;boolean&gt; getAvailability();
+  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getDisplays();
+
+  attribute EventHandler ondisplayconnect;


I would also [SecureContext]

perhaps we add a displayadd event (à la OpenVR's IServerDriverHost. TrackedDeviceAdded)?

@cvan , Are you suggesting we add displayadd in addition to displayconnect or replace displayconnect with displayadd? In your mind, what is the difference between 'adding' a display and 'connecting' a display?

sorry, replace. I find the displayadd verbiage make a bit more sense.

cvan · 2017-01-28T01:18:54Z

index.bs

+  Promise&lt;sequence&lt;VRDisplay&gt;&gt; getDisplays();
+
+  attribute EventHandler ondisplayconnect;
+  attribute EventHandler ondisplaydisconnect;


I would add onavailabilitychange (imagine this could change based on a UA-presented dialogue via the Permissions API, for example).

see Web Bluetooth's, for measure

cvan · 2017-01-28T01:37:32Z

index.bs

@@ -169,6 +169,12 @@ interface VRDisplay : EventTarget {
   * created without preserveDrawingBuffer set to true will be cleared.
   */
  void submitFrame();
+
+  attribute EventHandler onactivate;


if there's an event for this, IMO there should be a boolean (isMounted) à la isConnected and isPresenting. for example, during page navigation, I wouldn't expect activate to fire, but I may want to know (and can't assume) whether a headset is (still) worn.

cvan · 2017-01-28T01:38:49Z

index.bs

@@ -619,6 +676,19 @@ The user agent MAY request start VR presentation mode. This allows user agents t
 <dfn enum-value for="VRDisplayEventReason">unmounted</dfn>
 The {{VRDisplay}} has detected that the user has taken it off.



should we have navigationstart and/or beforenavigation events (similar to window's beforeunnload and unload events)?

cvan · 2017-02-02T14:25:55Z

@toji: this looks quite good to me. if you can address the last few comments, I'm comfortable with this getting merged. these in particular could use comments, and, if necessary, spec tweaks (or new issues filed):

awesome work. and, thanks for the patience. apologies for derailing the conversation; moving some properties and changing the interfaces involved dropping some and considerable rethinking. doing this in piecemeal is smart, and this is starting to shape up nicely. thanks!

toji · 2017-02-02T18:54:10Z

Addressed several more points of feedback from @cvan, especially around how the events are documented (now has an attribute doc for the on* attributes which points to a separate event) but I'm leaving out any further changes to the events themselves because I'd like that conversation to be in a place that's easier to follow. :)

Going to merge this now, and follow up with creating some separate issues for items that weren't addressed in the initial pull.

DigiTec reviewed Oct 14, 2016

View reviewed changes

AlbertoElias reviewed Oct 14, 2016

View reviewed changes

NellWaliczek reviewed Oct 27, 2016

View reviewed changes

toji mentioned this pull request Oct 30, 2016

Removed VREyeParameters and related refactors #139

Closed

cvan suggested changes Oct 31, 2016

View reviewed changes

RafaelCintron suggested changes Nov 3, 2016

View reviewed changes

toji added this to the 1.2 milestone Nov 14, 2016

cvan mentioned this pull request Nov 15, 2016

Move API methods and events to navigator.vr namespace #152

Closed

cvan mentioned this pull request Dec 4, 2016

vr event type names vs. event attribute names #160

Closed

cvan mentioned this pull request Dec 18, 2016

UA-driven presentation without user gesture. #165

Closed

toji force-pushed the vr_namespace branch from 61d729a to b3d1ed8 Compare January 20, 2017 20:46

toji force-pushed the vr_namespace branch from b3d1ed8 to a4d1892 Compare January 20, 2017 21:12

RafaelCintron approved these changes Jan 23, 2017

View reviewed changes

cvan mentioned this pull request Jan 25, 2017

Use the Feature Policy for provisioning access to VR from embedded content/frames #86

Closed

cvan mentioned this pull request Jan 26, 2017

F2F Followup: Magic Window (and friends) Proposal #179

Closed

cvan suggested changes Jan 27, 2017

View reviewed changes

Addressing some of @cvan's feedback

d55523b

cvan mentioned this pull request Jan 28, 2017

What is the purpose of isConnected and do we need it? #181

Closed

cvan suggested changes Jan 28, 2017

View reviewed changes

A few last tweaks from @cvan's feedback

fdcc3ab

toji force-pushed the vr_namespace branch from 9382e05 to fdcc3ab Compare February 2, 2017 18:50

toji merged commit 35b70e1 into master Feb 2, 2017

toji deleted the vr_namespace branch February 2, 2017 18:55

cvan mentioned this pull request Feb 7, 2017

Introduce navigator.xr.getAvailability method #184

Closed

cwilso modified the milestones: Spec-Complete for 1.0, 1.0 Apr 30, 2019


		User agents implementing this specification MUST provide the following new DOM events. The corresponding events must be of type {{VRDisplayEvent}} and must fire on a {{VRDisplay}} object. Registration for and firing of the events must follow the usual behavior of DOM4 Events.

		<dfn event for="VRDisplay" id="vrdisplayactivate-event">vrdisplayactivate</dfn>


		The UA MUST provide the following new events. The corresponding events must be of type {{VRDisplayEvent}} and must fire on a {{VRDisplay}} object. Registration for and firing of the events must follow the usual behavior of DOM4 Events.

		<dfn event for="VRDisplay" id="vrdisplay-onactivate-event">onactivate</dfn>


		<dfn event for="VRDisplay" id="vrdisplay-onactivate-event">onactivate</dfn>
		A user agent MAY dispatch this event type to indicate that something has occured which suggests the {{VRDisplay}} should be presented to. For example, if the {{VRDisplay}} is capable of detecting when the user has put it on, this event SHOULD fire when they do so with the {{VRDisplayEvent/reason}} "mounted".

		@@ -619,6 +676,19 @@ The user agent MAY request start VR presentation mode. This allows user agents t
		<dfn enum-value for="VRDisplayEventReason">unmounted</dfn>
		The {{VRDisplay}} has detected that the user has taken it off.

Update navigator interface and event targets #116

Update navigator interface and event targets #116

Conversation

toji commented Oct 14, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DigiTec Oct 14, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cvan commented Oct 17, 2016

Detection

Querying for displays

Presenting to displays

Link traversal

AlbertoElias commented Oct 18, 2016

toji commented Oct 18, 2016

cvan commented Oct 19, 2016

toji commented Oct 19, 2016

Choose a reason for hiding this comment

toji Oct 27, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenic commented Nov 1, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MortimerGoro commented Nov 15, 2016 • edited

toji commented Jan 20, 2017

cvan commented Jan 20, 2017

toji commented Jan 21, 2017

RafaelCintron commented Jan 23, 2017

cvan commented Jan 25, 2017

toji commented Jan 26, 2017

cvan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toji Jan 27, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toji commented Oct 14, 2016 •

edited

DigiTec Oct 14, 2016 •

edited

toji Oct 27, 2016 •

edited

MortimerGoro commented Nov 15, 2016 •

edited

toji Jan 27, 2017 •

edited