Unity export doesn't load

[jdm]

https://de-panther.github.io/unity-webxr-export/Build/

Possible cause:

So it seems I can't iterate on xrSession.inputSources on firefox reality for Hololens 2.

godot:servo jdm$ ./mach run -d https://de-panther.github.io/unity-webxr-export/Build/
[UnityCache] indexedDB database could not be opened
Warning: Unsupported graphics API WebGL 2.0
[UnityCache] 'https://de-panther.github.io/unity-webxr-export/Build/Build/Build.wasm.framework.unityweb' successfully downloaded but not stored in the indexedDB cache due to the error: Error: indexedDB access denied
[CachedXMLHttpRequest] indexedDB is not available
IndexedDB is not available. Data will not persist in cache and PlayerPrefs will not be saved.
Assertion failed: IDBFS used, but indexedDB not supported
Assertion failed: IDBFS used, but indexedDB not supported
[UnityCache] 'https://de-panther.github.io/unity-webxr-export/Build/Build/Build.data.unityweb' successfully downloaded but not stored in the indexedDB cache due to the error: Error: indexedDB access denied
Loading player data from data.unity3d

Initialize engine version: 2019.3.0f6 (27ab2135bccf)

Creating WebGL 1.0 context.
Renderer: Mozilla/Servo

Vendor:   Mozilla/Servo

Version:  OpenGL ES 2.0 (WebGL 1.0)

GLES:     2

 EXT_shader_texture_lod GL_EXT_shader_texture_lod OES_texture_half_float GL_OES_texture_half_float ANGLE_instanced_arrays GL_ANGLE_instanced_arrays OES_element_index_uint GL_OES_element_index_uint OES_standard_derivatives GL_OES_standard_derivatives OES_texture_float_linear GL_OES_texture_float_linear EXT_frag_depth GL_EXT_frag_depth EXT_color_buffer_half_float GL_EXT_color_buffer_half_float WEBGL_color_buffer_float GL_WEBGL_color_buffer_float EXT_texture_filter_anisotropic GL_EXT_texture_filter_anisotropic WEBGL_compressed_texture_s3tc GL_WEBGL_compressed_texture_s3tc OES_texture_float GL_OES_texture_float OES_texture_half_float_linear GL_OES_texture_half_float_linear OES_vertex_array_object GL_OES_vertex_array_object EXT_blend_minmax GL_EXT_blend_minmax

OPENGL LOG: Creating OpenGL ES 2.0 graphics device ; Context level  <OpenGL ES 2.0> ; Context handle 18005728

UnloadTime: 1.540000 ms

Active Graphics Tier: Tier2
(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)


Input Manager initialize...

[De-Panther]

Hi,
for now I replaced
for (let inputSource of inputSources) {
with

    if (!inputSources || !inputSources.length) {
      return vrGamepads;
    }
    for (var i = 0; i < inputSources.length; i++) {
      let inputSource = inputSources[i];

Before that I got an error "inputSources is not iterable".

Now I don't get an error, but it doesn't recognize the hands as controllers.

I uploaded the page with the error to this URL:
https://de-panther.github.io/unity-webxr-export/Build/index_input_bug.html

I don't have my own device, I ask people to try it, so I can't debug it on my own.

Thanks for the help.

[jdm]

Thanks for the testcase! We'll try and figure out what's going wrong.

[De-Panther]

One of my guesses is that there's one frame with no inputSources.

And I think that we can't see the "controllers" because that I check for gamepad.
https://github.com/De-Panther/unity-webxr-export/blob/master/Build/webxr.js#L264

It works on the Oculus Quest Browser with hands, as they emulate the controllers.
So I guess that I need to figure out what XRInputSource.profiles are being used in Hololens.

I added a page with console log, to test how many InputSources there are on a frame and if there's XRInputSource.gripSpace XRInputSource.gamepad or XRInputSource.targetRaySpace.
https://de-panther.github.io/unity-webxr-export/Build/index_log_hololens.html

[jdm]

The problem is actually that this line is silently ignored for XRInputSourceArray, so it is in fact not iterable. I've got some code changes that should fix that and an automated test that verifies it.

[De-Panther]

Thanks.

So after this fix the InputSources in the specific page will work?
Or they still won't show, as it try to get the XRInputSource.gripSpace and XRInputSource.gamepad?
Is there a place where I can find info about the inputSources used for Hololens? as there's no profile for that in https://github.com/immersive-web/webxr-input-profiles

[jdm]

@Manishearth please see the questions in the previous comment.

[Manishearth]

@De-Panther We do not expose a gamepad, as there are no buttons. You can use onselect to get a select event. The profile we use is "generic-hand", which is being discussed in immersive-web/webxr-input-profiles#105

gripSpace and targetRaySpace will work.

[De-Panther]

Thanks, I'm currently reading that and other related issues.
Also found that one:
#26092

I do prefer the way that the Oculus Quest Browser works, with the emulated gamepad with one trigger.

I tried to avoid onselect, as I wanted to use it only for touchpad events (handheld AR or cardboard VR).

Is it right to assume that:
onselect - will work with any device that supports hands, and is the equivalent for the trigger button?
onsqueeze - won't work with all of those(hands supported) devices, may work in some, and instead I can check if gamepad exist?

[Manishearth]

onselect is spec-wise guaranteed to be emitted when the trigger button is pressed, but it can theoretically be emitted when other buttons are pressed, up to the UA choice (it is allowed to designate multiple select buttons). No device/UA does this as of now, and I don't expect that to change

Squeeze won't work for hands right now because for the hololens (and Quest iirc) there is overlap between select and squeeze. See immersive-web/webxr#997 for more info, input from a user's perspective is appreciated!

I'm not super comfortable exposing a fake button just yet because the resolution of immersive-web/webxr#997 may involve exposing select and squeeze as buttons through a specially named profile.

[Manishearth]

Remember, .gamepads is a separate module, browsers may decide to implement WebXR without implementing gamepads.

[De-Panther]

Thanks.

I think that I'll check onselect, and if in the inputSource it has gamepad, I'll check it's button value for cases of analog buttons.
That will help in a more generic approach:
If hands but not gamepad - it'll work.
If hands and gamepad - it might have analog value.
If no hands and no gamepad(e.g. cardboard) - It'll work.
If some generic controller - it'll work, and it'll have analog value if it supports it.

[Manishearth]

@De-Panther (re: your comment on the webxr issue) you misunderstand, the issue I linked is not about "some browsers do not implement the gamepad module". It's the reason I haven't added the gamepad module to servo yet, because there's a chance it may be resolved in a way that affects how we do it.

The core issue issue is that for physical controllers "select" and "squeeze" are independent events, and you can treat them as separate buttons that the user may press separately. However, on hand tracking devices, select and squeeze can happen simultaneously in resposne to the same gesture, and in fact most select events trigger squeeze events. Content relying on this to not happen breaks.

[De-Panther]

The reason I commented it there, is that I thought that my question is relevant to that discussion.

"Why won't XRInputSource have trigger and squeeze state regardless of gamepad available or not?
Or if an XRInputSource does have trigger or squeeze events, it should have gamepad data, even if it's not an actual gamepad? (e.g. in the case of hands)"

The part that may not related, is the fact that you don't have the data about trigger down/up or squeeze down/up. But this is why I asked about adding a gamepad object in case of hands, regarding of having actual gamepad.

Working with events is great and simple in most cases, but when I anyway need to iterate on the InputSources, I want to have the data about the trigger/squeeze state there, rather than getting it from another place.
It's (events+inputSource) usable, but it's not straight forward to implement.

[Manishearth]

@De-Panther Neither of your questions make sense in the context of the WebXR specification, however. The WebXR specification does not mandate hands to not expose gamepads, that's a choice we've made. The point of the select/squeeze events is precisely to expose trigger and squeeze state regardless of the availability of the gamepad, and if you want them in button form that's what the gamepad spec is for.

I want to have the data about the trigger/squeeze state there, rather than getting it from another place

It's straightforward enough to handle the events and cache a boolean on the input source yourself.

[De-Panther]

IMO,
Straightforward:
Iterating each frame on an object and child objects to get Pose and input events states.

Usable:
Registering to input events,
Cache boolean states of those events,
Iterating each frame on an object and child objects to get Pose and compare with the cached input events states.

It's usable, and this is what I'll end up doing. But it's not straightforward.

Having the Hand input events in the inputSource (as gamepad), would be the straightforward solution IMO. And as I understand, this is the direction that the discussion there is going to. So it's a good thing and I encourage it.

[Manishearth]

Iterating each frame on an object and child objects to get Pose and compare with the cached input events states.

Just cache on inputSource.selected and check that value instead of inputSource.gamepads.button[0].

And as I understand, this is the direction that the discussion there is going to

Not quite, that's the issue. We can do this for select , but doing this for squeeze will break content that expects the two to not overlap. What might end up happening is that squeeze will be exposed as a different gamepad button via a hands-overlappingsqueeze profile.

Exposing select on the gamepad is not controversial. It's squeeze that's a problem.

[De-Panther]

Just cache on inputSource.selected and check that value instead of inputSource.gamepads.button[0].

But I do want to get the analog value of the trigger/squeeze buttons if there is one.

Not quite, that's the issue. We can do this for select , but doing this for squeeze will break content that expects the two to not overlap. What might end up happening is that squeeze will be exposed as a different gamepad button via a hands-overlappingsqueeze profile.

Exposing select on the gamepad is not controversial. It's squeeze that's a problem.

So developers would need to check both squeeze button and overlappingsqueeze button?
Or it's the same button, but with a profile named hands-overlappingsqueeze in the profiles list?

[Manishearth]

But I do want to get the analog value of the trigger/squeeze buttons if there is one.

Right, read from gamepad if it exists, or read from selected if it doesn't. Hand input doesn't expose an analog value.

[De-Panther]

And this is why I think that it's not so straightforward.

Also, does the onselect event is called before the rAF iteration is called? If so, adding inputSource.selected will work, but if it may happen after, there may be some issues.

[Manishearth]

And this is why I think that it's not so straightforward.

it's a couple lines of code:

if (input.gamepad) {
  selected = input.gamepad.buttons[0];
} else {
  selected = input.selected;
}

Ultimately it is not the goal of web specifications to do additional stuff that can be handled by frameworks. From the WebXR side, this is code that people can write, and ideally user agents implement the gamepad API, the problem is solved. Servo doesn't do the latter but you can still do the former.

Also, does the onselect event is called before the rAF iteration is called? If so, adding inputSource.selected will work, but if it may happen after, there may be some issues.

It doesn't matter, if it happens after it will be exposed on the next frame. Frames occur quite rapidly after each other. When you are accessing .gamepads.button you have this exact same issue as well, if a button was pressed shortly after a frame was kicked off, it will show up on the next frame.

Servo does indeed trigger it before the rAF, though; we only check for button presses right before rAF.

[De-Panther]

Thanks :)

[De-Panther]

Managed to run it on the Hololens emulator using the version from the store.
The hands works now :)
Thanks again