Passthrough not working on Godot 4 OpenXR with a Varjo XR3 Headset

[hitmuri]

Godot version

4.1

System information

Windows 10 - Godot 4.1.1 - Compatibility

Issue description

When trying to use the Varjo XR3 headset (in OpenXR mode) with Godot 4 (with OpenXR and passthrough activated), the passthrough does not work, i.e. the scene is rendered on a black background instead of the camera stream.

It seems to me that are two problems in the openxr module, more precisely in the openx_api.cpp file (fixing them solves the issue) :

1. The XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT is only set when the number of layers is > 1, which is fine for the Meta Quest headset but does not work for the Varjo since it only requires one layer.
2. The environment blend mode XR_ENVIRONMENT_BLEND_MODE_ALPHA_BLEND is only really set when the XRFbPassthrough extension is present, which is not the case with the Varjo runtime.

I am not sure how to fix this without causing issues for other headsets.

1. Would it be an issue to always activate the XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT ?
2. Could blend modes be activated even if they are not supported by an extension ?

Steps to reproduce

• Create a Godot 4 project with OpenXR activated in Compatibility mode
• Add the code to activate OpenXr and to enable passthrough : https://docs.godotengine.org/en/stable/tutorials/xr/openxr_passthrough.html#passthrough-extension
• Start the Varjo Base software and activate OpenXR
• Run the Godot project

Minimal reproduction project

N/A

[BastiaanOlij]

Hi @hitmuri ,

I'm missing something, enabling the passthrough system (as in enabling Metas passthrough extension) set the blend mode and indeed adds the extra layer, which is just one of the two options.

If all that is needed is to set the blend mode, than don't enable the passthrough extension, instead just set the blend mode through code (xr_interface.set_environment_blend_mode(xr_interface.XR_ENV_BLEND_MODE_ALPHA_BLEND)).
I planned adding that to the project settings as well but looks like I forgot about that.

So it's basically following https://docs.godotengine.org/en/stable/tutorials/xr/openxr_passthrough.html#passthrough-through-ar

We use this for ML2 and Lynx R1

[hitmuri]

Hi @BastiaanOlij ,

Thanks for the answer.

After checking again, it seems that the blend mode is correctly selected but that the issue comes from the layers_list.size()>0 test in the projection layer (in openxr_api.cpp), which seems to be false for the Varjo, which means that the XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT is not set.

Just removing the test fixes the passthrough rendering.

[BastiaanOlij]

Ah, that looks like it's not respecting that setting, looks like two bits of functionality have crossed paths.

I want to make sure the settings becomes a project setting as well so you can change the default, and I'll polish that up as well.

[hitmuri]

Great, thank you for that, and thanks for all the work on XR in Godot by the way.