How does Depth Lab provide 0-65m depth values?

[valantano]

I noticed that the Depth Lab uses AR Foundation v.4 but provides depth estimation for 0-65m, although AR Foundation v4 does not support this.
Currently, I am working with AR Foundation v.5 since I need to work with the whole 0-65m depth estimation, but I also need to work with the ARCore Extensions, which are only available for AR Foundation v.4.
When I try to use AR Foundation v.4, I cannot access the 0-65m depth estimation but only the 0-8m depth estimation since everything above 8m is set to 0. However, when using the Depth Lab app and activating the Depth Map, it seems that Depth Lab is able to detect the depth even for higher values than 8m.
I looked into https://github.com/googlesamples/arcore-depth-lab/blob/22cd7f1ce4eb2ed73bda19a0ea1bf3e636831799/Assets/ARRealismDemos/Common/Scripts/DepthSource.cs, but I don't get it why Depth Lab is able to provide the 65m depth values.

It would be great if someone could help me with that. Maybe someone can explain where the values greater than 8m are set to 0 and how Depth Lab bypasses this. And perhaps also how I can do this as well.

I really appreciate any help you can provide.

[ruofeidu]

Hi valantano,

I updated DepthLab legacy SDK (deprecated) to 30m depth sensing around Google I/O. We have an engineer updating DepthLab AR Foundation SDK but it's not complete yet. The paper and the Play Store app were built with the legacy SDK.
If you can get normalized depth from 0 to 1 without the 8 meter clipping, would you like to try just converting it to 0-30m?

I have an updated shader here:

https://github.com/googlesamples/arcore-depth-lab/blob/arcore_unity_sdk/Assets/ARRealismDemos/DepthEffects/Shaders/BackgroundToDepthMapCore.cginc#L48

and a new fine-tuned colormap :) : https://github.com/googlesamples/arcore-depth-lab/blob/arcore_unity_sdk/Assets/ARRealismDemos/Common/Textures/DepthColorRamp.png

Thanks,
Ruofei

[valantano]

Hi Ruofei,

thank you for the fast response.
Sadly, as soon as I know, AR Foundation only provides the Depth in meters in a texture. And there they have already set all values above 8m to 0. At least in AR Foundation v.4.
It seems that I will need to use AR Foundation v.5. But your explanation helped me to understand the reason why you guys are able to provide the 30m in Depth Lab :)

Thanks,
Valentino

[ruofeidu]

Let me reopen this and ask relevant engineers tomorrow! Thanks for letting me know!

[ruofeidu]

Hi Valentino,

Please check out https://docs.unity.cn/Packages/com.unity.xr.arcore@4.1/changelog/CHANGELOG.html and from 4.1.0-preview.13, it switches to using all 16-bits from the pixels in the ARCore environment depth image, rather just the lower 13 bits :)

Cheers,
Ruofei