good job！ I have some problem about Depth Map Bits

[DD20-dai]

Hello!
Could you please tell me what is the unit of the depth map? I want to import the depth map of my own picture, but it seems that the depth image's units do not match so the results of warp are different.

[NagabhushanSN95]

Hi,
Thank you for your appreciation. The unit of depth map should be the same as the unit of translation vector of your camera poses. Hope that helps.

[DD20-dai]

Thank you！ Sorry to bother you.
I am still curious about this. Could you please tell me how to obtain the example's depth(depth1.npy) you have given, or how can I convert my depth map into the unit of the depth map given in your example.
I'm sorry that I don't know much about the information of depth map. The value of my depth map ranges from 570 to 5991. The example you gave is from 2.0033264 to 6.292195, so I wonder if there is any conversion method? such as normalization?
Thank you most sincerely.

[NagabhushanSN95]

1. The example scene was rendered in blender. Along with the RGB frames, blender also provides the camera poses and the depth maps.
2. Yes, depth maps can have any arbitrary range.
3. To help you further, I need to know how you obtained the RGB frames, camera poses and the depth map. Also, in what context are you trying this.

[DD20-dai]

I'm trying to do few shot image to panorama generation with depth map data augmentation. So I try to warp under this dataset to get my training data. According to the paper(n-Place Scene Labelling and Understanding with Implicit Scene Representation) of the dataset,

Habitat simulator(Habitat: A platform for embodied ai research.).
He uses Ai habitat for rendering to get depth maps and pictures.

Here is an example containing image and depth of the datasets.
Thanks for your reply!

[DD20-dai]

[NagabhushanSN95]

Hi,

Sorry for the delayed response. For this, you need to go through the habitat documentation on how depth maps are saved.

For example, we used Scene-Net, and for this dataset, we had to multiply the depth maps provided by 0.001 to bring them to the same scale as translation vectors. Let me reiterate - the scale of the depth map is not dependent on the code I've provided. It should be the same scale as the translation vector. This will be defined by the dataset. In other words, if you scale the translation vector and the depth maps by the same factor, the output should not change at all.

Ideally, if the depth map is saved without further post-processing as a raw data file (npy for example), you wouldn't need to scale them anymore. If some post-processing is done to save the depth map in a specific format (png for example in scenenet), then you will need to undo the post-processing steps. This will be defined in the dataset. If you're completely in the dark and have no idea what to do, I would suggest the following.

1. Try multiplying the depth map by 1,0.1,0.01,0.001, 10, 100, 1000, and so on. Check if, for any of the depth maps, the warped image makes sense. If so, then try to tweak the scale around the value you have converged to. You might be able to find the actual scale. However, this works if the post-processing done is a simple scaling, as is done in Scene-Net.
2. Run colmap sparse reconstruction. This will give you depth values at sparse pixels in your input images. Compare these values to the depth map from the dataset. You might be able to figure out the scaling or transformation that is needed.

However, if the depth map has some unique post-processing as done on the NeRF-Synthetic dataset, it is very hard to figure out by yourself unless the database owner provides you with the information.

[DD20-dai]

Thank you for such a detailed answer.
I tried 0.001 and it seems that there should be no problem with the depth, but it seems that the world coordinate system is not consistent, because the direction of movement seems to be inconsistent. I will first go to the relevant usage code of the data set author to see its related processing operations of external parameters.
If I have other questions later, can I ask you again?
Thank you again for such a detailed answer, this is the most patient answer I have ever seen!

[NagabhushanSN95]

Sure. Good luck!