-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to read the depth map? #77
Comments
I believe you need to multiply it by the far bound (6 for the blender dataset): |
We never used the depth data in the project so I can't confirm that it is correct. These files were mistakenly included in the dataset. We decided not to delete them just in case someone found them useful. |
@markomih thanks! It works :) |
Hello! I meet the same problem. May I ask for the script for checking the consistency of depth map in multi-view? |
Hi @LangHiKi, unfortunately I think I didn't save the exact code I used to verify. But what I did is as follows:
PS: Also, when I was trying, there was some shift between the warped and the true images. I didn't think much about it at that time since the pose alignment was correct. But now that I think about it, there should be some residual scale factor when converting the depth. It should be easy to find the scale factor by trial-and-error. |
@NagabhushanSN95 Thank you for your reply! |
If you used a code similar to mine, camera pose has to be in world2camera format. I think (don't remember for sure), the dataset poses are in camera2world format. So, you might have to take the inverse of poses before any processing. |
Thanks for the hints! However, I found that the transformation provided by @markomih didn't actually work. I tried some variants and found that the depth should be for idx in [1,10,100]:
rays_o, rays_d = get_rays(H, W, K, torch.Tensor(poses[idx])) # (H, W, 3), (H, W, 3)
norm = rays_d.norm(p=2,dim=2,keepdim=True)
depth_im = cv2.imread(os.path.join('data/nerf_syn/ship/test', 'r_{}_depth_0002.png'.format(idx))).astype(float)[:,:,0]
transformed = 8*(255-depth_im)/255
depth_im[depth_im!=0] = transformed[depth_im!=0]
points = rays_o + rays_d * depth_im[:,:,None] / norm
trimesh.Trimesh(points.reshape(-1,3)[::2]).export('points{}.ply'.format(idx)) It can be seen the pointclouds from 3 views(green, red, yellow) are accurately fused together. TSDF fusion based on this depth map further demonstrates the correctness. |
I read the depth map in synthetic test file by d[u][v] = (255-depth[u][v])/255, but the final results seems not consistent in multi-view. So how can I get the accurate depth map?
The text was updated successfully, but these errors were encountered: