Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not consistent depth colormap between nerfacto and depth-nerfacto #1388

Open
ManuConcepBrito opened this issue Feb 9, 2023 · 2 comments
Open

Comments

@ManuConcepBrito
Copy link
Contributor

Describe the bug
I was comparing the results from depth-nerfacto and nerfacto in tensorboard, but I realized there is not a consistent way of applying the colormaps to the output depths. In depth_nerfacto.py:

predicted_depth_colormap = colormaps.apply_depth_colormap( outputs["depth"], accumulation=outputs["accumulation"], near_plane=torch.min(ground_truth_depth), far_plane=torch.max(ground_truth_depth), )
And in nerfacto:

depth = colormaps.apply_depth_colormap( outputs["depth"], accumulation=outputs["accumulation"], )
If a value is not provided to apply_depth_colormap, the max and min values of the depth map are taken which results in squashing the histogram like in the following image (left is depth nerfacto and right is nerfacto):

imagen

Also, by clipping the colormap of depth nerfacto to the largest value, it prevents from showing further details

By setting something a fixed value for both nerfacto and depth-nerfacto for the far plane (5 meters) like:

depth = colormaps.apply_depth_colormap( outputs["depth"], accumulation=outputs["accumulation"], near_plane=0.0, far_plane=5.0, )

We get much more coherent results (left is depth nerfacto and right is nerfacto):

imagen

However, there should be a better or more flexible solution.

@machenmusik
Copy link
Contributor

My first thought was sliders for near and far, with defaults, but I notice there are already sliders for range in there; not sure how they should interop...

@machenmusik
Copy link
Contributor

It looks like if you change the depth colormap from default to, say, turbo, you can then manipulate the range sliders; range seems to default to [0, 1] but the sliders let one set it to [-2, 5].

Maybe you can try setting the sliders to [0, 5] and see if that is the same as your near/far plane settings in code? Because if so, we should probably change the default range to NOT be [0, 1].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants