How to get nice lego geometry?

[YueChenGithub]

Thank you first for the great work.
I tried the model on NeRFactor dataset (512x512 with low res. env. map). The results of the Lego scene make me confused.

Result in the paper:

Result of the standard Lego config, the result is blur and has an unexcepted 'bottom' part:

Result of the 'tricky' Lego config (enable the those three lines), the result looks better but still it has 1. tricky normal direction in some part, 2. noisy normal for the flat surface and 3. not correct surface reconstruction inside the digging bucket:

As can be seen, training with either standard lego config or the tricky one can not produce results as good as in the paper. One reason is that I trained it with a smaller image size (512x512, in the paper 800x800). Do you have any suggestions for getting a better geometry?

[Haian-Jin]

Thank you for your interest in this work.

I apologize for the delay in my response, as I have been busy with my recent NeurIPS submission.

I noticed that I uploaded an incorrect version of the config file for single/lego.txt. The incorrect version had a very high albedo and roughness smoothness weight compared to the other config files. I have now uploaded the correct version.

If you run the correct config file on TensoIR-synthetic data, you should get the expected results, as shown in the image below:

However, when I tried the same config file on NeRFactor's data, I encountered the same phenomenon that you described, which is unexpected because NeRFactor's data was generated using a low-resolution environment map and should be easier to handle than my data, which was rendered using a high-resolution environmental map.

Nevertheless, you can still obtain a high-quality geometry reconstruction of the lego on NeRFactor's data by using the "tricky lego config" as you have done before (uncomment the three lines). If you do so and run the config file on NeRFactor's data, you should obtain the following results:
(Please ensure that you run all iteration to completion if you use the tricky lego config file)

I suspect that the unexpected bottom phenomenon may be caused by the normal prediction network having a negative effect on the density field. I found that if I use a smaller normals diff weight, I do not encounter the unexpected bottom phenomenon on the Lego dataset. However, in this case, the normals prediction may overfit on RGB supervision.

In summary, you can now obtain high-quality lego geometry on NeRFactor's dataset by downloading the new and correct version of the single/lego.txt config file and enabling the three lines you mentioned. I will endeavor to fix the problem of the unexpected bottom phenomenon when not enabling the three lines in the near future.

If you have any insights or suggestions related to this issue, please feel free to share them with me.

[YueChenGithub]

Thank you for your detailed answer! Good luck with your NeurIPS publication.