abnormal near_depth in render result

[ds22058]

Hi, author. Thank you for your great work first. I try to render horns' pictures by using the pretrained-model on generalization dataset. But when I run the render.py with parameters "--config configs/gnt_llff.txt --eval_dataset llff_render --eval_scenes horns --expname gnt_horns --chunk_size 500 --N_samples 192", it return "ValueError: loaded state dict contains a parameter group that doesn't match the size of optimizer's group"

Traceback (most recent call last):
File "/home/zheshi/Documents/GNT/render.py", line 191, in
render(args)
File "/home/zheshi/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, **kwargs)
File "/home/zheshi/Documents/GNT/render.py", line 66, in render
model = GNTModel(
File "/home/zheshi/Documents/GNT/gnt/model.py", line 76, in init
self.start_step = self.load_from_ckpt(
File "/home/zheshi/Documents/GNT/gnt/model.py", line 161, in load_from_ckpt
self.load_model(fpath, load_opt, load_scheduler)
File "/home/zheshi/Documents/GNT/gnt/model.py", line 127, in load_model
self.optimizer.load_state_dict(to_load["optimizer"])
File "/home/zheshi/.local/lib/python3.8/site-packages/torch/optim/optimizer.py", line 146, in load_state_dict
raise ValueError("loaded state dict contains a parameter group "
ValueError: loaded state dict contains a parameter group that doesn't match the size of optimizer's group

Can you help me with this problem? Thanks in advance.

[MukundVarmaT]

Hi @ds22058, thank you for your interest in our work! and apologies for the delay in our response.

The released checkpoints were trained from a private codebase, which I cleaned for public release. During the cleaning, I renamed a layer for better readability but this led to a naming mismatch with the checkpoints. I have fixed this and can confirm that it works from my end. Please let me know in case you run into further trouble!

Thanks

[ds22058]

Thank you for your reply and I have already solved this issue

[ds22058]

Hi, I met another trouble when I try to render on my own dataset with angle ranged from 0 to 180. It could output result but the depth was abnormal when it came to the angle near 90.

Just like it was magnified when angle came to 90 and when it went close to 0 and 180 it became normal. I modified the dataset's angel from 0 to 60 and it worked.

I modified the 'depth_range' in the llff_render.py and the coefficient is 'depth_range[0] * 1.8, depth_range[1] * 1.0', the result's depth seemed to be normal but the render quality became worse.

I tried to retrained the model with the modified coefficient in depth_range but didn't work. I really want to know how can I solve this issue. Thanks in advance!

[MukundVarmaT]

Apologies for the (super) delayed response. Yes, I have also observed that GNT seems to be sensitive to the depth range and can't especially handle scenes where the ranges are beyond whats present in the pre-training dataset (for example in your case an outdoor scene). However, setting it to a fixed range can help render reasonable results.

It is surprising retraining didn't help. Perhaps I can comment better if you could share your dataset?