Encountering the issue when run this code "CUDA_VISIBLE_DEVICES=0 python run_mv.py --config configs/blender.txt --scene lego --eval_only --grad_normal"

[WuyiWong]

hi, when I follow you tip to evaluate the ray-surface distance network through this code "CUDA_VISIBLE_DEVICES=0 python run_mv.py --config configs/blender.txt --scene lego --eval_only --grad_normal". I encounter this error:
//
Traceback (most recent call last):
File "run_mv.py", line 405, in
evaluate(args)
File "run_mv.py", line 331, in evaluate
depth = convert_d(vis_results['dist_abs'].squeeze().cpu().numpy(), dataloader.scene_info, out='dep')
KeyError: 'dist_abs'
//

why could this thing happen?

[zhuomanliu]

vis_results['dist_abs'] is stored when using --denoise in the previous code. The code has been updated to fix this issue when using only --grad_normal.

[WuyiWong]

thanks for your answer. But it seems that there is another issue I've encountered when I run this code "CUDA_VISIBLE_DEVICES=0 python run_mv.py --config configs/blender.txt --scene lego --eval_only" to evaluate the distance network.

[train] ADE=1.1995 CD=955.4302 CD_median=827.3363 PSNR=nan SSIM=nan LPIPS=nan
[test] ADE=1.1883 CD=949.7096 CD_median=835.1572 PSNR=nan SSIM=nan LPIPS=nan

why are the the values of CD, CD_median, PSNR, SSIM and LPIPS so strange, I wonder if it has anything to do with "scene lego"?

[zhuomanliu]

thanks for your answer. But it seems that there is another issue I've encountered when I run this code "CUDA_VISIBLE_DEVICES=0 python run_mv.py --config configs/blender.txt --scene lego --eval_only" to evaluate the distance network.

[train] ADE=1.1995 CD=955.4302 CD_median=827.3363 PSNR=nan SSIM=nan LPIPS=nan [test] ADE=1.1883 CD=949.7096 CD_median=835.1572 PSNR=nan SSIM=nan LPIPS=nan

why are the the values of CD, CD_median, PSNR, SSIM and LPIPS so strange, I wonder if it has anything to do with "scene lego"?

Have you loaded the pre-trained model during evaluation? The print log should be like this: 'Loading ckpt from: logs/multiview/...'

[WuyiWong]

Hi,

Thanks for your patience.
It seems like the pretrained model loaded failed in my code, as shown below.

But I followed you evaluation steps and downloaded the checkpoints from the link you've shared. Is my step wrong?

[zhuomanliu]

Hi,

Thanks for your patience. It seems like the pretrained model loaded failed in my code, as shown below.

But I followed you evaluation steps and downloaded the checkpoints from the link you've shared. Is my step wrong?

How about the folder name you downloaded? It should be 'blender-lego_mv20d13w1024rgb0w1.0_lr1e-05bs8192iters80k' for Lego (geometry only). To be more specific, you can run this: "CUDA_VISIBLE_DEVICES=0 python run_mv.py --config configs/blender.txt --scene lego --eval_only --grad_normal --expname mv20d13w1024rgb0w1.0_lr1e-05bs8192iters80k" to make sure the loaded folder name is correct.

[WuyiWong]

Hi, I've found the issue of my implementation. During adjustment, I've finally loaded the pre-trained model during evaluation. Thank your so much. The print log be like this: 'Loading ckpt from: logs/multiview/...'

But, there are still some issue in it. the value of PSNR and SSIM is really weird, is it normal? as shown below.

[zhuomanliu]

Hi, I've found the issue of my implementation. During adjustment, I've finally loaded the pre-trained model during evaluation. Thank your so much. The print log be like this: 'Loading ckpt from: logs/multiview/...'

But, there are still some issue in it. the value of PSNR and SSIM is really weird, is it normal? as shown below.

It is because this is the geometry-only mode (experiment Group 1 in the paper).
For geometry+radiance mode (experiment Group 2 in the paper), adding --rgb_layer 2 is required, and then the code will load the checkpoint from the folder "scannet-scene0004_00_mv20d13w1024rgb2w1.0_lr1e-05bs8192iters80k".

[WuyiWong]

I see, thank you again for your patient explanation.

Best regards.