The Hotdog result is terriable？

[yanghbcodes]

Hi， your job is brilliant and i'm very interested, i follow your description, run the code on the nerf_synthetic hotdog data, but
the result I got was terriable, the psnr in your paper is 25.250, but i just got 8.37. and when i train the model, I found the val images were always splited.

Is there any trick that you dont mentioned when train the model? Thanks!

[ajayjain]

Hi Haibo,

Are you using the hotdog command in https://github.com/ajayjain/DietNeRF/blob/master/dietnerf/scripts/run_synthetic_dietnerf_8v.sh? I just tried on a fresh install of the conda environment (torch 1.10.1) as well as my old environment (torch 1.7.1), but have similar results for both without the object splitting. Note this is very early in training, so the model hasn't converged yet.

In your email, you mentioned that the semantic consistency loss is negative - it is supposed to be negative since it's negative cosine similarity, which is minimized. The total loss can be volatile since the semantic loss is added every 10th iteration.

You could try rerunning with another seed, or with these package versions:

absl-py==1.0.0
attrs==21.4.0
beautifulsoup4==4.10.0
certifi==2021.10.8
charset-normalizer==2.0.10
click==8.0.3
-e git+ssh://git@github.com/ajayjain/DietNeRF.git@ae5d7b4b314d9e4ba66f1fd422d1265f187e4182#egg=clip&subdirectory=CLIP
cloudpickle==2.0.0
ConfigArgParse==1.5.3
configparser==5.2.0
cycler==0.11.0
Deprecated==1.2.13
docker-pycreds==0.4.0
filelock==3.4.2
fire==0.4.0
fonttools==4.28.5
ftfy==6.0.3
gdown==4.2.0
gitdb==4.0.9
GitPython==3.1.26
grpcio==1.43.0
idna==3.3
imageio==2.13.5
imageio-ffmpeg==0.4.5
importlib-metadata==4.10.1
jsonschema==4.4.0
kiwisolver==1.3.2
lpips==0.1.4
Markdown==3.3.6
matplotlib==3.5.1
msgpack==1.0.3
networkx==2.6.3
numpy==1.22.1
opencv-python==4.5.5.62
packaging==21.3
pathtools==0.1.2
Pillow==9.0.0
promise==2.3
protobuf==3.19.3
psutil==5.9.0
pyparsing==3.0.6
pyrsistent==0.18.1
PySocks==1.7.1
python-dateutil==2.8.2
PyWavelets==1.2.0
PyYAML==6.0
ray==1.9.2
redis==4.1.1
regex==2021.11.10
requests==2.27.1
scikit-image==0.17.2
scipy==1.7.3
sentry-sdk==1.5.2
shortuuid==1.0.8
six==1.16.0
smmap==5.0.0
soupsieve==2.3.1
subprocess32==3.5.4
tensorboard==1.14.0
termcolor==1.1.0
tifffile==2021.11.2
timm==0.5.4
torch==1.10.1
torchvision==0.11.2
tqdm==4.62.3
typing_extensions==4.0.1
urllib3==1.26.8
wandb==0.12.9
wcwidth==0.2.5
Werkzeug==2.0.2
wrapt==1.13.3
yaspin==2.1.0
zipp==3.7.0