Skip to content

Latest commit

 

History

History
409 lines (378 loc) · 9.83 KB

benchmark.md

File metadata and controls

409 lines (378 loc) · 9.83 KB
Raw

Benchmark

We compare our results with some popular frameworks and official releases in terms of speed.

Settings

Software Environment

  • Python 3.7
  • PyTorch 1.10
  • CUDA 11.1
  • CUDNN 8.1.0

Main Results

SceneNeRF

NeRF

test data PSNR SSIM
NeRF XRNeRF NeRF XRNeRF
blender_chair 33.927 34.528 0.967 0.985
blender_drums 25.600 25.685 0.925 0.946
blender_ficus 30.13 29.300 0.964 0.972
blender_hotdog 36.18 35.905 0.974 0.985
blender_materials 29.62 29.014 0.949 0.967
blender_mic 32.58 32.95 0.980 0.986
blender_ship 28.65 29.46 0.856 0.932
llff_fern 25.17 26.277 0.792 0.892
llff_flower 27.40 26.592 0.827 0.884
llff_fortress 31.16 31.485 0.881 0.952
llff_horns 27.45 26.162 0.828 0.895
llff_leaves 20.92 19.749 0.690 0.668

Kilo-NeRF

test data PSNR SSIM elapsed_time(ms)
KiloNeRF XRNeRF KiloNerf XRNeRF KiloNerf XRNeRF
nsvf_Synthetic_NeRF_chair 33.044 33.037 0.971 0.979 384.98 407.78
nsvf_Synthetic_NeRF_drums 25.327 25.308 0.931 0.949 413.03 353.62
nsvf_Synthetic_NeRF_ficus 30.1 30.176 0.967 0.975 351.04 337.22
nsvf_Synthetic_NeRF_hotdog 32.316 33.408 0.974 0.986 484.22 491.49
nsvf_Synthetic_NeRF_lego 33.398 33.381 0.971 0.982 379.1 365.16
nsvf_Synthetic_NeRF_materials 29.193 29.175 0.951 0.966 380.28 358.57
nsvf_Synthetic_NeRF_mic 33.186 33.346 0.982 0.987 370.31 346.71
nsvf_Synthetic_NeRF_ship 28.892 29.295 0.874 0.933 491.92 488.35
Average 30.68 30.89102 0.9526 0.9697 406.86 393.61

Mip-NeRF

MultiScale Blender PSNR
800x800 400x400 200x200 100x100
Jax XRNeRF Jax XRNeRF Jax XRNeRF Jax XRNeRF
blender_ship 29.599 28.522 31.955 30.754 33.845 32.848 34.868 33.754
blender_mic 33.739 32.478 36.353 35.008 38.837 37.958 39.011 38.064
blender_materials 30.128 29.278 31.424 30.505 33.163 32.192 34.174 33.122
blender_lego 33.971 32.803 35.248 34.123 35.796 34.848 35.223 34.382
blender_hotdog 36.457 35.803 38.382 37.631 39.831 39.096 39.935 39.038
blender_ficus 31.490 29.222 32.267 30.093 33.255 31.655 33.606 31.785
blender_drums 25.297 24.790 26.463 26.020 27.808 27.510 28.791 28.369
blender_chair 33.351 32.429 36.517 35.618 38.056 37.342 37.950 37.257
Average 31.754 30.666 33.576 32.469 35.074 34.181 35.445 34.472

InstantNGP

test data PSNR
InstantNGP XRNeRF
blender_chair 32.927 32.71
blender_drums 26.02 26.9
blender_ficus 33.51 33.97
blender_hotdog 37.40 37.17
blender_lego 36.39 35.1
blender_materials 29.78 30.73
blender_mic 36.22 34.05
blender_ship 31.1 30.0
average 32.92 32.58

HumanNeRF

Neural Body

test data PSNR SSIM
Neural Body XRNeRF Neural Body XRNeRF
313 35.21 37.76 0.985 0.993
315 33.07 35.99 0.988 0.992
377 33.86 33.86 0.985 0.986
386 36.07 34.24 0.984 0.984
387 31.39 31.99 0.975 0.979
390 34.48 35.45 0.980 0.984
392 35.76 35.11 0.984 0.986
393 33.24 33.50 0.979 0.985
394 34.31 35.61 0.980 0.984

Animatable NeRF

test data (Novel pose) PSNR SSIM
Animatable NeRF XRNeRF Animatable NeRF XRNeRF
S1 30.11 31.98 0.981 0.984
S5 32.60 33.25 0.987 0.990
S6 29.49 30.12 0.972 0.974
S7 31.54 34.47 0.984 0.988
S8 30.77 32.01 0.983 0.985
S9 31.94 28.61 0.980 0.976
S11 33.12 33.43 0.986 0.986

GNR

test data PSNR SSIM
GNR XRNeRF GNR XRNeRF
amanda 23.62 25.35 0.93 0.95
barry 29.28 30.71 0.94 0.95
fuzhizhi 21.96 21.42 0.90 0.89
jinyutong 23.90 24.08 0.90 0.91
joseph 26.30 24.46 0.94 0.92
maria 21.51 23.69 0.90 0.90
mahaoran 28.41 30.93 0.93 0.94
natacha 28.71 27.98 0.91 0.91
soufianou 27.64 28.83 0.93 0.93
zhuna 25.40 24.32 0.93 0.92