Post SplaTAM Optimization & Gaussian Splatting for iPhone Data

[pablovela5620]

After reconstructing an iphone demo using both the online and offline version. I have not yet managed to get

python scripts/post_splatam_opt.pt configs/iphone/post_splatam_opt.py

OR

python scripts/gaussian_splatting.py configs/iphone/gaussian_splatting.py

working

For post_splatam_opt

I get the following error

Loading Params
dict_keys(['means3D', 'rgb_colors', 'unnorm_rotations', 'logit_opacities', 'log_scales', 'cam_unnorm_rots', 'cam_trans', 'timestep', 'keyframe_time_indices'])
/home/pablo/0Dev/repos/SplaTAM/scripts/post_splatam_opt.py:204: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  variables['timestep'] = torch.tensor(params['timestep']).cuda().float()
Traceback (most recent call last):
  File "/home/pablo/0Dev/repos/SplaTAM/scripts/post_splatam_opt.py", line 596, in <module>
    rgbd_slam(experiment.config)
  File "/home/pablo/0Dev/repos/SplaTAM/scripts/post_splatam_opt.py", line 420, in rgbd_slam
    params, variables, optimizer, intrinsics, w2c, cam = initialize_first_timestep_from_ckpt(ckpt_path,mapping_dataset, num_frames,
  File "/home/pablo/0Dev/repos/SplaTAM/scripts/post_splatam_opt.py", line 206, in initialize_first_timestep_from_ckpt
    optimizer = initialize_optimizer(params, lrs_dict)
  File "/home/pablo/0Dev/repos/SplaTAM/scripts/post_splatam_opt.py", line 164, in initialize_optimizer
    param_groups = [{'params': [v], 'name': k, 'lr': lrs[k]} for k, v in params.items()]
  File "/home/pablo/0Dev/repos/SplaTAM/scripts/post_splatam_opt.py", line 164, in <listcomp>
    param_groups = [{'params': [v], 'name': k, 'lr': lrs[k]} for k, v in params.items()]
KeyError: 'keyframe_time_indices'

For Gaussian Splatting

I get

Loading Dataset ...
Traceback (most recent call last):
  File "/home/pablo/0Dev/repos/SplaTAM/scripts/gaussian_splatting.py", line 626, in <module>
    rgbd_slam(experiment.config)
  File "/home/pablo/0Dev/repos/SplaTAM/scripts/gaussian_splatting.py", line 373, in rgbd_slam
    desired_height=dataset_config["desired_image_height_init"],
KeyError: 'desired_image_height_init'

I have tried replacing desired_image_height_init with just desired_image_height but there seems to be a bunch of incorrect keys inside the config for gaussian_splatting.py and its not clear that this version is working

[Nik-V9]

Hi, Thanks for flagging this! I will resolve this soon.

1. The first error occurs because we don't save keyframe info for the iPhone demo as opposed to our standard dataset inference code.
2. The gaussian_splatting.py config has not been verified for the iPhone data. We will test and update it soon.