Prepare customized dataset

[lzhangbj]

Hi Riegler, thank you for this great work in view synthesis. It is really appreciated that you have provided you preprocessed dataset and training/eval code. May I know how to preprocess our customized dataset (a few images) so that we can apply this method directly on it?

[griegler]

You can run the full colmap pipeline on your images to get the rectified images, camera parameters and poses, and the surface mesh. After that, you have to render the depth maps. I used pyrender, but anything works. With that you can use ext.preprocess.count_nbs to compute the overlap between the images. That's it in a nutshell.

[lzhangbj]

Great. Thanks for your quick response

[phongnhhn92]

@griegler In your uploaded dataset, each scene has a delaunay_photometric.ply file. Is that the mesh that you have used to render the depth map of any poses using pyrender ?

[griegler]

@phongnhhn92 yes.

[phongnhhn92]

@phongnhhn92 yes.

Great! It would be helpful if you can share the script you have used to render the depth map. I am curious about which camera parameter you have chosen to render it.

[visonpon]

@phongnhhn92, in the preprocess.cpp, mat3_t K=(src_Ks_np.data() + vidx * 3 * 3, 3, 3), src_Ks_np's size is (n_views,3,3), but K'size is (3,3), I wonder whether this operation has some problem?

[griegler]

@phongnhhn92 Most likely I can not upload it to this repo. But we are working on releasing the code for Stable View Synthesis and hopefully this will include the necessary scripts.

@visonpon src_Ks_np has size (n_views,3,3) because it projects into all source views (n_views).

[jly0810]

You can run the full colmap pipeline on your images to get the rectified images, camera parameters and poses, and the surface mesh. After that, you have to render the depth maps. I used pyrender, but anything works. With that you can use ext.preprocess.count_nbs to compute the overlap between the images. That's it in a nutshell.

Hello, I am very interested in your work. May I now open up the code for this part? I want to use some of my own photos to obtain a dataset, but I have encountered some difficulties. I hope you can provide relevant codes and instructions. Thank you!