How to build and parse my own data

[WN-Wolf]

Hello @Anttwo,
I want to build my own dataset. I still have some questions.
First, what type of camera should I use? Do you have any recommendations?
Next, how do I parse data? Like the below struct, how to get cameras.bin && images.bin && points3D.bin && points3D.ply && project.ini files?
Thank you again, and looking forward to your reply!

[kitmallet]

Hi @WN-Wolf I could probably help you out here as I have done photogrammetry for many years, although there are some special things to consider for Gaussian Splatting.
You can use any type of camera really. A better quality one would do a better job of course. It is important to not shoot too wide, like a go-pro. With my Canon I use a 18mm lens which works very well for me. Also, use as high f-stop as possible to get everything in focus. Shooting video is easier, but not as good quality. Whether you shoot video or photos it is also best to sharpen the images with Topaz, DarkTable or any other photo editing software. If you shoot objects it is good to get about 30-40 pictures for each circle around it. I Would do three rotations at 3 different heights. One looking straight at it, one looking up at it and one looking down. So ideally you would have around 100 pictures. For Gaussian splatting it is important that every photo overlaps with the previous photo. It doesn't work well if you jump to another angle. Continuous movement is important. if you are shooting areas like outdoors make sure that each photo has more than a 50% overlap. I use about 70% for my area dataset.
If you use video you will have to defragment the video with ffmpeg.
Create a folder where you will put in your video, and ffmpeg.exe, and also create an empty folder named "input".
You can get ffmpeg here: https://www.ffmpeg.org/download.html
This is a sample of the command.
ffmpeg -ss 00:00:00 -t 00:02:46 -i myvideo.mp4 -r 1.0 input/%004d.jpg
Make sure you put the end time of your video, put in the correct name of your video, and choose how many frames you cant to take out. -r 1.0 takes one frame each second. -r 0.5 does every 2 seconds. -r 2.0 does every half second.
Open command prompt, go to your directory that you created, and put these objects. Next simply put in the command above.
You should see the input folder populate with deframed pictures. These would now be used in Gaussian splatting.
Hope this helps.
One last thing, in SuGaR folder, create a folder called "data" and inside create a folder with the name of your project. Inside the project folder place your input folder with pictures.
Then...
python gaussian_splatting/convert.py -s data/"project_name"
Inside the data/"Project_name" folder will be the following:
distorted folder
images folder
input folder
sparse folder
stereo folder
run-colmap-geometric Shell Script
run-colmap-photometric Shell Script
Then:
python gaussian_splatting/train.py -s data/"project_name"
Inside the output/good folder will be 4 files:
Point_cloud Folder
cameras JSON File
cfg_args File
input PLY file

Best,
Kit

[WN-Wolf]

@kitmallet, thanks for your help! Based on your guidance, I have run the results using my dataset. Thank you again for your detailed guidance!

[clement-chupin]

Have you any informations to create the points3D.ply in the dataset folder ?
It's needed to run correctly the SuGaR reconstruction, but there is no mention on that on colmap or initial gaussian splatting documentation.

[Anttwo]

Thank you so much @kitmallet for your detailed, and very useful explanation!

Hello @clement-chupin,
Sure, I just saw the issue you created about this problem, and I'm interested in helping you to solve it!
Feel free to provide more details about this in the dedicated issue.
I agree this looks strange, as the PLY file should be generated by the initial Gaussian Splatting script.
Let's investigate this!