High resolution image training by dividing image patchs.

[ZZy129326999]

Excuse me. By training the high-resolution image in some blocks and using corresponding patch offsets of cx and cy, the rendering effect looks correct, but why do the dense points become much more than the original resolution image?

[hot-dog]

Hi ZZy129326999, i am also trying to train gaussian splatting with cropped image, but cannot get good result, may i ask how do you train with divided image patches?

[ZZy129326999]

Hi ZZy129326999, i am also trying to train gaussian splatting with cropped image, but cannot get good result, may i ask how do you train with divided image patches?

projection matrix may is not correct.

[hot-dog]

Hi ZZy129326999, i am also trying to train gaussian splatting with cropped image, but cannot get good result, may i ask how do you train with divided image patches?

projection matrix may is not correct.

Yeah, i also think the projection matrix cause the quastion but i dont konw how to construct the correct projection matrix. Could you show me your code for constructing the projection matrix? Thank you:)

[jaco001]

High density is a good thing. The overall density depends on the size of the scene and the amount of detail. If you are taking photos of a single element, taking a long perspective reduces the details. (algorithm averages density).
An array of photos from a single photo can still generate more points because the algorithm uses all the photos at the same time and 'looks' for common points. (there is noise, perspective imperfections, number rounds, etc.) More steps also generate more points.

[jaco001]

Hi ZZy129326999, i am also trying to train gaussian splatting with cropped image, but cannot get good result, may i ask how do you train with divided image patches?

Maybe masks will be better solution than cropping?

[hot-dog]

Hi ZZy129326999, i am also trying to train gaussian splatting with cropped image, but cannot get good result, may i ask how do you train with divided image patches?

Maybe masks will be better solution than cropping?

Thanks for your advice, i have applied mask in my training. I construct my training data as follows: given a large scene(200meters x 200meters), take thounds of images with UAV(the gsd is about 5mm), align all images with colmap to get sfm points, then split the sfm points to small patchs(e.g 7m x 7m ), for a certain sfm points patch, constructing a bounding box with its xyz's range, then applying ray casting with each camera to get the foreground mask and filter out cameras that do not cover the specified area, at last crop the foreground with rectangle box and modify the cx, cy accordingly.
I construct the projection matrix according to #399 (comment), the result is better but there is another question as the following image show, there are lots of gaussian out side of the scene. Why?

If i dont crop the image, i can get quiet good result.

[jaco001]

Even if you think that camera aligment is perfect, this isn't trivial task, so often you get some of them in different places

Just one camera can generate splats at the other side. Croping is ok in this case, but also you can loose some data.
Try this approach https://github.com/kangpeilun/VastGaussian

[TarzanZhao]

If you have multiple GPU, Grendel-GS can support high-resolution reconstruction by dividing image patches: https://github.com/nyu-systems/Grendel-GS