Any way to pair this with segmentation masks?

[caxel-ap]

Is there any way to adapt this method to work with segmentation maps? For example, imagine i have a scene with 10 RGB images of trees in a forest. If I run a segmentation model and have binary masks of all the tree pixels for each RGB image, is there anyway to run vggt and get out point maps that have r,g,b, classification (background vs tree). ultimately looking to run analytics on a classified point cloud generated from your model

[jytime]

Hi @caxel-ap ,

I feel there is no problem, if I did not misunderstand anything. I guess you could try two ways:

1. Run VGGT over images as usual, and then directly apply your segmentation masks over pixels.
2. For each type of tree, first specifying all other pixels (or all other tree pixels) as black or white, which will let the model ignore them. Then run the model and get the reconstructed results. Note that in this way you need to filter out the points corresponding to black or white pixels after the run