Question on keypoint computation

[jamt9000]

Sharing a question received by email:

Thanks for sharing your DVE codes. I found the new keypoints computed in your code is somehow counter-intuitive. Could you kindly explain: Why you construct a KD tree with the warped grid instead of the regular one?

This is a link to the questioned line in your code

DVE/utils/tps.py

Line 164 in 2085144

def warp_keypoints(self, keypoints, grid_unnormalized):

[jamt9000]

And my answer

The warp grid defines an inverse image warp - ie, each element in the grid is an index into the original image (the alternative would be to use a forward warp, where we give the index of where each pixel should end up in the new image, but that can give an image with holes). Because the keypoints are in original image coordinates, to find where it has moved to we need to locate the nearest neighbour in the warp grid.

[jamt9000]

Whether it makes sense to use a KD-tree to do this is another matter. It was ported from the use of knnsearch in matlab as a simple way to find the transformed keypoints when we don't have the warp in that direction. One limitation is that the transformed keypoints will be integer locations.

[flowtcw]

Thanks a lot!