match

[Horzion0314]

Firstly, thank you for your work. I have a question: the paper introduces several methods for matching. Why were methods like LightGlue and SuperGlue not used in the experiments?

[Horzion0314]

I wonder if it's possible

[georg-bn]

It would be possible, but the benefits of using steerers diminishes with more costly matching methods. Still, you could use the lightglue for dedode that is in kornia and steer the descriptions with our steerers from "setting A".

One could also retrain lightglue for our descriptors from settings B and C. I think interesting future work would be creating a learnt matcher like lightglue that is aware of the steerer and perhaps uses it in the intermediate layers.

[Horzion0314]

thank you so much

[guipotje]

It would be possible, but the benefits of using steerers diminishes with more costly matching methods. Still, you could use the lightglue for dedode that is in kornia and steer the descriptions with our steerers from "setting A".

One could also retrain lightglue for our descriptors from settings B and C. I think interesting future work would be creating a learnt matcher like lightglue that is aware of the steerer and perhaps uses it in the intermediate layers.

Hi @georg-bn, I was wondering if it would be possible to apply a steerer right before the final correlation matrix in LightGlue. Intuitively, the token embeddings before the final correlation are no different from an embedding from a local descriptor. By training a projection matrix on these final token embeddings, it should become rotation-equivariant, if I understood correctly?

[georg-bn]

Something like that would be very nice, I have not thought about that! Intuitively I would guess that lightglue gets confused in early layers if we input rotated images, so it might destroy the descriptions in some sense. I.e. the lightglue net tries to reason about the similarity of descriptions, but this gets very tricky if the images are rotated and the descriptions hence not similar. Perhaps something like you suggest could nonetheless be possible, it's an interesting idea for sure!

[guipotje]

It makes sense if Lightglue gets confused; nevertheless, it's worth a try to see the outcome. Thanks for replying!