2d registration

[d5423197]

Hello author,

I am new to this area. Recently I found this paper, does this repo support 2d registration?

Thanks,

ZD

[cwmok]

Hi @d5423197,

Welcome to image registration and thanks for your interest in our work.

Currently, I don't have any plan to extend this repo into 2D. Affine registration in 2D is computationally tractable for convex optimization/multi-start procedure (see here). You can basically compute the optimal solutions without any learning/CNN/ViT.

Yet, if there are more people requesting 2D C2FViT, I will update this repo.

[d5423197]

Hello author,

Thanks for your quick response. I am sorry. Because I am new to this area, I can barely understand what you are talking about.

"You can basically compute the optimal solutions without any learning/CNN/ViT."

What do you mean by this? Do you mean I do not need to use any deep learning method for 2d registration?

I may need some pre-knowledge for this, could you please introduce me with some relevant papers or tutorials?

My current situation is:
I have two images. There are obtained in different ways so they look a bit different. We are currently using another method to register these two images. I need to apply rescale, rotation, and offset alignment to register them. From my personal innocent understanding, this is an affine registration (cause rigid registration has no scale involved.)

Thanks for your kindly help.

ZD

[cwmok]

Hi ZD,

What do you mean by this? Do you mean I do not need to use any deep learning method for 2d registration?

Exactly. You don't have to use any deep learning method for 2d registration. Because there are only 6 learnable parameters in 2D affine matrix (https://en.wikipedia.org/wiki/Affine_transformation). You can easily find the optimal solutions by searching the whole space (effectively).

If you are looking for off-the-self tools for 2D registration, you may check out Elastix (https://simpleelastix.readthedocs.io/Introduction.html) and ANTs (https://antspy.readthedocs.io/en/latest/registration.html). These methods achieve descent results in 3D brain registration as reported in our paper. More importantly, these methods support 2D images.

If you are looking for a keypoint detection and matching-based affine registration method, you can simply compute the SIFT keypoints for each image and match the keypoints with RANSAC (https://ieeexplore.ieee.org/document/7738198). Then, the affine transformation can be derived by the least-square fitting of the matching. The latest development of such a method in deep learning can be found at https://openreview.net/forum?id=OrNzjERFybh.

[d5423197]

Hello there,

I really appreciate your reply. I will definitely look into the info and tools you advised. This info really helps me out.

Best,

ZD

[ChrisBtt]

Hi @d5423197,

Welcome to image registration and thanks for your interest in our work.

Currently, I don't have any plan to extend this repo into 2D. Affine registration in 2D is computationally tractable for convex optimization/multi-start procedure (see here). You can basically compute the optimal solutions without any learning/CNN/ViT.

Yet, if there are more people requesting 2D C2FViT, I will update this repo.

Hey @cwmok,

Thank you very much for your capable model. The architecture with long- and short-range dependencies sounds very promising.
Your results compared to other models look great.

I was also looking for a 2D image registration neural network because I cannot use analytical algorithms for my problem. I want to "puzzle" neighboring pieces together. Therefore, I would also appreciate a hint where I can change your Training loop or model in order to support 2D image registration.

Cheers,
Chris