how to handle aligning images that are mirrored/flipped automatically

[snibbor]

Hi,

So I noticed that VALIS is not aligning these H&E and IF images that are mirrored/flipped.

Below are the processed images (which look good to me), and the rigid overlap that was calculated. I had to increase the max image dimension to ~10,000 because the original images are at different resolutions (0.325 MPP vs 0.5 MPP).

Is there a feature in VALIS that can handle this case? I will try to flip the H&E image along the vertical axis and retry the image alignment, I think this will fix the problem. Just wondering if there is a VALIS parameter that I can turn on to test for mirroring/flips before finalizing the registration. Maybe there's an elegant way to test that the measured registration error is below some value or else it will test the alignment with flipping the current image along an axis?

My apologies if this is in the docs already and I just haven't found it.

Rigid overlap:

Nonrigid overlap:

[cdgatenbee]

Hi @crobbins327,
At the moment, valis uses scikit-image to estimate the rigid transforms from matched features, but I'm not sure that they can account for this sort of mirroring. So, at the moment, there isn't a good way to handle this sort of situation. But it does seem like it might be possible to test for this by checking if error is reduced when including reflection in the transform. If it substantially reduces the error, then the reflection could be kept as part of the transform. I can test something like this out using these images, but I'm in the process of setting up a new laptop so it might take a little longer than usual. I'll keep you updated.

Best,
Chandler

[snibbor]

@cdgatenbee That's sounds great Chandler! If you want the raw data I'll add a couple files for you to download in a Google drive link.

I will also try to play with it and see if I can come up with anything. I also have the temporary solution of just flipping the image before VALIS registration which works since most of the image pairs I have are reflected. No rush.

Cheers,
Jack

[cdgatenbee]

@crobbins327, thank you for the offer, but I think I'll start with the images linked in the first post. But if I run into some issues I'll let you know and then I could check out the raw data (which I imagine is pretty big). I haven't run into this mirroring issue before, but it is it common in types of images you're working with? Just wondering how common this sort of issue might be.

Best,
Chandler

[snibbor]

Not sure, but our fluorescence microscope produces images reflected across the horizontal axis.

I can compress the files to be ~10GB in total if you are interested. Just let me know.

[cdgatenbee]

Hi @crobbins327,
I added some tests that will determine if the alignment is improved after reflecting the images along the x and or y axes. However, it's an option that has to be set when initializing the Valis object, which is done by setting check_for_reflections=True. I made it optional for now because it requires repeating the process of feature detection and matching for each variant of the image, and so can be time consuming and maybe not necessary all of the time. Anyhow, I tested it with the images you provided, and here are the results I got:

Original:

Rigid registered:

Non-rigid registered:

I don't have any other images to test this with, so please let me know if you run into any problems and we can reopen this, but I'll close it for now.

Best,
Chandler

[snibbor]

@cdgatenbee This is perfect, thank you Chandler! Looking forward to testing it out and digging into the code to see how you solved this problem. :)

Thank you again for this awesome repo!