ERROR: mutual_information is not a valid blockmetric type. Exiting!

[nwschurink]

Hi,

I am interested in trying out your registration algorithm as an initialization for rectal registration. (I would like to transfer rectal tumor segmentations to PET/CT)

Compiling the algorithm on Linux goes smoothly if I leave the cmake parameters to default.

Then, running mirorr with

time /opt/Mirorr-master/build/bin/mirorr -m CT_image.nii.gz -f T2_TRA_image.nii.gz -a 1 -c -2 -d 5 -t affine -l output/CT_to_MR-affine.tfm --save-resampled-moving output/CT_in_MR-affine.nii.gz --fresh --blockmetric mi
I get the error message as displayed in the title.

ERROR: mutual_information is not a valid blockmetric type. Exiting!

Looking into mirror.cxx I also found out that a different mutual_information metric has been implemented, in which I can define the number of histogram bin.
However, running ccmake and configuring with USE_NPW=ON, it requests a path to a milx-view library which I cannot find in the repository.

Is there any way you can help me to get this work?

Kindly thank you in advance

[ashgillman]

Hi, sorry for the late reply.

Any reason you are trying to build with NPW? I have tried chasing up with some of the original authors as to what it was originally for, as far as I can see it doesn't do much. I might look at removing the option.

I have also sent an email off to Nick, who implemented the MI stuff as this shouldn't happen. I find that Mirorr tends to work well with the default without specifying a blockmetric. I believe this is how it was used with results reported in the paper.

[drhenault]

Hi Niels and Ash, Sorry for the delay, I'm just back from paternity leaves. The mutual information implementation is working and can improve the results in some specific cases. That said, the NCC block metric generally work well even for cases where it's a bit counter-intuitive to use a (global) NCC metric. E.g.: CT-MRI, MRI-PET, MRI-US... What I'm unsure of is the publication status for the MI/NPW block algorithm. Maybe it was not pushed to the public repository, awaiting a publication or other? Nick, what's your input? If it's OK to release to code, I could probably add it and fix the CMake. Neils: what platform are you targeting? Linux 64bits? Cheers, David 2018-04-04 6:25 GMT-04:00 Ashley Gillman <notifications@github.com>:

…

Hi, sorry for the late reply. Any reason you are trying to build with NPW? I have tried chasing up with some of the original authors as to what it was originally for, as far as I can see it doesn't do much. I might look at removing the option. I have also sent an email off to Nick, who implemented the MI stuff as this shouldn't happen. I find that Mirorr tends to work well with the default without specifying a blockmetric. I believe this is how it was used with results reported in the paper. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKkfu-vg5jGVaLDcJrnAW__5EykdzuCkks5tlJ-tgaJpZM4S_DOY> .

[nwschurink]

Hi Ash, thank you for your response.

The reason I tried to change the blockmetric is that for my test images (pelvic T2 MRI and CT) the results are not what I was hoping for (using either a rigid or affine transform). Mirror works excelent for the test images provided, however I suspect that perhaps the different field-of-fiew of my MRI and CT image (see attached image) makes the registration harder. (I will attach an image of the result later today when I have access to my linux pc). I noticed that in the paper describing the Mirorr algorithm it was noted that a different field of view might potentially result in parameters collapsing into singularities when using the affine transform, however also the rigid transform seems to be far off in my case. Is it truly neccesary for the FOV to be the same? As this would greatly hamper the applicability of Mirorr for my purpose (i.e. registering a few hundred pelvic scans), if I would need to alter the FOV for each of these scans.

The reason why I selected MI as blockmetric is that I was hoping that this would improve the registration result. Which is also the why I tried to build it with the NPW option after running Mirorr with the mutual information metric failed. I saw something about a different mutual information metric option in mirorr.cxx if it was built with the NPW option. Unfortunately selecting this option resulted in a failed build because it depends on a private library.

If you have any thoughts about how to improve the registration results without using MI, I'm all ears :)

[drhenault]

Hi, Unless you chose to go with a much more global approach, e.g. something based on deep-learning the whole torso anatomy, I suspect you'll have problem with any/most geometric optimisation based methods when your field of views present such large dissimilarity. Keep in mind that most registration algorithm rely on some _local_ optimization process: you don't want to start close to a minima that is far from the desired solution. The good news is that you probably don't need a complicated global approach. Check the following: - How's your initialization? Do the image reasonably overlap at the beginning? If your initialization is wrong, correct it beforehand using basic engineering methods -- there is nothing wrong with that. The job of the registration algorithm is to provide a finer alignment, not to correct for engineering mistakes. (Either update the image position matrices or provide an initial transform to mirorr) - You can validate the initialization transform by running Mirorr with a very low number of iteration (e.g. 1 -- can't remember if 0 works) - If this is not sufficient, try clipping the upper/lower part of the CT using the MR a bounding box, potentially adding a margin of 1-3 cm, no need to be accurate. You can easily automate that using ITK code / command line tools. My guess is that the above will work. Mirorr has been tested on hundreds of cases with generally good results, and T2-MR to CT was the main focus. I do not think MI/NPW would make a huge difference for your problem, although I'd be happy to add it to the repo if needed and possible. David 2018-04-04 9:22 GMT-04:00 Niels Schurink <notifications@github.com>:

…

Hi Ash, thank you for your response. The reason I tried to change the blockmetric is that for my test images (pelvic T2 MRI and CT) the results are not what I was hoping for (using either a rigid or affine transform). Mirror works excelent for the test images provided, however I suspect that perhaps the different field-of-fiew of my MRI and CT image (see attached image) makes the registration harder. (I will attach an image of the result later today when I have access to my linux pc). I noticed that in the paper describing the Mirorr algorithm it was noted that a different field of view might potentially result in parameters collapsing into singularities when using the affine transform, however also the rigid transform seems to be far off in my case. Is it truly neccesary for the FOV to be the same? As this would greatly hamper the applicability of Mirorr for my purpose (i.e. registering a few hundred pelvic scans), if I would need to alter the FOV for each of these scans. The reason why I selected MI as blockmetric is that I was hoping that this would improve the registration result. Which is also the why I tried to build it with the NPW option after running Mirorr with the mutual information metric failed. I saw something about a different mutual information metric option in mirorr.cxx if it was built with the NPW option. Unfortunately selecting this option resulted in a failed build because it depends on a private library. If you have any thoughts about how to improve the registration results without using MI, I'm all ears :) [image: t2_ct_images] <https://user-images.githubusercontent.com/12720130/38307184-391364c4-3813-11e8-81bf-2bec43bdd09c.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKkfu7801n3tEdX_I9Aue4cgMeH1aeBuks5tlMktgaJpZM4S_DOY> .

[ashgillman]

Some additional info Niels @nwschurink:

An email came back from Nick also which was not included in this public forum.

Mutual information will not work well as the blocks are too small.
NPW stands for non-parametric windows which addresses this issue.

Block matching works just fine without MI even for multimodal registration, so default should be fine.

At this point, the NPW code is unreleased. But as highlighted by David, within local patches, NCC is much more an appropriate metric - there are often only two contrasts within a patch.

[ashgillman]

Thanks for highlighting this, for now I will look at removing the NPW references within the code here since they aren't helpful.

@NicholasDowson, does this mean it also won't make sense to have an mi blockmetric option, or is this issue in Niels' original message distinct?

[nwschurink]

Hi David,
Thank you for your reply, and congratulations on becoming a father!

I have checked the initialization of Mirorr, which looks fine (see attached image). Then I ran the first 3/5 levels of registration by using the code in the first post. This resulted in a registration that was more off than the initialization.

However, your tip on cropping the CT image really worked out (see 2nd image). I noticed that this option is included in the Mirorr algorithm it self (using --crop).

Also regarding your note on the NCC, I am really amazed that this metric works so well. As typically correlation based metrics assume a linear intensity mapping between the volumes. Nonetheless, as was also mentioned in the Mirorr paper, apparently the NCC² behaves similar to MI. Which probably has to do with the comment of @ashgillman that a single patch often contains only two contrasts? Do I understand it correctly then and is the NCC² only evaluated on a patch by patch basis, and not on the set of patched around a voxel?

As the NCC metric is working fine, I don't believe I need the MI metric, which means that the current implementation is fine. Thank you guys for your time.

Cheers,

Niels

[drhenault]

Hi Niels, Good to know that it's working nicely for you now! Unfortunately, sometime cropping is indeed necessary, but you might be able to avoid it by looking at the re-sampling space. By default, Mirorr use a resampling space (voxel number + dimension) that is at some sort of average of both input images (CT, MR). This is to avoid favoring one image over the other, and allows to revert the order of input images without changing the result too much (except for inverting the solution). There are command line options to define the resampling space (something like --resampling fixed / moving - it don't have mirorr in front of me) that you could experiment with. If you do so, just remember to make sure that which image is "moving" and which is "fixed" always correspond to the same modality or you will get funky results (e.g. MR is moving and CT is fixed). Or maybe --crop is fine. In what concerns the metric, I believe your understanding of the local NCC is basically correct, and so is @ashgillman <https://github.com/ashgillman> comment. Local NCC is based on local search and is thus local (as opposed to global MI) and highly non-linear (as opposed to global NCC). It does not use any kind of local averaging (except for image resampling). It is quite sensible to the presence of edge and other contrast pattern (regardless of which side is white and which is black). Cheers, David 2018-04-05 5:33 GMT-04:00 Niels Schurink <notifications@github.com>:

…

Hi David, Thank you for your reply, and congratulations on becoming a father! I have checked the initialization of Mirorr, which looks fine (see attached image). Then I ran the first 3/5 levels of registration by using the code in the first post. This resulted in a registration that was more off than the initialization. However, your tip on cropping the CT image really worked out (see 2nd image). I noticed that this option is included in the Mirorr algorithm it self (using --crop). Also regarding your note on the NCC, I am really amazed that this metric works so well. As typically correlation based metrics assume a linear intensity mapping between the volumes. Nonetheless, as was also mentioned in the Mirorr paper, apparently the NCC² behaves similar to MI. Which probably has to do with the comment of @ashgillman <https://github.com/ashgillman> that a single patch often contains only two contrasts? Do I understand it correctly then and is the NCC² only evaluated on a patch by patch basis, and not on the set of patched around a voxel? As the NCC metric is working fine, I don't believe I need the MI metric, which means that the current implementation is fine. Thank you guys for your time. Cheers, Niels [image: mirorr_initialization] <https://user-images.githubusercontent.com/12720130/38356143-3c8d3236-38bf-11e8-8f8b-7b3622e92343.png> [image: mirorr_registration_affine] <https://user-images.githubusercontent.com/12720130/38358013-72344834-38c4-11e8-82c5-3437276ab5bc.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKkfu29BPLQ05pUKkv-tUZYSj_NrLzI4ks5tleTMgaJpZM4S_DOY> .