Add checks to ensure image and label correspondence

[vnarayan13]

Add checks (in base class?):

1. image and label have similar (within tolerance) base pixel spacing
2. image and label have similar origins (does this vary with dimension?)
3. image and label have similar 3-space orientation (space directions in nrrd header?)
4. image and label have similar dimensions (not necessary, but need to somehow check registration)

[fedorov]

good idea!

@naucoin worked on this kind of checks in Slicer core, so we can just reuse here

[fedorov]

Re 2) and 4), I don't think this should be checked - label can paint a sub-volume of the input image. We can check if the label image is a sub-volume of the input image.

[vnarayan13]

True, they may be of different dimensions - I don't see it being a problem if they are properly registered.

Just a reminder, the whole image and whole label are independently resampled (i.e. to 3x3x3 with some interpolation function) and THEN cropped to a prism with the dimensions of the tumor label.

Also, I think sub-volume inputs are a minority case (and maybe should not be encouraged?).

[fedorov]

I don't see it being a problem if they are properly registered.

I can't think of a way to automatically check if they are properly registered. If you have ideas, please propose!

Just a reminder, the whole image and whole label are independently resampled (i.e. to 3x3x3 with
some interpolation function) and THEN cropped to a prism with the dimensions of the tumor label.

I don't see this done in pyradiomics.

Also, I think sub-volume inputs are a minority case (and maybe should not be encouraged?).

I agree, they should not. The question is different though: should they be prohibited (this is what is implied by your original suggestion).

[vnarayan13]

I'm not sure about automatic registration checks - though, I think I will update the cropping function to use SimpleITK cropping filters (which would account for the image orientation/origin) rather than the (naive) array index matching cropping function that we currently have.

The sitk-based resampling function is in preprocessing.py. The interpolation function differs from the baseline's function so we don't have a testing strategy...yet. The cropping function is already implemented and used, but assumes that the image and label have the same dimensions.

We don't have to prohibit sub-volumes for now, but we can at least check and log a warning to the user.

[fedorov]

The sitk-based resampling function is in preprocessing.py.

Yes, but my point is that it is not used, if you look in the base class or individual feature classes. We pass resampling to the feature classes, so probably interpolation and cropping should be done in the base class. It probably also makes sense to do padding in the base class, instead of repeating it in each of the individual feature classes.

[fedorov]

@naucoin does it make sense to reuse the geometry checks from Slicer for this purpose?

[naucoin]

@fedorov The checks in Slicer rely on the volumes being inside of volume nodes with vtkImageData, for pure image arrays I'd have to rewrite them.

[naucoin]

Check in simple ITK for quick geometry mismatch comparisons (spacing, orientation, origin not as important (as long as overlap?))

[fedorov]

related functionality in ITK: https://itk.org/Doxygen/html/classitk_1_1ImageToImageFilter.html#acc8f8667757385e7b607650620c58616

[JoostJM]

Currently, origin, spacing and direction are checked by simple ITK functionality in cropToTumorMask. However, this is after any resampling has been applied, which resamples both labelmap and image using the same grid (and will therefore output two images in the same geometry).

So as long as resampling is enabled, both are forced to the same space. Still it may be an idea to calculate the size/location of the resamplegrid completely on the labelmap (currently, bounding box is extracted from labelmap, but spacing, origin, direction, etc. is extracted from image). This would ensure the correct area is resampled in case the labelmap is but a slingle slice.

If resampling is disabled and the geometry is different, the package will throw an error due to the check that is applied during cropping.

[fedorov]

The idea was to throw error in the constructor during initialization if there is a mismatch in spacing and directions.

[JoostJM]

In that case, I think it is easiest to add a check in featureextractor.loadImage.

[fedorov]

I would add the check to the base class, since as we discussed before, featureextractor is not the only approach to use the functionality.

[JoostJM]

@fedorov It's true that featureextractor is not the only approach to the functionality, but adding it to the baseclass seems a bit strange to me, as preprocessing of the image is done outside of the baseclass (in featureextractor), and such a check should be done prior to this preprocessing, I think (certainly when you consider that resampling forces both image and labelmap to the same geometry). Additionally, such a check would only have to be done once per image, not every time it is passed to a featureclass. This is similar to why we moved resampling and cropping out of the baseclass

[fedorov]

The difference is that this check would be a very lightweight operation, just few comparisons, unlike resampling and cropping. We could also just add it to imageoperations.

[JoostJM]

True, but does not handle the other part, namely that preprocessing is done prior to the image being passed to the featureclass. Upon implementation in imageoperations, where do you propose to put it in the pipeline? Before cropping? Before resampling?

[fedorov]

I see your point. I think it would be good to have that check function standalone in imageoperations, so that developers composing their processing pipeline can use it as needed. For the featureextractor, it does make sense to add it to loadImages, and call it when no interpolation was requested. When resampling is requested, it would be good to confirm that fields of view overlap between the mask and image, and ignore inconsistencies in spacing and orientation.

[JoostJM]

In #223, I added some checks (including thoses suggested here). The checks described here are applied by SimpleITK when the bounding box is calculated using the LabelStatisticsImageFilter, with tolerance specified in the ProcessObject.

In addition to these checks, also checks some size constraints (can't extract features from a single voxel).

2 points with these checks:

1. checks will fail if ROI is a subvolume of original image. I left it like this, because these should not be encouraged, will represent (hopefully) a minority of cases and to write a check for this will most likely not be very performance efficient (as it requires you to compute min and max bound for both image and mask in realworld coordinates and then check validity)
2. these checks are applied after resampling, as the size constraints are important for the ROI after resampling (otherwise, there still could be a single voxel extraction). This does mean that if there is a mask and image that does not match up before resampling, no error will be thrown, as resampling forces them to the same geometry.

[JoostJM]

What would be possible is to use the same LabelStatisticsImageFilter during resampling (also requires the bounding box). This would do the geometry check also before resampling...

[JoostJM]

resolved by #223