Refactor transforms #213

charleygros · 2020-05-01T06:10:24Z

Following #209, some refactoring in the transforms:

im and seg transforms run separetely, see comment here.
Only use numpy operations (i.e. no PIL)
Be robust to both "sample" and "list of sample"
Split file between different small files, using Folder imports #212
Implement test functions

…-medical-imaging into cg/im-seg-roi_transforms # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

…nd noise

charleygros · 2020-05-01T06:15:26Z

In order to be compatible for both "sample" and "list of sample" I implemented the following decorator, @list_capable:

def list_capable(wrapped):
    @functools.wraps(wrapped)
    def wrapper(self, sample, metadata):
        if isinstance(sample, list):
            list_data, list_metadata = [], []
            for s_cur, m_cur in zip(sample, metadata):
                # Run function for each sample of the list
                data_cur, metadata_cur = wrapped(self, s_cur, m_cur)
                list_data.append(data_cur)
                list_metadata.append(metadata_cur)
            return list_data, list_metadata
        return wrapped(self, sample, metadata)
    return wrapper

The idea is:

the transformation function (and undo) can only receive a sample
if the input is a list, then the decorator will iterate through the sample and call the tranformation for each of them, and return a list of "transformed sample"
same idea for the metadata

charleygros · 2020-05-01T06:19:09Z

For instance, the class HistogramClipping:

Before:

class HistogramClipping(IMEDTransform):

    def __init__(self, min_percentile=5.0, max_percentile=95.0):
        self.min_percentile = min_percentile
        self.max_percentile = max_percentile

    def do_clipping(self, data):
        data = np.copy(data)
        # Ensure that data is a numpy array
        data = np.array(data)
        # Run clipping
        percentile1 = np.percentile(data, self.min_percentile)
        percentile2 = np.percentile(data, self.max_percentile)
        data[data <= percentile1] = percentile1
        data[data >= percentile2] = percentile2
        return data

    def __call__(self, sample):
        input_data = sample['input']

        # TODO: Decorator?
        if isinstance(input_data, list):
            output_data = [self.do_clipping(data) for data in input_data]
        else:
            output_data = self.do_clipping(input_data)

        # Update
        rdict = {'input': output_data}
        sample.update(rdict)
        return sample

After:

class HistogramClipping(IMEDTransform):

    def __init__(self, min_percentile=5.0, max_percentile=95.0):
        self.min_percentile = min_percentile
        self.max_percentile = max_percentile

    @list_capable
    def __call__(self, sample, metadata={}):
        data = np.copy(sample)
        # Run clipping
        percentile1 = np.percentile(sample, self.min_percentile)
        percentile2 = np.percentile(sample, self.max_percentile)
        data[sample <= percentile1] = percentile1
        data[sample >= percentile2] = percentile2
        return data, metadata

Note: it also simplifies the code regarding the labeled data.

charleygros · 2020-05-01T06:20:40Z

Another example with RandomShiftIntensity:

Before:

class RandomTensorChannelShift(IMEDTransform):

    def __init__(self, shift_range):
        self.shift_range = shift_range

    @staticmethod
    def get_params(shift_range):
        sampled_value = np.random.uniform(shift_range[0],
                                          shift_range[1])
        return sampled_value

    @staticmethod
    def sample_augment(input_data, params):
        np_input_data = np.array(input_data)
        np_input_data += params
        input_data = Image.fromarray(np_input_data, mode='F')
        return input_data

    def __call__(self, sample):
        input_data = sample['input']
        params = self.get_params(self.shift_range)

        if isinstance(input_data, list):
            ret_input = [self.sample_augment(item, params) for item in input_data]
        else:
            ret_input = self.sample_augment(input_data, params)

        rdict = {'input': ret_input}

        sample.update(rdict)
        return sample

After

class RandomShiftIntensity(IMEDTransform):

    def __init__(self, shift_range):
        self.shift_range = shift_range

    @list_capable
    def __call__(self, sample, metadata={}):
        # Get random offset
        offset = np.random.uniform(self.shift_range[0], self.shift_range[1])
        # Update metadata
        metadata['offset'] = offset
        # Shift intensity
        data = sample + offset
        return data, metadata

    @list_capable
    def undo_transform(self, sample, metadata={}):
        assert 'offset' in metadata
        # Get offset
        offset = metadata['offset']
        # Substract offset
        data = sample - offset
        return data, metadata

charleygros · 2020-05-01T06:21:48Z

For testing purposes, I implemented a function to create dummy data with labels:

def create_test_image_2d(width, height, num_modalities, noise_max=10.0, num_objs=1, rad_max=30, num_seg_classes=1):
    """Create test image.

    Create test image and its segmentation with a given number of objects, classes, and maximum radius.

    Args:
        width (int): width image
        height (int): height image
        num_modalities (int): number of modalities
        noise_max (float): noise from the uniform distribution [0,noise_max)
        num_objs (int): number of objects
        rad_max (int): maximum radius of objects
        num_seg_classes (int): number of classes
    Return:
        list, list: image and segmentation, list of num_modalities elements of shape (width, height).

    Adapted from: https://github.com/Project-MONAI/MONAI/blob/master/monai/data/synthetic.py#L17
    """

charleygros · 2020-05-01T06:22:37Z

We can now run tests such as:

@pytest.mark.parametrize('im_seg', (create_test_image_2d(100, 100, 1),
                                    create_test_image_2d(100, 100, 3)))
def test_RandomShiftIntensity(im_seg):
    im, _ = im_seg
    # Transform
    transform = RandomShiftIntensity(shift_range=[0., 10.])

    # Apply Do Transform
    metadata_in = [{} for _ in im] if isinstance(im, list) else {}
    do_im, do_metadata = transform(sample=im, metadata=metadata_in)
    # Check result has the same number of modalities
    assert len(do_im) == len(im)
    # Check metadata update
    assert all('offset' in m for m in do_metadata)
    # Check shifting
    for idx, i in enumerate(im):
        assert isclose(np.max(do_im[idx]-i), do_metadata[idx]['offset'], rel_tol=1e-02)

    # Apply Undo Transform
    undo_im, undo_metadata = transform.undo_transform(sample=do_im, metadata=do_metadata)
    # Check result has the same number of modalities
    assert len(undo_im) == len(im)
    # Check undo
    for idx, i in enumerate(im):
        assert np.allclose(undo_im[idx], i, rtol=1e-02)

charleygros · 2020-05-21T02:55:16Z

ivadomed/transforms.py

@@ -302,7 +304,6 @@ def __getitem__(self, item):
 class Crop(ImedTransform):
    def __init__(self, size):
        self.size = size if len(size) == 3 else size + [0]
-        self.is_2D = True if len(size) == 2 else False


The reason why I check the 2D versus 3D here instead of via sample --> was to be robut to 3D data with shape[2] == 1 (yes I know.. not common). While by checking with the crop_size --> we are certain that the user wants a 2D transform.
What do you think @andreanne-lemay ?

Yes I wanted to discuss that as well! The way it was before worked fine, why I changed it is so the transform automatically adapts to 2D or 3D output without having to change the tranforms. For instance, in the test_orientation.py I can use the same transforms for 2D and 3D. Also, I found it useful to not have to change multiple parameters when I want to train in 2D or 3D (now I only have to change the unet_3D bool).

It is convenient indeed. Ok, we can go with that.

charleygros · 2020-05-21T02:57:50Z

ivadomed/transforms.py

-        params_do = metadata["resample"]
-        params_undo = [1. / x for x in params_do]
+        original_shape = metadata["data_shape"]
+        current_shape = sample.shape
+        params_undo = [x / y for x, y in zip(original_shape, current_shape)]


Works too. Did you have issues with the other version? curious

I also wanted to comment on this! I changed it because using the inverse was not enough precise so I didn't have the same input and output shape after doing and undoing transforms. (e.i 51 instead of 52). When using the dimensions, I get a zoom of example 2.01343 instead of 2, which always give me the right output size.

Excellent :)

charleygros · 2020-05-21T02:58:46Z

ivadomed/transforms.py

-
-        # Save params
-        metadata['resample'] = params_resample
+        params_resample = (hfactor, wfactor, dfactor) if not is_2d else (hfactor, wfactor, 1.0)


charleygros · 2020-05-21T03:00:44Z

ivadomed/transforms.py

@@ -20,45 +20,45 @@

 def multichannel_capable(wrapped):
    @functools.wraps(wrapped)
-    def wrapper(self, sample, metadata):
+    def wrapper(self, sample, metadata, data_type):


Oh.. what I meant in Slack was to add data_type in the metadata --> Is there advantages in doing the way you did? as it is only used for the resampling.
By adding it in the metadata, we would only need to modify 1 line in the loader.

Changed it :)!

charleygros · 2020-05-21T03:01:28Z

ivadomed/transforms.py

@@ -23,9 +23,9 @@ def multichannel_capable(wrapped):
    def wrapper(self, sample, metadata, data_type):
        if isinstance(sample, list):
            list_data, list_metadata = [], []
-            for s_cur, m_cur, d_cur in zip(sample, metadata, data_type):


Same comment

charleygros · 2020-05-21T03:03:46Z

ivadomed/loader/utils.py

@@ -21,6 +18,8 @@
    'uint8': torch.ByteTensor,
 }

+TRANSFORM_PARAMS = ['resample', 'elastic', 'rotation', 'offset', 'crop_params', 'reverse', 'affine', 'gaussian_noise']


TODO charley: resample can be removed.

andreanne-lemay · 2020-05-21T03:50:31Z

On my end, everything runs well, I'm running my usual pipeline to check the performance -> looks good for now. I didn't test or modify the files in dev (we probably should check if there are all still functional?).

I didn't change 'gt' wasn't sure if we wanted label or seg?

As requested here I removed the explicits calls on all tests.

charleygros · 2020-05-21T03:51:41Z

testing/test_transforms.py

@@ -155,7 +155,9 @@ def test_NumpyToTensor(im_seg):
 def _test_Resample(im_seg, resample_transform, native_resolution, is_2D=False):
    im, seg = im_seg
    metadata_ = {'zooms': native_resolution,
-                 'data_shape': im[0].shape if len(im[0].shape) == 3 else list(im[0].shape) + [1]}
+                 'data_shape': im[0].shape if len(im[0].shape) == 3 else list(im[0].shape) + [1],
+                 'data_type': 'im'


TODO: charley or.. : make another metadata_ for seg with data_type = 'gt'

andreanne-lemay · 2020-05-21T03:59:01Z

@andreanne-lemay & @olix86 : Just so that you are aware: test_RandomAffine and test_RandomRotation may "fail" from time to time because of the problem illustrated here below, even if the do and undo transforms are technically correct.
For now, we could just (i) reduce the translation / rotation params (e.g. test only rotation of 5°) (ii) add a specific warning (eg "the test may have fail because the segmentation lost 80% of its coverage after the transform") and (iii) rerun the test. If it happens too often and becomes annoying, we could think of an alternative. Does it sound okay?

In the long run, we might want to change. If we test a small shift or rotation, will this problem disappear? Or for these we might want to always use the same image (as opposed to randomly generated)

andreanne-lemay

🥇 🥇 🥇

…otation

charleygros added 19 commits May 1, 2020 11:29

ivadomed/loader/loader: __getitem__ clean

8fe303d

testing/test_transforms.py: create_test_image_2d

1b59aa2

testing/test_transforms.py: create_test_image_2d add num_modalities a…

8bc887b

…nd noise

testing/test_transforms.py: add metadata to IMEDTransform

7b941a2

testing/test_transforms.py: list_capable

26dc855

testing/test_transforms.py: apply list_capable to IMEDTransform

b2cafdd

testing/test_transforms.py: create_test_image_2d return lists

3b7fa76

ivadomed/transforms.py: list_capable return list

489dbba

ivadomed/transforms.py: list_capable to Clipping

f673d2b

ivadomed/transforms.py: simplify clipping

2c88a85

testing/test_transforms.py: test_HistogramClipping

b86432e

ivadomed/transforms.py: RandomTensorChannelShift to RandomShiftIntensity

6fec357

ivadomed/transforms.py: RandomShiftIntensity update metadata

fb03371

ivadomed/transforms.py: RandomShiftIntensity undo_transform

b796aa4

ivadomed/transforms.py: metadata default is dict

3c33bbe

ivadomed/transforms.py: modify decorator

dbe2bfd

testing/test_transforms.py: RandomShift undo

685bfdb

testing/test_transforms.py: move rescale to transforms

9bbbbd9

charleygros self-assigned this May 1, 2020

charleygros added 5 commits May 5, 2020 17:15

fix conflicts

a1843f3

ivadomed/transforms: ToPIL to TensorToNumpy

c6a73fa

ivadomed/transforms: test NumpyToTensor

7040d00

testing/test_transforms: test NumpyToTensor

db125f7

ivadomed/transforms.py: rm ToTensor

6b19157

charleygros commented May 21, 2020

View reviewed changes

clean code + pep8

e2859e3

charleygros commented May 21, 2020

View reviewed changes

andreanne-lemay added 4 commits May 20, 2020 23:20

add data_type to metadata in 3D

2b8b383

loader: put data type else where

404c3fb

adaptative.py: add data_type to metadata

5fddb67

test_transforms: add data type to metadata

1051a03

charleygros commented May 21, 2020

View reviewed changes

andreanne-lemay added 6 commits May 21, 2020 00:15

test_segment_volume: include 1 and 0 in value range

50ce234

test_segment_volume: include 1 and 0 in value range

7386c9a

test_segment_volume: typo change max for min

f3fb055

Resample: clip values between 0 and 1

9486166

Resample: undo clipping

786958c

segment_volume: set metadata to gt when undo tranforms

ce04c02

andreanne-lemay approved these changes May 21, 2020

View reviewed changes

charleygros added 4 commits May 21, 2020 16:04

ivadomed/utils: segment_volume robust to multi_label

814d8e8

ivadomed/loader/utils: rm resample from updated transform params

22d67b6

testing/test_transforms: metadata designed for gt resample

5eead15

testing/test_transforms: reduce transform of RandomAffine and RandomR…

d2d3c18

…otation

charleygros merged commit 5cd654c into master May 21, 2020

charleygros deleted the cg/im-seg-roi_transforms branch May 21, 2020 06:46

olix86 mentioned this pull request May 26, 2020

Warning scheduler get_lr #214

Closed

andreanne-lemay mentioned this pull request May 31, 2020

Some pytest tests fail #251

Closed

charleygros mentioned this pull request Jun 1, 2020

Fix testing/test_transforms.py fails #254

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor transforms #213

Refactor transforms #213

charleygros commented May 1, 2020 •

edited

charleygros commented May 1, 2020 •

edited

charleygros commented May 1, 2020

charleygros commented May 1, 2020

charleygros commented May 1, 2020

charleygros commented May 1, 2020 •

edited

charleygros May 21, 2020

andreanne-lemay May 21, 2020

charleygros May 21, 2020

charleygros May 21, 2020

andreanne-lemay May 21, 2020

charleygros May 21, 2020

charleygros May 21, 2020

charleygros May 21, 2020 •

edited

andreanne-lemay May 21, 2020

charleygros May 21, 2020

charleygros May 21, 2020

andreanne-lemay commented May 21, 2020

charleygros May 21, 2020

andreanne-lemay commented May 21, 2020

andreanne-lemay left a comment

Refactor transforms #213

Refactor transforms #213

Conversation

charleygros commented May 1, 2020 • edited

charleygros commented May 1, 2020 • edited

charleygros commented May 1, 2020

Before:

After:

charleygros commented May 1, 2020

Before:

After

charleygros commented May 1, 2020

charleygros commented May 1, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charleygros May 21, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andreanne-lemay commented May 21, 2020

Choose a reason for hiding this comment

andreanne-lemay commented May 21, 2020

andreanne-lemay left a comment

Choose a reason for hiding this comment

charleygros commented May 1, 2020 •

edited

charleygros commented May 1, 2020 •

edited

charleygros commented May 1, 2020 •

edited

charleygros May 21, 2020 •

edited