Possible bug: All images in subject transformed when using patches

[dmus]

With this piece of code:

import numpy as np
import torchio
from torch.utils.data import DataLoader

subject = torchio.Subject(
    mr=torchio.ScalarImage(tensor=np.random.rand(1, 8, 16, 16)),
    out=torchio.ScalarImage(tensor=np.random.rand(1, 8, 16, 16)),
    sampling_map=torchio.ScalarImage(tensor=np.random.rand(1, 8, 16, 16))
)

rescale_mr = torchio.transforms.RescaleIntensity(percentiles=(0.5, 99.5), out_min_max=(-1, 1), keys='mr')

sampler = torchio.data.WeightedSampler(4, 'sampling_map')

transforms = [rescale_mr]
transform = torchio.transforms.Compose(transforms)

subjects_dataset = torchio.SubjectsDataset([subject], transform=transform)

patches_queue = torchio.Queue(
    subjects_dataset,  # instance of torchio.SubjectsDataset
    32,
    16,
    sampler,
    num_workers=4,
    shuffle_subjects=True,
    shuffle_patches=True,
)

dataloader = DataLoader(patches_queue, batch_size=4)

for batch in dataloader:
    inputs = batch['mr'][torchio.DATA]
    print(inputs.shape)

The following error occurs:
ValueError: Negative values found in probability map "sampling_map"

It looks like the RescaleIntensity does also apply the transform to the sampling map, is this a bug or am I doing something wrong here?

[fepegar]

Hi, @dmus. Thanks for the MWE, I appreciate it.

Can you try this?

subject = torchio.Subject(
    mr=torchio.ScalarImage(tensor=np.random.rand(1, 8, 16, 16)),
    out=torchio.ScalarImage(tensor=np.random.rand(1, 8, 16, 16)),
    sampling_map=torchio.Image(tensor=np.random.rand(1, 8, 16, 16), type=torchio.SAMPLING_MAP)  # this line changed
)

The transform thinks that the image named sampling_map is a "normal" intensity image.

The documentation is not very clear about this. I'll try to improve it soon.

[dmus]

Thanks, that works

[dmus]

Do the transforms work on the complete images or on the patches only? Ideally only the patch I guess, because of efficiency, but for the RescaleIntensity for example the min and max from the input should be computed on the complete image I think.

Can you tell me how this works?

[fepegar]

As you can see in the docs, transforms are applied to the volumes:

[dmus]

Okay, and can you tell me when the 'keys' argument of a transform is used. What if I have an intensity transform that should only apply to 'mr' and another intensity transform that should only apply to 'out' (which are both intensity images)?

[fepegar]

Okay, and can you tell me when the 'keys' argument of a transform is used.

Have you read the docs for Transform?

Mandatory if the input is a Python dictionary. The transform will be applied only to the data in each key.

---

What if I have an intensity transform that should only apply to 'mr' and another intensity transform that should only apply to 'out' (which are both intensity images)?

At the moment, this can only happen if you use a dict as input to the transform. There's an old issue about this (#54) but it's not implemented as no one has asked about it. Would you like to contribute?

[dmus]

I saw that torchio.Image is inheriting from dict class Subject(dict):, so that was leading to some confusion.

I think this is an important feature. Because Subject is already inheriting from dict it should not be that much work? Do you have ideas? Maybe I want to contribute, but I am still taking a dive into the codebase

[fepegar]

I'll move the discussion to #54.