TorchIO dosen't work for 2D image

[Chao86]

The transform is perfect in 3D(z,y,x) medical images, however,when I try to apply this transform in 2D(x*y) medical image, the result make it seem like the TorchIO API dosen't work for 2D images. Anybody can help me to apply RandomMotion transform in 2D medical images?My code and result as follow:

mage_info=sitk.ReadImage('124_801_P_16.mhd')  #shape Z-Y-X：1-384-384
image = sitk.GetArrayFromImage(image_info).astype("float32")  
image_nor = np.expand_dims(image, axis=-1)
image = np.tile(image_nor, (1,1,1,1)) #shape C-Y-X-Z：1-384-384-1

all_image_info =sitk.ReadImage('124_801_P.mhd')  #shape Z-Y-X：53-384-384 
all_image = sitk.GetArrayFromImage(all_image_info).astype("float32")  @
all_image = np.expand_dims(all_image, axis=0) 
all_image=all_image.transpose(0,2,3,1) #shape C-Y-X-Z：1-384-384-53

spatial_transforms={tio.RandomMotion(degrees=(-2,2), translation=(-10,10),num_transforms=5):1.0,}
[transform=tio.Compose([tio.OneOf(spatial_transforms,p=1.0)])])
transformed_image=transform(image)
transformed_images=transform(all_image)

fig, axes=plt.subplots(2,2,figsize=(7, 7))
axes[0][0].imshow(image[0,:,:,0], 'gray')
axes[0][1].imshow(transformed_image[0,:,:,0],'gray')
axes[1][0].imshow(all_image[0,:,:,16],'gray')
axes[1][1].imshow(transformed_images[0,:,:,0],'gray')

Result:

[romainVala]

Hi
The motion transform, is mean to apply on 3D volumes. Trying to apply it to 2D images, may not be very pertinent, because motion is happening in 3D even though you take a 2D image after it.

Applying 3D motion on a 2D image, allow you to simulate only "in plane" motion (and not through plane motion)
So I would definitively go with motion apply on 3D (and make it 2 D after if needed)

An other point is that the motion is "averaging" (via fourier transform) only the third dimension. Other encoding axes are assume to be acquired to quick so that they are not perturbed by motion. this is why you do not see an effect

So to see an effect on a 2D image, you need to squeeze the first or the second axis

Here is an example

import torchio as tio

#I change the rotation amplitude, to be sure it will not be near 0
spatial_transforms={tio.RandomMotion(degrees=(5,20), translation=(-10,10),num_transforms=5):1.0,}

transform=tio.Compose([tio.OneOf(spatial_transforms,p=1.0)])
suj=tio.datasets.Colin27()

image2D = suj.t1.data[0,:,:,100]
image2D=image2D.unsqueeze(0).unsqueeze(-1)
print(image2D.shape)

# note that it is better to go with Subject
suj2D=tio.Subject({"t1":tio.ScalarImage(tensor=image2D)})
suj_transformed=transform(suj2D)
suj_transformed.plot()
#it seems there is no motion

# as RandomMotion will change the parameters at each call, in order to compare the same thing you want to take the same Motion transform. that is where history is usefull

#So get the same Motion transform so I'll compare the same transform !
random_motion = suj_transformed.history[0]

#make a 2D image but extend the first axis
image2D = suj.t1.data[0,100,:,:]
image2D=image2D.unsqueeze(0).unsqueeze(0)

suj2D=tio.Subject({"t1":tio.ScalarImage(tensor=image2D)})
suj_transformed=random_motion(suj2D)
suj_transformed.plot()
#now you see motion

[Chao86]

Hi
The motion transform, is mean to apply on 3D volumes. Trying to apply it to 2D images, may not be very pertinent, because motion is happening in 3D even though you take a 2D image after it.

Applying 3D motion on a 2D image, allow you to simulate only "in plane" motion (and not through plane motion)
So I would definitively go with motion apply on 3D (and make it 2 D after if needed)

An other point is that the motion is "averaging" (via fourier transform) only the third dimension. Other encoding axes are assume to be acquired to quick so that they are not perturbed by motion. this is why you do not see an effect

So to see an effect on a 2D image, you need to squeeze the first or the second axis

Here is an example

import torchio as tio

#I change the rotation amplitude, to be sure it will not be near 0
spatial_transforms={tio.RandomMotion(degrees=(5,20), translation=(-10,10),num_transforms=5):1.0,}

transform=tio.Compose([tio.OneOf(spatial_transforms,p=1.0)])
suj=tio.datasets.Colin27()

image2D = suj.t1.data[0,:,:,100]
image2D=image2D.unsqueeze(0).unsqueeze(-1)
print(image2D.shape)

# note that it is better to go with Subject
suj2D=tio.Subject({"t1":tio.ScalarImage(tensor=image2D)})
suj_transformed=transform(suj2D)
suj_transformed.plot()
#it seems there is no motion

# as RandomMotion will change the parameters at each call, in order to compare the same thing you want to take the same Motion transform. that is where history is usefull

#So get the same Motion transform so I'll compare the same transform !
random_motion = suj_transformed.history[0]

#make a 2D image but extend the first axis
image2D = suj.t1.data[0,100,:,:]
image2D=image2D.unsqueeze(0).unsqueeze(0)

suj2D=tio.Subject({"t1":tio.ScalarImage(tensor=image2D)})
suj_transformed=random_motion(suj2D)
suj_transformed.plot()
#now you see motion

Thank you for your answer and code! I still confused about the sentences "Applying 3D motion on a 2D image, allow you to simulate only "in plane" motion (and not through plane motion)
So I would definitively go with motion apply on 3D (and make it 2 D after if needed)

An other point is that the motion is "averaging" (via fourier transform) only the third dimension. Other encoding axes are assume to be acquired to quick so that they are not perturbed by motion. this is why you do not see an effect"
can you offer me some information of these two points about MR imaging motion ? I would appreciate it if.

[romainVala]

to see the difference it on the same slice :
juste change

image2D = suj.t1.data[0,100,:,:]
image2D=image2D.unsqueeze(0).unsqueeze(0)

by

image2D = image2D.permute([0,3,1,2])

[romainVala]

Not easy to explain, because it is not a trivial transform, and depend on MR physics acquisition properties... but I give a try

through plane motion is when the affine move the plane, but if you only have a 2D image as input, there is noting to mixt with.
So only within plane motion will have an effect.
The current implementation choose 3D affine motion, but only the part that move within the plane will have an effect ...

Note here that the motion transform is only valid, for MR acquisition. In MR the volume/image is acquired in fourier domain. In 3D your have 3 axis, and the kspace (== fourrier domaine) axis are sample differently, (same thing as when you order a 3D matrix, you have fast_varying index and the slowest varying index)
So for motion, only the slowest varying index (Currently the third axis, but this should be possible to change, as plane by #81). Motion occuring during the sampling of the first 2 axis is ignored

so if the 2D image has no dimension on the third axis, then no motion will be consider ...

I hope it helps

[Chao86]

Thank you for your detailed explanation. That would be really helpful.