output window size of 3D U-net

[enzo-cin]

Hi,
I modified demos/BRATS17 for 3D unet segmentation with my data (dimension :120x120x120, resolution 1x1x1). If I set the parameter: spatial_window_size = (8,8,8) for label, and run the command:
net_run.py train -c unet_config.ini . Everything is OK.

My questions are:
Q1: Why I can not modify spatial_window_size for a bigger value?
Q2: With this spatial_window_size, when I do: inference, there is an error:
CRITICAL:niftynet: network output window can be cropped by specifying the border parameter in config file, but here the output window (8, 8, 8) is already smaller than the input window size minus padding: [96 96 96], not supported by this aggregator

Here is my config file:
[T1]
path_to_search = ./NN
filename_contains = T1
filename_not_contains =
spatial_window_size = (96,96,96)
pixdim = (1.0, 1.0, 1.0)
axcodes=(A, R, S)
interp_order = 3

[T2]
csv_file=
path_to_search = ./NN
filename_contains = T2
filename_not_contains =
spatial_window_size = (96,96,96)
pixdim = (1.0, 1.0, 1.0)
axcodes=(A, R, S)
interp_order = 3

[label]
csv_file=
path_to_search = ./NN
filename_contains = GT
filename_not_contains =
spatial_window_size = (8,8,8)
pixdim = (1.0, 1.0, 1.0)
axcodes=(A, R, S)
interp_order = 0

[SYSTEM]
cuda_devices = ""
num_threads = 2
num_gpus = 1
model_dir = ./unet

[NETWORK]
name = unet
activation_function = prelu
batch_size = 1
decay = 0
reg_type = L2

volume_padding_size = 0

histogram_ref_file = ./label_mapping_whole_tumor.txt
norm_type = percentile
cutoff = (0.01, 0.99)
normalisation = True
whitening = True
normalise_foreground_only=True
foreground_type = otsu_plus
multimod_foreground_type = and

queue_length = 56
window_sampling = uniform

[TRAINING]
sample_per_volume = 32
rotation_angle = (-10.0, 10.0)
scaling_percentage = (-10.0, 10.0)
lr = 0.01
loss_type = Dice
starting_iter = 0
save_every_n = 5
max_iter = 10
max_checkpoints = 20

[INFERENCE]
border = (0, 0, 0)
#inference_iter = 10
save_seg_dir = ./unet
output_interp_order = 0
spatial_window_size = (96,96,96)

############################ custom configuration sections
[SEGMENTATION]
image = T1,T2
label = label
output_prob = False
num_classes = 4
label_normalisation = True

[wyli]

Hi @nguyenhg thanks for the feedback. This is using the 3D U-net (https://arxiv.org/abs/1606.06650) and by the original design, it takes 132×132×116-voxel inputs and predicts 44×44×28-voxel outputs. Both of your Q1 and Q2 are due to this "cropping" effect of the network (also please note there's another spatial_window_size parameter in the [inference] section of the config).

You could also remove the cropping layer by deleting this line https://github.com/NifTK/NiftyNet/blob/dev/niftynet/network/unet.py#L122
the result is a network taking NxNxN inputs and predicting NxNxN outputs. Hope this helps!

[enzo-cin]

Hi @wyli, thank you for your comment. I did the interpolation for my data to original design: 132×132×116 for both images and labels, voxel size is 1.76×1.76×2.04 . And re-run the training step, but the dimension is not compatible.
Could you explain me:
Why image_size should be divisible by 8 and >89?
The parameter spatial_window_size of [label] should be same with the one of [images] or not?

[wyli]

Hi @nguyenhg,

Why image_size should be divisible by 8 and >89?

so that we could handle multiple layers of "2×2×2 downsampling/upsampling with strides of two" conveniently.

The parameter spatial_window_size of [label] should be same with the one of [images] or not?

Not for the 3D-Unet, please check out the original release for more info: https://lmb.informatik.uni-freiburg.de/resources/opensource/unet.en.html
and the script for image/feature dims: https://lmb.informatik.uni-freiburg.de/resources/opensource/findsize.m

[YilinLiu97]

Hi@wyli, with the cropped version, the border has to be specified to 44, which makes the inference too long and results in bad performance. Do you have any suggestion on this? Thank you.

[NicoYuCN]

Does it mean the maximum size of input volumetric images is no larger than [132 132 116]? If so, it makes the NiftyNet is not so flexiable.