DFCAN 3D

[akas0018]

What changes would one need to make to the DFCAN 2D notebook for it to be able to handle 3D datasets. I see that the key components are already written for handling 3D data. Has anyone given this a try ?

[esgomezm]

Maybe @iarganda or @IvanHCenalmor know a bit more about it?

[IvanHCenalmor]

Hi @akas0018,

I have been looking the at DFCAN 2D notebook and in order to make the notebook to be able to handle 3D datasets some changes in the code are needed. Those changes can be divided in two.

First of all, even if the key components are written, the final model and some components need some changes:

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 497 in 77feded

"def FCALayer(input, channel, size_psc, reduction=16):\n",

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 508 in 77feded

"def FCAB(input, channel, size_psc):\n",

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 518 in 77feded

"def ResidualGroup(input, channel, size_psc, n_RCAB = 4):\n",

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 526 in 77feded

"def DFCAN(input_shape, scale=4, n_ResGroup = 4, n_RCAB = 4, pretrained_weights=None):\n",

These components need to be adapted to the changes explained in the Note 6 from the Supplementary information of the original paper. Therefore a nearly new model has to be implemented.

Secondly, the function that create_random_patches which is defined in

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 310 in 77feded

"def create_random_patches( lr_path, hr_path, file_names, scale, num_patches,\n",

and then used in

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 1730 in 77feded

"train_patches_wf, train_patches_gt = create_random_patches( Training_source,\n",

has to be adapted. The following lines of code are only adapted to create 2D patches:

  input_patches.append(lr_img[ r : r + lr_shape[0],  
                               c : c + lr_shape[1] ] ) 
  output_patches.append(hr_img[ r*scale : (r + lr_shape[0])*scale, 
                                c*scale : (c + lr_shape[1])*scale ] )`

I don't really know how the patches work in 3D data, if the third dimension is also reduced for the patch or if it is completely taken but those lines need to be changed.

Finally, the code cell from section 4.1 has two lines of code related with the input and output shapes, that should be changed:

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 1750 in 77feded

"input_shape = ( train_patches_wf[0].shape[0], train_patches_wf[0].shape[1], 1 ) # 128x128x1\n",

ZeroCostDL4Mic/Colab_notebooks/Beta notebooks/DFCAN_ZeroCostDL4Mic.ipynb

Line 1756 in 77feded

"output_shape = ( train_patches_gt[0].shape[0], train_patches_gt[0].shape[1] ,1 ) # 256x256x1\n",

At this moment, they are also fixed to two dimensions with a single channel, this should be adapted to your data or make it modular with something like, for example:

input_shape = train_patches_wf[0].shape[1:] + (1,)
output_shape = train_patches_gt[0].shape[1:] + (1,)

A desired input shape (the one that you should try to obtain) based on tf.keras.layers.Conv3D should be 5+D tensor with shape: batch_shape + (channels, conv_dim1, conv_dim2, conv_dim3) if data_format='channels_first' or 5+D tensor with shape: batch_shape + (conv_dim1, conv_dim2, conv_dim3, channels) if data_format='channels_last'.

This would only be until section 4 included, for sections 5 and 6 maybe other changes should be made for the reading of the 3D data, but the changes would be similar to the ones described.

I hope this helps you if you desire to implement a 3D version of the DFCAN notebook :)

[esgomezm]

@akas0018 did this help?