Tutorial cascaded architecture (#389)

* update download dataset with csf info * cascaded tuto: update description * add new tutorial in index tree * add prerequisite * config.json: change path to data * tuto1: add config flag to ivadomed command * finalize prerequisite * add description of task * add config object_detection_model * add all config params to modify * visualize data section * add CenterCrop parameter * add threshold * add images and minor fixes * tuto1: add reference to evaluate model section * change safety factor * add precision on centercrop preprocessing * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * modify learned features * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * add comment on safety factor 3rd dim * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * add precision on CenterCrop use * add Charley's suggestion on preprocessing explanation * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * remove warning about cpu and gpu * use os to create path * Update docs/source/tutorials/cascaded_architecture.rst Co-authored-by: charleygros <charley.gros@gmail.com> * correct typo * modify model name to respect convention * Correct model name according to convention: add mt in contrasts * add images * utils.py: PEP8 + change attribute self.output to self._output * add GIF in tutorial * add GIF * rename training_gif to training * update path to gif * change matplotlib version for gif Co-authored-by: charleygros <charley.gros@gmail.com>
ivadomed · Aug 12, 2020 · b78a545 · b78a545
1 parent 1244572
commit b78a545
Show file tree

Hide file tree

Showing 9 changed files with 172 additions and 15 deletions.
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -36,6 +36,7 @@ The purpose of the ``ivadomed`` project is to:
    :caption: Tutorials
 
    tutorials/one_class_segmentation_2d_unet.rst
+   tutorials/cascaded_architecture.rst
 
 .. toctree::
    :maxdepth: 1

diff --git a/docs/source/tutorials/cascaded_architecture.rst b/docs/source/tutorials/cascaded_architecture.rst
@@ -0,0 +1,158 @@
+Cascaded architecture
+=====================
+
+In this tutorial we will learn the following features:
+
+- Design a training scheme composed of two cascaded networks.
+- Visualize the training with tensorboard.
+- Generate a GIF to visualize the learning of the model.
+- Find the optimal threshold to binarize images based on the validation sub-dataset.
+
+In our example, the model will first locate the spinal cord (step 1). This localisation will then be used to crop the images around this region of interest, before segmenting the cerebrospinal fluid (CSF, step 2).
+
+.. _Prerequisite:
+
+Prerequisite
+------------
+
+In this tutorial, the spinal cord segmentation model generated from :doc:`../tutorials/one_class_segmentation_2d_unet`
+will be needed since it will be used for the step 1. Please make sure you did this tutorial prior to this present example and that a folder named ``seg_sc_t1-t2-t2s-mt`` is available.
+
+
+Configuration file
+------------------
+
+In this tutorial we will use the configuration file: ``ivadomed/config/config.json``.
+First off, copy this configuration file in your local directory to avoid modifying the source file:
+
+.. code-block:: bash
+
+   cp <PATH_TO_IVADOMED>/ivadomed/config/config.json .
+
+Then, open it with a text editor. As described in the tutorial :doc:`../tutorials/one_class_segmentation_2d_unet`, make
+sure the ``command`` is set to "train" and ``bids_path`` point to the location of the dataset. Below, we will discuss
+some of the key parameters to use cascaded models.
+
+- ``object_detection_params:object_detection_path``: Location of the object detection model. This parameter corresponds
+  to the prerequisite model path  to the trained model (see :ref:`Prerequisite`). This spinal cord segmentation model
+  will be applied to the images and a bounding box will be created around this mask to crop the image.
+
+  .. code-block:: xml
+
+     "object_detection_path": "<SPINAL_CORD_SEG_LOG_DIRECTORY>/spineGeneric/seg_sc_t1-t2-t2s-mt"
+
+- ``object_detection_params:safety_factor``: Multiplicative factor to apply to each dimension of the bounding box. To
+  ensure all the CSF is included, a safety factor should be applied to the bounding box generated from the spinal cord.
+  A safety factor of 200% on each dimension is applied on the height and width of the image. The original depth of the
+  bounding box is kept since the CSF should not be present past this border.
+
+  .. code-block:: xml
+
+     "safety_factor": [2, 2, 1]
+
+- ``loader_parameters:target_suffix``: Suffix of the ground truth segmentation. The ground truth are located under the
+  ``DATASET/derivatives/labels`` folder. The suffix for CSF labels in this dataset is ``_csfseg-manual``:
+
+  .. code-block:: xml
+
+     "target_suffix": ["_csfseg-manual"]
+
+- ``loader_parameters:contrast_params``: Contrast(s) of interest. The CSF labels are only available in T2w contrast in
+  the example dataset.
+
+  .. code-block:: xml
+
+     "contrast_params": {
+         "training_validation": ["T2w"],
+         "testing": ["T2w"],
+         "balance": {}
+     }
+
+- ``transformation:CenterCrop:size``: Crop size in voxel. Images will be cropped or padded to fit these dimensions. This
+  allows all the images to have the same size during training. Since the images will be cropped around the spinal cord,
+  the image size can be reduced to avoid large zero padding. The ``preprocessing`` parameter indicates this
+  transformation will only be applied once at the beginning of the training (i.e. not at each epoch as commonly done for
+  data augmentation operations).
+
+  .. code-block:: xml
+
+     "CenterCrop": {
+         "size": [64, 64],
+         "preprocessing": true
+     }
+
+Train model
+-----------
+
+Once the configuration file is ready, run the training. `ivadomed` has an option to find a threshold value which optimized the dice score on the validation dataset. This threshold will be further used to binarize the predictions on testing data. Add the flag `-t` with an increment
+between 0 and 1 to perform this threshold optimization (i.e. ``-t 0.1`` will return the best threshold between 0.1,
+0.2, ..., 0.9)
+
+To help visualize the training, the flag ``--gif`` or ``-g`` can be used. The flag should be followed by the number of
+slices by epoch to visualize. For example, ``-g 2`` will generate 2 GIFs of 2 randomly selected slices from the
+validation set.
+
+.. code-block:: bash
+
+   ivadomed -c config.json -t 0.01 -g 1
+
+At the end of the training, the optimal threshold will be indicated:
+
+.. code-block:: console
+
+   Running threshold analysis to find optimal threshold
+	Optimal threshold: 0.01
+	Saving plot: spineGeneric/roc.png
+
+
+Visualize training data
+-----------------------
+If the flag ``--gif`` or ``-g`` was used, the training can be visualized through gifs located in the folder
+<LOG_DIRECTORY>/gifs.
+
+.. figure:: ../../../images/training.gif
+   :width: 300
+   :align: center
+
+   Training visualization with GIF
+
+Another way to visualize the training is to use Tensorboard. Tensorboard helps to visualize the augmented input images,
+the model's prediction, the ground truth, the learning curves, and more. To access this data during or after training,
+use the following command-line:
+
+.. code-block:: bash
+
+   tensorboard --logdir <PATH_TO_LOG_DIRECTORY>
+
+The following should be displayed in the terminal:
+
+.. code-block:: console
+
+   Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
+   TensorBoard 2.2.1 at http://localhost:6006/ (Press CTRL+C to quit)
+
+Open your browser and type the URL provided, in this case ``http://localhost:6006/``.
+In the scalars folder, the evolution of metrics, learning rate and loss through the epochs can be visualized.
+
+.. image:: ../../../images/tensorboard_scalar.png
+   :align: center
+
+In the image folder, the training and validation ground truth, input images and predictions are displayed. With this
+feature, it is possible to visualize the cropping from the first model and confirm that the spinal cord
+was correctly located.
+
+.. image:: ../../../images/tensorboard_images.png
+   :align: center
+
+Evaluate model
+--------------
+- ``testing_parameters:binarize_prediction``: Threshold at which predictions are binarized. Before testing the model,
+  modify the binarization threshold to have a threshold adapted to the data:
+
+.. code-block:: xml
+
+    "binarize_prediction": 0.01
+
+
+To test and apply this model on the testing dataset, go to the `Evaluate model` section of the tutorial
+:ref:`One-class segmentation with 2D U-Net<evaluate model>`.
diff --git a/docs/source/tutorials/one_class_segmentation_2d_unet.rst b/docs/source/tutorials/one_class_segmentation_2d_unet.rst
@@ -15,20 +15,15 @@ Download dataset
 
 We will use a publicly-available dataset consisting of MRI data of the spinal cord. This dataset is a subset of the
 `spine-generic multi-center dataset <https://github.com/spine-generic/data-multi-subject>`_ and has been pre-processed
-to facilitate training/testing of a new model. Namely, for each subject, all six contrasts were co-registered together,
-and semi-manual cord segmentation label was created. More details
+to facilitate training/testing of a new model. Namely, for each subject, all six contrasts were co-registered together. Semi-manual cord segmentation for all modalities and manual cerebrospinal fluid labels for T2w modality were created. More details
 `here <https://github.com/ivadomed/ivadomed/blob/master/dev/prepare_data/README.md>`_.
 
 To download the dataset (~450MB), run the following commands in your terminal:
 
 .. code-block:: bash
 
    # Download data
-   curl -o ivadomed_example_spinegeneric.zip -L https://github.com/ivadomed/data_example_spinegeneric/releases/download/r20200907/data_spinegeneric_registered-r20200907.zip
-   unzip ivadomed_example_spinegeneric.zip
-   # Rename folder
-   mv ivadomed-data_example_spinegeneric* data_example_spinegeneric
-
+    ivadomed_download_data -d data_example_spinegeneric
 
 Configuration file
 ------------------
@@ -103,7 +98,7 @@ Once the configuration file is ready, run the training:
 
 .. code-block:: bash
 
-   ivadomed config.json
+   ivadomed -c config.json
 
 .. note::
 
@@ -158,6 +153,8 @@ on training and validation sets at every epoch. To know more about the meaning o
 After 100 epochs (see ``"num_epochs"`` in the configuration file), the Dice score on the validation set should
 be ~90%.
 
+.. _Evaluate model:
+
 Evaluate model
 --------------
 
@@ -172,7 +169,7 @@ Then run:
 
 .. code-block:: bash
 
-   ivadomed config.json
+   ivadomed -c config.json
 
 The model's parameters will be displayed in the terminal, followed by a preview of the results for each image.
 The resulting segmentation is saved for each image in the ``<log_directory>/pred_masks`` while a csv file,
@@ -220,4 +217,4 @@ The output and ground truth segmentations of the spinal cord are presented in re
 contrast T2w):
 
 .. image:: ../../../images/sc_prediction.png
-   :align: center
+   :align: center
diff --git a/images/tensorboard_images.png b/images/tensorboard_images.png
diff --git a/images/tensorboard_scalar.png b/images/tensorboard_scalar.png
diff --git a/images/training.gif b/images/training.gif
diff --git a/ivadomed/config/config.json b/ivadomed/config/config.json
@@ -2,13 +2,13 @@
     "command": "train",
     "gpu": 0,
     "log_directory": "spineGeneric",
-    "model_name": "seg_sc_t1_t2_t2s_mt",
+    "model_name": "seg_sc_t1-t2-t2s-mt",
     "debugging": false,
     "object_detection_params": {
         "object_detection_path": null
     },
     "loader_parameters": {
-        "bids_path": "data_spinegeneric_registered",
+        "bids_path": "data_example_spinegeneric",
         "target_suffix": ["_seg-manual"],
         "roi_params": {
             "suffix": null,

diff --git a/ivadomed/utils.py b/ivadomed/utils.py
@@ -2,25 +2,26 @@
 import os
 
 import matplotlib
-import matplotlib.pyplot as plt
 import matplotlib.animation as anim
+import matplotlib.pyplot as plt
 import nibabel as nib
 import numpy as np
 import onnxruntime
 import torch
 import torch.nn as nn
 import torch.nn.functional as F
 import torchvision.utils as vutils
+
 from ivadomed import models as imed_models
 from ivadomed import postprocessing as imed_postpro
 from ivadomed import transforms as imed_transforms
 from ivadomed.loader import utils as imed_loader_utils, loader as imed_loader
 from ivadomed.object_detection import utils as imed_obj_detect
+
 from scipy.ndimage import label, generate_binary_structure
 from torch.autograd import Variable
 from torch.utils.data import DataLoader
 from tqdm import tqdm
-import logging
 
 AXIS_DCT = {'sagittal': 0, 'coronal': 1, 'axial': 2}
 

diff --git a/requirements.txt b/requirements.txt
@@ -1,7 +1,7 @@
 bids-neuropoly>=0.2
 h5py
 joblib
-matplotlib>=3.0.3
+matplotlib>=3.3.0
 nibabel
 onnxruntime
 pandas