Motion Correction and Volumetric Image Reconstruction of 2D Ultra-fast MRI
NiftyMIC is a Python-based open-source toolkit for research developed within the GIFT-Surg project to reconstruct an isotropic, high-resolution volume from multiple, possibly motion-corrupted, stacks of low-resolution 2D slices. The framework relies on slice-to-volume registration algorithms for motion correction and reconstruction-based Super-Resolution (SR) techniques for the volumetric reconstruction.
The algorithm and software were developed by Michael Ebner at the Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London (UCL).
If you have any questions or comments, please drop an email to
NiftyMIC applied to Fetal Brain MRI
Given a set of low-resolution, possibly motion-corrupted, stacks of 2D slices, NiftyMIC produces an isotropic, high-resolution 3D volume. As an example, we illustrate its use for fetal MRI by computing a high-resolution visualization of the brain for a neck mass subject. Standard clinical HASTE sequences were used to acquire the low-resolution images in multiple orientations. The associated brain masks for motion correction were obtained using the automatic segmentation tool fetal_brain_seg.
Figure 1. NiftyMIC -- a volumetric MRI reconstruction tool based on rigid slice-to-volume registration and outlier-robust super-resolution reconstruction steps -- applied to fetal brain MRI.
Figure 2. Qualitative comparison of the original low-resolution input data and the obtained high-resolution volumetric reconstructions in both the original patient-specific and standard anatomical orientations. Five input stacks (two axial, one coronal and two sagittal) were used.
Several methods have been implemented to solve the Robust Super-Resolution Reconstruction (SRR) problem
to obtain the (vectorized) high-resolution 3D MRI volume from multiple, possibly motion corrupted, low-resolution stacks of (vectorized) 2D MR slices with for for a variety of regularizers and data loss functions . The linear operator represents the combined operator describing the (rigid) motion , the blurring operator and the downsampling operator .
The toolkit relies on an iterative motion-correction/reconstruction approach whereby complete outlier rejection of misregistered slices is achieved by iteratively solving the SRR problem for a slice-index set containing only slices that are in high agreement with their simulated counterparts projected from a previous high-resolution iterate according to a similarity measure and parameter . In the current implementation, the similarity measure corresponds to Normalized Cross Correlation (NCC).
The provided data loss functions are motivated by SciPy and allow for additional robust outlier handling during the SRR step. Implemented data loss functions are:
The available regularizers include
- Zeroth-order Tikhonov (TK0):
- First-order Tikhonov (TK1):
- Isotropic Total Variation (TV):
- Huber Function:
Additionally, the choice of finding optimal reconstruction parameters is facilitated by the Numerical Solver Library (NSoL).
NiftyMIC supports medical image registration and volumetric reconstruction for ultra-fast 2D MRI. It has not been tested on larger cohort data yet. NiftyMIC is not intended for clinical use.
How to cite
If you use this software in your work, please cite
- Ebner, M., Wang, G., Li, W., Aertsen, M., Patel, P. A., Melbourne, A., Doel, T., David, A. L., Deprest, J., Ourselin, S., & Vercauteren, T. (2018). An Automated Localization, Segmentation and Reconstruction Framework for Fetal Brain MRI. In Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2018 (pp. 313–320). Springer.
- Ebner, M., Chung, K. K., Prados, F., Cardoso, M. J., Chard, D. T., Vercauteren, T., & Ourselin, S. (2018). Volumetric reconstruction from printed films: Enabling 30 year longitudinal analysis in MR neuroimaging. NeuroImage, 165, 238–250.
NiftyMIC was developed in Ubuntu 16.04 and Mac OS X 10.12 and tested for Python 2.7.12 and 3.5.2.
It builds on a couple of additional libraries developed within the GIFT-Surg project including
whose installation requirements need to be met. Therefore, the local installation comes in three steps:
Alternatively, NiftyMIC can also be used as virtual machine via VirtualBox without having to go through all the manual installation steps. The respective information can be found here.
Provided the input MR image data in NIfTI format (
nii.gz), NiftyMIC can reconstruct an isotropic, high-resolution volume from multiple, possibly motion-corrupted, stacks of low-resolution 2D slices.
A recommended workflow is [associated applications in square brackets]:
- Segmentation of the anatomy of interest for all input images. For fetal brain MRI reconstructions, we recommend the use of the fully automatic segmentation tool fetal_brain_seg for this step.
- Bias-field correction [
- Volumetric reconstruction in subject space using two-step iterative approach based on rigid slice-to-volume registration and SRR cycles [
In case reconstruction in a template space is desired (like for fetal MRI) additional steps could be:
- Register obtained SRR to template and update respective slice motion corrections [
- Volumetric reconstruction in template space [
Additional information on how to use NiftyMIC and its applications is provided in the following.
Volumetric MR Reconstruction from Motion Corrupted 2D Slices
Leveraging a two-step registration-reconstruction approach an isotropic, high-resolution 3D volume can be generated from multiple stacks of low-resolution slices.
An example for a basic usage reads
niftymic_reconstruct_volume \ --filenames path-to-stack1.nii.gz ... path-to-stackN.nii.gz \ --filenames-masks path-to-stack1_mask.nii.gz ... path-to-stackN_mask.nii.gz \ --output path-to-srr.nii.gz \
whereby complete outlier removal during SRR is activated by default (
A more elaborate example could be
niftymic_reconstruct_volume \ --filenames path-to-stack1.nii.gz ... path-to-stackN.nii.gz \ --filenames-masks path-to-stack1_mask.nii.gz ... path-to-stackN_mask.nii.gz \ --alpha 0.01 \ --outlier-rejection 1 \ --threshold-first 0.5 \ --threshold 0.85 \ --intensity-correction 1 \ --isotropic-resolution 0.8 \ --two-step-cycles 3 \ --output path-to-output-dir/srr.nii.gz \ --subfolder-motion-correction motion_correction \ # created in 'path-to-output-dir' --verbose 1
The obtained motion-correction transformations in
motion_correction can be used for further processing, e.g. by using
niftymic_reconstruct_volume_from_slices.py to solve the SRR problem for a variety of different regularization and data loss function types.
Transformation to Template Space
If a template is available, it is possible to obtain a SRR in its associated standard anatomical space. Using the subject-space SRR outcome of
niftymic_reconstruct_volume a rigid alignment step maps all slice motion correction transformations accordingly using
niftymic_register_image \ --fixed path-to-template.nii.gz \ --fixed-mask path-to-template_mask.nii.gz \ --moving path-to-subject-space-srr.nii.gz \ --moving-mask path-to-subject-space-srr_mask.nii.gz \ --dir-input-mc dir-to-motion_correction \ --output path-to-registration-transform.txt \
- Gholipour, A., Rollins, C. K., Velasco-Annis, C., Ouaalam, A., Akhondi-Asl, A., Afacan, O., Ortinau, C. M., Clancy, S., Limperopoulos, C., Yang, E., Estroff, J. A. & Warfield, S. K. (2017). A normative spatiotemporal MRI atlas of the fetal brain for automatic segmentation and analysis of early brain growth. Scientific Reports 7, 476.
and abide by the license agreement as described in
SRR Methods for Motion Corrected (or Static) Data
After performed/updated motion correction (or having static data in the first place) several options are available:
- Volumetric reconstruction in template space
- Parameter tuning for SRR:
- different solvers and regularizers can be used to solve the SRR problem for comparison
- parameter studies can be performed to find optimal reconstruction parameters.
SRR from Motion Corrected (or Static) Slice Acquisitions
Solve the SRR problem for motion corrected data (or static data if
--dir-input-mc is omitted):
niftymic_reconstruct_volume_from_slices \ --filenames path-to-stack1.nii.gz ... path-to-stackN.nii.gz \ --filenames-masks path-to-stack1_mask.nii.gz ... path-to-stackN_mask.nii.gz \ --dir-input-mc dir-to-motion_correction \ # optional --output path-to-srr.nii.gz \ --reconstruction-type TK1L2 \ --reconstruction-space path-to-template.nii.gz \ # optional --alpha 0.01
niftymic_reconstruct_volume_from_slices \ --filenames path-to-stack1.nii.gz ... path-to-stackN.nii.gz \ --dir-input-mc dir-to-motion_correction \ --output path-to-srr.nii.gz \ --reconstruction-type HuberL2 \ --alpha 0.003
Slices that were rejected during the
niftymic_reconstruct_volume run are recognized as outliers based on the content of
dir-input-mc and will not be incorporated during the volumetric reconstruction.
Parameter Studies to Determine Optimal SRR Parameters
The optimal choice for reconstruction parameters like the regularization parameter or data loss function can be found by running parameter studies. This includes L-curve studies and direct comparison against a reference volume for various cost functions. In case a reference is available, similarity measures are evaluated against this "ground-truth" as well.
niftymic_run_reconstruction_parameter_study \ --filenames path-to-stack1.nii.gz ... path-to-stackN.nii.gz \ --filenames-masks path-to-stack1_mask.nii.gz ... path-to-stackN_mask.nii.gz \ --dir-input-mc dir-to-motion_correction \ --dir-output dir-to-param-study-output \ --reconstruction-type TK1L2 \ --reconstruction-space path-to-reconstruction-space.nii.gz \ # define reconstruction space --alphas 0.005 0.01 0.02 0.05 0.1 # regularization parameters to sweep through --append # if given, append a previously performed parameter study in output directory (if available)
niftymic_run_reconstruction_parameter_study \ --filenames path-to-stack1.nii.gz ... path-to-stackN.nii.gz \ --filenames-masks path-to-stack1_mask.nii.gz ... path-to-stackN_mask.nii.gz \ --dir-input-mc dir-to-motion_correction \ --dir-output dir-to-param-study-output \ --reconstruction-type HuberL2 \ --reference path-to-reference-volume.nii.gz \ # in case reference ("ground-truth") is available (reconstruction space is defined by this reference) --measures MAE RMSE PSNR NCC NMI SSIM \ # evaluate reconstruction similarities against reference --reference-mask path-to-reference-volume_mask.nii.gz \ # if given, evaluate similarities (--measures) on masked region only --alphas 0.001 0.003 0.005 0.001 0.003 \ # regularization parameters to sweep through --append # if given, append a previously performed parameter study in output directory (if available)
The results can be assessed by accessing the NSoL-script
niftymic_show_parameter_study \ --dir-input dir-to-param-study-output \ --study-name TK1L2 \ --dir-output-figures dir-to-figures
Licensing and Copyright
This work is partially funded by the UCL Engineering and Physical Sciences Research Council (EPSRC) Centre for Doctoral Training in Medical Imaging (EP/L016478/1), the Innovative Engineering for Health award (Wellcome Trust [WT101957] and EPSRC [NS/A000027/1]), and supported by researchers at the National Institute for Health Research University College London Hospitals (UCLH) Biomedical Research Centre.
Associated publications are
- [EbnerWang2018] Ebner, M., Wang, G., Li, W., Aertsen, M., Patel, P. A., Melbourne, A., Doel, T., David, A. L., Deprest, J., Ourselin, S., & Vercauteren, T. (2018). An Automated Localization, Segmentation and Reconstruction Framework for Fetal Brain MRI. In Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2018 (pp. 313–320). Springer
- [Ebner2018] Ebner, M., Chung, K. K., Prados, F., Cardoso, M. J., Chard, D. T., Vercauteren, T., & Ourselin, S. (2018). Volumetric reconstruction from printed films: Enabling 30 year longitudinal analysis in MR neuroimaging. NeuroImage, 165, 238–250.
- [Ranzini2017] Ranzini, M. B., Ebner, M., Cardoso, M. J., Fotiadou, A., Vercauteren, T., Henckel, J., Hart, A., Ourselin, S., and Modat, M. (2017). Joint Multimodal Segmentation of Clinical CT and MR from Hip Arthroplasty Patients. MICCAI Workshop on Computational Methods and Clinical Applications in Musculoskeletal Imaging (MSKI) 2017.
- [Ebner2017] Ebner, M., Chouhan, M., Patel, P. A., Atkinson, D., Amin, Z., Read, S., Punwani, S., Taylor, S., Vercauteren, T., and Ourselin, S. (2017). Point-Spread-Function-Aware Slice-to-Volume Registration: Application to Upper Abdominal MRI Super-Resolution. In Zuluaga, M. A., Bhatia, K., Kainz, B., Moghari, M. H., and Pace, D. F., editors, Reconstruction, Segmentation, and Analysis of Medical Images. RAMBO 2016, volume 10129 of Lecture Notes in Computer Science, pages 3–13. Springer International Publishing.