fastMRI reproducible benchmark

The idea of this repository is to have a way to rapidly benchmark new solutions against existing reconstruction algorithms on the fastMRI dataset single-coil track. The reconstruction algorithms implemented or adapted to the fastMRI dataset include to this day:

• zero filled reconstruction
• LORAKS, using the LORAKS Matlab toolbox
• Wavelet-based reconstruction (i.e. solving an L1-based analysis formulation optimization problem with greedy FISTA), using pysap-mri
• U-net
• DeepCascade net
• KIKI net
• Learned Primal Dual, adapted to MRI reconstruction
• XPDNet, a modular unrolled reconstruction algorithm, in which you can plug your best denoiser.
• NCPDNet, an unrolled reconstruction algorithm for non-Cartesian acquisitions, with density-compensation.

All the neural networks are implemented in TensorFlow with the Keras API. The older ones (don't judge this was the beginning of my thesis) are coded using the functional API. The more recent ones are coded in the subclassed API and are more modular. For the LORAKS reconstruction, you will not be able to reconstruct the proper fastMRI data because of the A/D oversampling.

Examples

Various examples are available in the examples folder.

• The mri_reconstruction_intro.ipynb notebook compares the performance of classical wavelet-based reconstruction (using the pysap-mri package) and deep learning based approaches like the CascadeNet and the UPDNet on a slice from the fastMRI single coil dataset in a 2D Cartesian setting.
• The non_cartesian_reconstruction.ipynb notebook showcases the use of the NCPDNet.

These examples can be run in binder.

Pretrained model checkpoints

Some model checkpoints are available in the HuggingFace Hub. You can either download them manually, clone the model repository, or use the huggingface-hub package like:

from huggingface_hub import hf_hub_download

REPO_ID_BASE = 'zaccharieramzi/{model_name}'
repo_id = REPO_ID_BASE.format(model_name='... the model name')
model_weights_path = hf_hub_download(
    repo_id=repo_id,
    filename='model_weights.h5',
)

Use cases for each model are explained in the respective model cards on the HuggingFace Hub under the "How to use" section.

Reconstruction settings

The main reconstruction settings are Cartesian single-coil and multi-coil 2D reconstruction with random and "periodic sampling". These settings are covered by almost all the networks in this repo, mainly because they are the settings of the fastMRI challenge.

Other reconstruction settings

Non-cartesian: you can reconstruct non-Cartesian data using the NCPDNet. It relies on the TensorFlow implementation of the NUFFT, tfkbnufft. This network will allow you to work on 2D single-coil and multi-coil data, as well as 3D single-coil data.

Deep Image Prior: you can also reconstruct non-Cartesian data in an untrained fashion using the DIP model. This idea originated from the Deep Image Prior paper, and was later adapted to MRI reconstruction by different works (Accelerated MRI with untrained Neural networks, Time-Dependent Deep Image Prior for Dynamic MRI). It currently is only used for 2D non-Cartesian data (primarily for computation time reasons), but you can extend it easily to 2D Cartesian data and 3D (PRs welcome).

How to train the neural networks

The scripts to train the neural networks are located in fastmri_recon/training_scripts/. You just need to install the package and its dependencies:

pip install . &&\
pip install -r requirements.txt

How to write a new neural network for reconstruction

The simplest and most versatile way to write a neural network for reconstruction is to subclass the CrossDomainNet class. An example is the PDnet

Reproducing the results of the paper

To reproduce the results of the paper for the fastMRI dataset, run the following script (here for the PD contrast):

python fastmri_recon/evaluate/scripts/paper_eval.py --contrast CORPD_FBK

To reproduce the results of the paper for the OASIS dataset, run the following script (here for fewer samples):

python fastmri_recon/evaluate/scripts/paper_eval_oasis.py --n-samples 100

Finally, to reproduce the figures of the paper, you will need to use the qualitative_validation_for_net notebook.

Downloading the model checkpoints

The model checkpoints are stored in the HuggingFace Hub. You can download them using the following script, which will automatically put them in the correct directory (for example here the fastMRI models):

python fastmri_recon/evaluate/scripts/download_checkpoints.py

Data requirements

fastMRI

The fastMRI data must be located in a directory whose path is stored in the FASTMRI_DATA_DIR environment variable. It can be downloaded on the official website after submitting a request (bottom of the page).

The package currently supports public single coil and multi coil knee data.

OASIS

The OASIS data must be located in a directory whose path is stored in the OASIS_DATA_DIR environment variable. It can be downloaded on the XNAT store after creating an account. You can for example use the ZIP download available on this page. The project is OASIS3. The whole list of sessions that need to be downloaded is available in OASIS_list.csv.

Citation

This work will be presented at the International Symposium on Biomedical Imaging (ISBI) in April 2020. An extended version has been published in MDPI Applied sciences. If you use this package or parts of it, please cite one of the following work:

• Benchmarking Deep Nets MRI Reconstruction Models on the FastMRI Publicly Available Dataset
• Benchmarking MRI Reconstruction Neural Networks on Large Public Datasets
• XPDNet for MRI Reconstruction: an Application to the fastMRI 2020 Brain Challenge
• Density Compensated Unrolled Networks for Non-Cartesian MRI Reconstruction

Applications

This package has been used to perform MRI reconstruction in the following projects (in addition to the ones mentioned above):

• Is good old GRAPPA dead?
• Learning the sampling density in 2D SPARKLING MRI acquisition for optimized image reconstruction
• Denoising Score-Matching for Uncertainty Quantification in Inverse Problems