The code in this repo accompanies the paper 'BayesGrad: A neural network explainability framework to assist diagnosis of coronary artery disease from PET myocardial perfusion imaging'.
This repository contains the following directories:
- Contains functions that can be reused across the code
- Contains csv files with vessel segmentation masks for each of the LAD, RCA, and LCX separately as well as combined. The masks for individual vessels were originally created in flowquant based on a 20 ring scanner, and they were combined in Python.
- Contains the results notebook where results figures are created
- Contains various output generated from running code including:
- saved saliency maps generated across 20 model retrainings
- saved models for detection and for localization (used as baseline in topK predictions) over 20 model retrainings
- results and artifacts of top K prediciton experiment
- results and artifacts related to performance comparison between deterministic and Bayesian models
-
Contains 4 scripts:
- train_VGGDrop_norm_abn.py
- Trains the detection model and the supervised localization model that is used as a baseline comparison in topK prediction
- make_saliency_map_generator_arg_file.py
- A script to create the arg file necessary to run the SLURM job array that generates test set saliency maps for all 20 model retrainings
- saliency_map_generator.py
- Generates and saves attention maps to be used in later analysis
- evaluate_performance.py
- Evaluates the Bayesian models and deterministic baselines in terms of AUC, accuracy, and ECE
- train_VGGDrop_norm_abn.py
-
The directory SLURM_scripts contains SLURM launch scripts to run the scripts described above with the appropriate parameters.
- run_train_VGGDrop_norm_abn_ja.sh starts a SLURM job array that runs 20 model retrainings of the detection model with the appropriate hyperparameters selected during hyperparameter tuning.
- run_train_VGGDrop_norm_abn_loc_ja.sh starts a SLURM job array that runs 20 model retrainings of the supervised localization model that is used as a baseline for comparison in topK predict with the appropriate hyperparameters selected during hyperarameter tuning.
- run_saliency_map_generator.sh starts a SLURM job array that generates and saves saliency maps using the BayesGrad Var method for the test set for all 20 models trained in the detection problem. This relies on being passed the appropriate arguments for the job array which are stored in /scripts/arg_files/saliency_map_generator_20_trials_args.txt.
- run_evaluate_performance.sh launches the SLURM job to evaulate all 20 Bayesian models and their deterministic counterparts in the detection task.
- xnn4rad.def
- Singularity def file used to build xnn4rad.sif Singularity environment
- Create singularity environment from Singularity/xnn4rad.def
- Data must be prepared and split in the format discussed in the following paper and its accompanying code: Machine and deep learning models for accurate diagnosis of coronary artery disease with myocardial blood flow PET imaging
The following SLURM launch scripts are located in scripts/SLURM_scripts
- Run run_train_VGGDrop_norm_abn_ja.sh to launch the job array that trains 20 instances of the detection model trained to predicts only normal/abnormal, and from which attention maps will later be generated. Optimal hyperparameters identified in hyperparameter tuning are included in this script.
- Run run_train_VGGDrop_norm_abn_loc_ja.sh to launch the job array that trains 20 instances of the localization model which is used as a baseline comparison method for localization predictions derived from attention maps in top K predict. Optimal hyperparameters identified in hyperparameter tuning are included in this script.
- Within a SLURM launch script, run make_saliency_map_generator_arg_file.py in order to generate the argument file needed to launch saliency_map_generator.py in a job array which will generate test set saliency maps for all 20 trained detection model instances.
- Run run_saliency_map_generator.sh in order to launch the job array which will generate test set saliency maps for all 20 trained detection model instances.
- Run run_evaluate performance.sh in order to evaluate detection performance for all 20 instances of the Bayesian models and deterministic baselines in terms of AUC, accuracy, and ECE.
- Run through the steps in notebooks/Results.ipynb in order to regenerate results figures
©2022 The MITRE Corporation and The Ottawa Heart Institute Approved for Public Release; Distribution Unlimited. Public Release Case Number 22-1847
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.