Official repository for the paper: "Development and Validation of an Automated Image-Based Deep Learning Platform for Sarcopenia Assessment in Head and Neck Cancer", published in JAMA Network Open 2023;6(8),e2328280.
Keywords: Sarcopenia, Head and Neck Cancer, Deep Learning, Convolutional Neural Network, CT
Estimated segmentations with varying Dice similarity coefficient (DSC) values (greater than, equal to, or less than the median DSC) were overlaid on axial CT images to show their similarities to ground truth segmentations for validation and internal test sets. Corresponding difference (Δh) between the estimated C3 section slice and ground truth slice and DSC values are annotated for each case in the validation and internal test sets. Model-estimated C3 slices on sagittal CT images and segmentation on axial CT images are also shown for the external test set. Acceptability scores from expert clinicians’ review are annotated for corresponding cases in the external test set.- DenseNet-163
- 3D U-Net
U-Net Segmentation Model Performance
- Python 3.8
- TensorFlow 2.4
- Keras
This code was developed and tested using Python 3.8.5.
For the code to run as intended, all the packages under requirements.txt should be installed. In order not to break previous installations and ensure full compatibility, it's highly recommended to create a virtual environment to run the DeepContrast pipeline in. Here follows an example of set-up using python virtualenv:
- install python's virtualenv
sudo pip install virtualenv
- parse the path to the python3 interpreter
export PY2PATH=$(which python3)
- create a virtualenv with such python3 interpreter named "venv" (common name, already found in .gitignore)
virtualenv -p $PY2PATH venv
- activate the virtualenv
source venv/bin/activate
At this point, (venv) should be displayed at the start of each bash line. Furthermore, the command which python3 should return a path similar to /path/to/folder/venv/bin/python3. Once the virtual environment is activated:
- once the virtualenv is activated, install the dependencies
pip install -r requirements.txt
At this stage, everything should be ready for the data to be processed by the DeepContrast pipeline. Additional details can be found in the markdown file under src.
The virtual environment can be deactivated by running:
deactivate
The codebase for auto-segmenting the C3 skeletal muscle mass and for calculating muscle cross-sectional area (CSA), and Skeletal Mass Index (SMI) from 3D CT images.
Consists of two deep-learning models: Slice Selection Model and Segmentation Model. The Slice Selection Model is adapted from DenseNet and the Segmentation model is adapted from UNet.
Model Weights are available in the "model/test" folder.
-
Download the model weights from the google drive link provided above, unzip the files, and save them in the 'model' subfolder. Please note that the CV folder within the zip file contains five-fold cross-validation sub-models for segmentation.
-
Make sure you set the env properities (pass as command line arguments to main.py script) <proj_dir> - Project Directories Path where this code is copied - instuition name of curated dataset e.g. BWH , default value is "test"
-
The input files are stored in the following folder
Raw scans - /data//img/
-
Following are default directories/paths that get created to stroe the pre-processed files, slice selection output as segmentation output. If you prefer, you can also set them using env variables <pre_process_dir>, <slice_csv_path> and <output_dir>. Passing them to main.py as command line arguments is also accepted.
Pre-processed Scans - /data//preprocessed Slice Selection Output CSV - /data//_C3_top_slice_pre.csv Segmentation Model Output - /data//output_segmentations
Main script executes pre-processing, slice selction and segmentation sequentially
STEP argument can be used to execute each of the above steps individually. The arguments takes values ALL, SLICE, PREPROCESS, SEGMENT
Other scripts provided here OPTIONAL, used to generate the needed CSA information, generates statistics, visualizations and final clinical file needed combining patient meta information with image segmenation data
The Main script step can be run by executing:
main.py
This script generates the Cross-Sectional Area (CSA) of the C3 Skeletal Muscle Mass from the auto-segmented masks.
Set the proj_dir, raw_img_dir, raw_seg_dir, Slice Prediction CSV, and output_dir directories before executing the code.
- Input#1: Auto segmentation files
- Input#2: Manual segmentation files of test data
- Input#3: C3_Top_Slice_Prediction.csv
- Output: CSA in a CSV file
The Calculate C3 Cross Sectional Area step can be run by executing:
get_CSA.py
This script generates the masks with contours for easy visualization.
Set the proj_dir, raw_img_dir, raw_seg_dir, Slice Prediction CSV, and output_dir directories before executing the code.
- Input#1: Auto segmentation files
- Input#3: C3_Top_Slice_Prediction.csv
- Output: Segmentation files with contours for visualizing the masks
The Visualize C3 Segmentation function can be run by executing:
visualize.py
This script generates the C3 Skeletal Muscle Index (SMI) and translates that to the corresponding L3 Skeletal Muscle Index (SMI).
Set the proj_dir, raw_img_dir, raw_seg_dir, Slice Prediction CSV, Clinical Meta-data files and output_dir directories before executing the code.
- Input#1: Auto segmentation files
- Input#2: Cross-Sectional Area CSV file
- Input#3: C3_Top_Slice_Prediction.csv
- Input#4: Patient Clinical meta information in a CSV file
- Output: SMI data in a CSV file
The Calculate C3 Skeletal Muscle Index (SMI) step can be run by executing:
clinical.py
Please cite the following article if you use this code or pre-trained models:
@article{Ye2023DeepSarcopenia,
title = {Development and Validation of an Automated Image-Based Deep Learning Platform for Sarcopenia Assessment in Head and Neck Cancer},
author = {Z Ye, A Saraf, Y Ravipati, F Hoebers, P Catalano, Y Zha, A Zapaishchykova, J Likitlersuang, C Guthier, R Tishler, J Schoenfeld, D Margalit, R Haddad, R Mak, M Naser, K Wahid, J Sahlsten, J Jaskari, K Kaski, A Mäkitie, C Fuller, H Aerts, and B Kann},
journal = {JAMA Network Open 6 (8), e2328280},
year = {2023},
}
Code development, testing, refactoring and documentation: YR and ZY.
The code and data of this repository are provided to promote reproducible research. They are not intended for clinical care or commercial use.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.