A novel deep learning model has been developed to classify the Venous Gas Emboli (VGE) grade in Doppler ultrasound audio recordings. To date, this is the first attempt to incorporate deep learning model into VGE detection and classification in audio Doppler ultrasound.
First, several hours of synthetic data were generated and collected according to Spencer scale in the laboratory settings. Then, the synthetic data were superimposed with cardiac signal to simulate the behavior of real audio Doppler ultrasound. Later, a novel squeeze-and-excitation deep learning model is designed to effectively classify VGE grade. A large amount of real data was annotated recently and used to fine-tune the developed model for practical purposes.
The synthetic phase of the developed model including synthetic data generation and model training is depicted in the following figure:
The architecture of the designed model is shown below:
Designed 1-D network for VGE grade classfication. Figure reprinted with permission [1].
The software requires Python >= 3.9 for best performance. Create an environment in your Anaconda using the following command:
conda create --name VGE_Doppler python==3.9
You need to install Keras and Tensorflow properly for training/inference on GPU. The developed MATLAB codes are based on R2021b version. All the required Python packages can be installed using the following command:
$ pip install -r requirements.txt
- Synthetic data organization/division
The synthetic data after collection is prepared and organized in MATLAB. First, the model is trained on synthetic data. Before running the code, you are required to organize the dataset into training, validation, and test sets. This step can be done by the MATLAB code called Train_test_validation_division.m, where the data folder must be set correctly for this MATLAB code. An example of data folder structure is shown below:
- Pre-processing steps for real data
The real audio Doppler recordings vary in duration. The very first step required to take is to reshape the real data to make it compatible with model developed for synthetic data. Moreover, the sampling frequency needs to be set to 8 kHz to consider 10-seconds of audio signal as input. These pre-processing steps are written in MATLAB code called Real_data_organzier.m, where the correct location of real audio Doppler recordings must be set before running the code. To date, the available data for fine-tuning step is depicted in the following figure:
Similar to synthetic data organization, the orgnazied real data must be split into Training, Validation, Test sets accordingly. The MATLAB code Train_test_validation_division_real_data.m is written to perform this division.
- Running the model
The Python code called AudioDopplerAutomaticVenousGasEmboliClassification.ipynb is written in Jupyter Notebook IDE. To make the usage of code easier, the code has been section-wise. It is critical to set all the required paths correctly before running the code.
- Synthetic data results
The 10-fold cross validation result on synthetic data is shown below:
5-class classification accuracy:
| VGE Grade | Ordinal Accuracy |
|---|---|
| Grade 0 | 97.8% |
| Grade 1 | 91.0% |
| Grade 2 | 93.6% |
| Grade 3 | 99.6% |
| Grade 4 | 99.4% |
| Average | 96.4% |
Binary classification accuracy (low VGE grade (0 - 2) vs. high VGE grade (3 and 4):
| VGE Level | Accuracy |
|---|---|
| Low grade | 99.6% |
| High grade | 99.0% |
| Average | 99.2% |
- Real data results
The confusion matrix for precordial region and subclavian veins are shown below (left: precordial, right: subclavian):
5-class classification accuracy:
Figure reprinted with permission [1].
The codes are licensed under GPL-2.0 license.
For any utilization of the code content of this repository, the following paper needs to be cited by the user:
- A. Azarang*, D.Q. Le*, A. Hoang, L. Blogg, P.A. Dayton, R. Lance, M. Natoli, A. Gatrell, Frauke Tillmans, R.E. Moon, P. Lindholm, and V. Papadopoulou, "Deep Learning-Based Venous Gas Emboli Grade Classification in Doppler Ultrasound Audio Recordings", IEEE Transactions on Biomedical Engineering, in press.
*Authors contributed equally
For bug reports and feature requests please contact us at: aazarang@unc.edu.