Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 6 additions & 12 deletions tutorials/cardiac-hemodynamics-assessment/tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,13 @@
"\n",
"In this tutorial, we demonstrate how to use low-cost, non-invasive modalities **Chest X-ray (CXR)** and **12-lead Electrocardiogram (ECG)** to assess **Cardiothoracic Abnormalities**.\n",
"\n",
"**Estimated runtime:** Completing the steps in this tutorial will take approximately 8 minutes.\n",
"**Estimated runtime:** Completing the steps in this tutorial will take approximately 10 minutes.\n",
"\n",
"## Problem Formulation\n",
"\n",
"We will use a multimodal dataset derived from MIMIC-CXR and MIMIC-IV-ECG, which contains approximately 50K paired CXR and ECG samples. In this tutorial, we pretrain a multimodal **CardioVAE** model using \\~49K CXR-ECG pairs via a tri-stream pretraining method. We then fine-tune this pretrained CardioVAE model on a smaller subset (\\~1K paired samples) with binary labels: **Healthy** and **Cardiothoracic Abnormality**. Lastly, we demonstrate how to interpret the trained CardioVAE model on both the CXR and ECG modalities.\n",
"\n",
"This notebook is based on the work of **Suvon et al. (MICCAI 2024)**, which introduced a tri-stream pretraining strategy using a **Multimodal Variational Autoencoder (VAE)** to learn both modality-shared and modality-specific representations for assessing **Pulmonary Arterial Wedge Pressure (PAWP)**—a critical indicator of cardiac hemodynamics. The resulting model, **CardioVAE**, is implemented in the [PyKale](https://github.com/pykale/pykale) library. Here, we provide a concise example of how to use this model through PyKale's APIs—from pretraining and fine-tuning to model interpretation.\n",
"\n",
"\n",
"## Objectives\n",
"\n",
"1. Understand the roles of CXR and ECG in evaluating cardiac and thoracic health, and the benefits of multimodal modeling with these low-cost modalities.\n",
"\n",
"2. Learn the standard PyKale workflow for pretraining, fine-tuning, and interpreting the CardioVAE model.\n"
"This notebook is based on the work of **Suvon et al. (MICCAI 2024)**, which introduced a tri-stream pretraining strategy using a **Multimodal Variational Autoencoder (VAE)** to learn both modality-shared and modality-specific representations for assessing **Pulmonary Arterial Wedge Pressure (PAWP)**—a critical indicator of cardiac hemodynamics. The resulting model, **CardioVAE**, is implemented in the [PyKale](https://github.com/pykale/pykale) library. Here, we provide a concise example of how to use this model through PyKale's APIs—from pretraining and fine-tuning to model interpretation."
],
"cell_type": "markdown"
},
Expand Down Expand Up @@ -135,8 +128,9 @@
"\n",
"if \"google.colab\" in str(get_ipython()):\n",
" sys.path.insert(0, site.getusersitepackages())\n",
" !git clone --single-branch https://github.com/pykale/embc-mmai25.git\n",
" %cp -r /content/embc-mmai25/tutorials/cardiac-hemodynamics-assesment/* /content/\n",
" !git clone --single-branch -b main https://github.com/pykale/embc-mmai25.git\n",
" %cp -r /content/embc-mmai25/tutorials/cardiac-hemodynamics-assessment/* /content/\n",
"\n",
" %rm -r /content/embc-mmai25"
],
"cell_type": "code",
Expand Down Expand Up @@ -487,7 +481,7 @@
"- A **classification head** (single or multi-layer MLP) \n",
"- A **training step** that supports standard supervised learning with cross-entropy loss\n",
"\n",
"**Estimated runtime:** 1 minute with 10 epoch\n"
"**Estimated runtime:** 2 minute with 10 epoch\n"
],
"cell_type": "markdown"
},
Expand Down