STE-SAMed

Spatiotemporal-Enhanced SAMed for Echocardiography Video Segmentation

STE-SAMed is a SAM-based spatiotemporal segmentation method designed for echocardiographic (cardiac ultrasound) video segmentation. It introduces a Deformable Spatiotemporal Feature Separator (DSFS) and a Spatiotemporal Interaction and Fusion Module (STIFM) on top of the Segment Anything Model (SAM) to jointly handle speckle noise, apical signal dropout, and cardiac motion across frames.

Table of Contents

• Requirements
• Checkpoints
• Datasets
  • Directory Structure
  • Data Format
• Training
• Testing

Requirements

Please make sure the following dependencies are installed before running the code:

pip install -r requirements.txt

Checkpoints

Download the pretrained SAM ViT-B checkpoint used to initialize STE-SAMed:

wget https://huggingface.co/Wky1218/STE-SAMed/resolve/main/sam_vit_b_01ec64.pth -P checkpoints/

Or download it manually from the Hugging Face repository and place it under a checkpoints/ directory in the project root.

Datasets

This project uses two public echocardiography datasets:

• CAMUS: available at https://www.creatis.insa-lyon.fr/Challenge/camus/
• EchoNet-Dynamic: available at https://echonet.github.io/dynamic/

Please follow each dataset's official instructions to request access and download the data.

Directory Structure

After downloading, organize the datasets as follows:

datasets
├── CAMUS
│   ├── train
│   ├── test
│   └── val
└── EchoNet_Dynamic
    ├── train
    ├── test
    └── val

Data Format

• Training data should be in .npz format.
• Testing data should be in .h5 format.

If your test data is not already in .h5 format, run the conversion script first:

python test_h5.py

This will convert the raw test data into the .h5 format required by test.py.

Training

Once the checkpoint and datasets are in place, start training with:

python train.py

Testing

After converting the test data to .h5 format (see Data Format), run:

python test.py