PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-Performance Cloud Removal from Multi-temporal Satellite Imagery

This repository is the official PyTorch implementation of the accepted paper PMAA of ECAI 2023 Oral.

Xuechao Zou^1,*, Kai Li^2,*, Junliang Xing², Pin Tao^1,2,†, Yachao Cui¹

Qinghai University¹ • Tsinghua University²

Paper Preprint | Project Page

News

• [2026/04/11] We open-sourced visualization results (including DiffCR and PMAA) from the paper for direct comparison in your own research papers.
• [2023/07/30] Code release.
• [2023/07/16] PMAA got accepted by ECAI 2023 Oral.
• [2023/03/29] PMAA is on arXiv now.

Visualization

The visualization results of 12 methods (including DiffCR) on the test sets of Sen2_MTC_Old and Sen2_MTC_New datasets, along with evaluation code for direct comparison by researchers, are available at: 🤗 HuggingFace Visualization

├── paper-report.png          ← reference metrics table from the paper
│
├── data/
│   ├── Sen2_MTC_New/
│   │   ├── GT/               ← 687 cloud-free ground-truth images  ({id}.png)
│   │   └── inputs/           ← 687 × 3 cloudy input images
│   │                            ({id}_A1.png  {id}_A2.png  {id}_A3.png)
│   └── Sen2_MTC_Old/
│       ├── GT/               ← 313 ground-truth images
│       └── inputs/           ← 313 × 3 cloudy inputs
│
├── results/
│   ├── Sen2_MTC_New/
│   │   ├── ae/               ← prediction images for each method ({id}.png)
│   │   ├── crtsnet/
│   │   ├── ctgan/
│   │   ├── ddpmcr/
│   │   ├── diffcr/           ← DiffCR [Ours]
│   │   ├── dsen2cr/
│   │   ├── mcgan/
│   │   ├── pix2pix/
│   │   ├── pmaa/
│   │   ├── stgan/
│   │   ├── stnet/
│   │   └── uncrtaints/
│   └── Sen2_MTC_Old/
│       └── (same 12 methods)
│
└── eval/
    ├── metrics.py            ← PSNR / SSIM / FID / LPIPS evaluation
    ├── plot.py               ← comparison figure generation
    └── requirements.txt      ← Python dependencies

Requirements

To install dependencies:

pip install -r requirements.txt

To download datasets:

• Sen2_MTC_Old: multipleImage.tar.gz

• Sen2_MTC_New: CTGAN.zip

Training

To train the models in the paper, run these commands:

python train_old.py
python train_new.py

Evaluation

To evaluate my models on two datasets, run:

python test_old.py
python test_new.py

Pre-trained Models

You can download pretrained models here:

• Our awesome model trained on Sen2_MTC_old: pmaa_old.pth
• Our awesome model trained on Sen2_MTC_new: pmaa_new.pth

Results

Quantitative Results

Qualitative Results

Citation

If you use our code or models in your research, please cite with:

@article{zou2023pmaa,
  title={PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-Performance Cloud Removal from Multi-temporal Satellite Imagery},
  author={Zou, Xuechao and Li, Kai and Xing, Junliang and Tao, Pin and Cui, Yachao},
  journal={European Conference on Artificial Intelligence (ECAI)},
  year={2023},
  pages={3165-3172},
}