Hongruixuan Chen1 #, Jian Song1,2 #, Chengxi Han3, Junshi Xia2, Naoto Yokoya1,2 *
1 The University of Tokyo, 2 RIKEN AIP, 3 Wuhan University.
# Equal contribution, * Corresponding author
Paper: (arXiv 2404.03425)
Overview | Get Started | Main Results | Reference | Q & A
Notice🐍🐍
: The code of this repo has been updated! Some of the retrained model weights have been uploaded for usage! We'd appreciate it if you could give this repo ⭐️ and stay tuned!April 18th, 2024
: The retrained weight of MambaBCD-Base on the WHU-CD (F1 score 94.19%) is now avaiable. You are welcome to use it!!April 15th, 2024
: The retrained weights of MambaBCD-Small on the SYSU (F1 score 83.36%), Mamba-BCD-Tiny on the LEVIR-CD+ (F1 score of 88.03%) and WHU-CD with (F1 score 94.09%) are now avaiable. You are welcome to use them!!April 12th, 2024
: The new [arXiv] version containing new accuracy and more experiments is now online! The weights for different models will be released soon!April 05th, 2024
: The [arXiv] version is online!April 05th, 2024
: The models and training code for MambaBCD, MambaSCD, and MambaBDA have been organized and uploaded. You are welcome to use them!!
- ChangeMamba serves as a strong benchmark for change detection tasks, including binary change detection (MambaBCD), semantic change detection (MambaSCD), and building damage assessment (MambaBDA).
- Spatio-temporal relationship learning methods of ChangeMamba
The repo is based on the VMama repo, thus you need to install it first. The following installation sequence is taken from the VMamba repo. Also, note that the code in this repo runs under Linux system. We have not tested whether it works under other OS.
Step 1: Clone the repository:
Clone this repository and navigate to the project directory:
git clone https://github.com/ChenHongruixuan/MambaCD.git
cd MambaCD
Step 2: Environment Setup:
It is recommended to set up a conda environment and installing dependencies via pip. Use the following commands to set up your environment:
Create and activate a new conda environment
conda create -n changemamba
conda activate changemamba
Install dependencies
pip install -r requirements.txt
cd kernels/selective_scan && pip install .
Dependencies for "Detection" and "Segmentation" (optional in VMamba)
pip install mmengine==0.10.1 mmcv==2.1.0 opencv-python-headless ftfy regex
pip install mmdet==3.3.0 mmsegmentation==1.2.2 mmpretrain==1.2.0
Also, please download the pretrained weights of VMamba-Tiny, VMamba-Small, and VMamba-Base and put them under
project_path/MambaCD/pretrained_weight/
Binary change detection
The three datasets SYSU, LEVIR-CD+ and WHU-CD are used for binary change detection experiments. Please download them and make them have the following folder/file structure:
${DATASET_ROOT} # Dataset root directory, for example: /home/username/data/SYSU
├── train
│ ├── T1
│ │ ├──00001.png
│ │ ├──00002.png
│ │ ├──00003.png
│ │ ...
│ │
│ ├── T2
│ │ ├──00001.png
│ │ ...
│ │
│ └── GT
│ ├──00001.png
│ ...
│
├── test
│ ├── ...
│ ...
│
├── train.txt # Data name list, recording all the names of training data
└── test.txt # Data name list, recording all the names of testing data
Semantic change detection
The SECOND dataset is used for semantic change detection experiments. Please download it and make it have the following folder/file structure. Note that the land-cover maps are RGB images in the original SECOND dataset for visualization, you need to transform them into single-channel. Also, the binary change maps should be generated by yourself and put them into folder [GT_CD
].
${DATASET_ROOT} # Dataset root directory, for example: /home/username/data/SECOND
├── train
│ ├── T1
│ │ ├──00001.png
│ │ ├──00002.png
│ │ ├──00003.png
│ │ ...
│ │
│ ├── T2
│ │ ├──00001.png
│ │ ...
│ │
│ ├── GT_CD # Binary change map
│ │ ├──00001.png
│ │ ...
│ │
│ ├── GT_T1 # Land-cover map of T1
│ │ ├──00001.png
│ │ ...
│ │
│ └── GT_T2 # Land-cover map of T2
│ ├──00001.png
│ ...
│
├── test
│ ├── ...
│ ...
│
├── train.txt
└── test.txt
Building damage assessment
The xBD dataset can be downloaded from xView 2 Challenge website. After downloading it, please organize it into the following structure:
${DATASET_ROOT} # Dataset root directory, for example: /home/username/data/xBD
├── train
│ ├── images
│ │ ├──guatemala-volcano_00000000_pre_disaster.png
│ │ ├──guatemala-volcano_00000000_post_disaster.png
│ │ ...
│ │
│ └── targets
│ ├──guatemala-volcano_00000003_pre_disaster_target.png
│ ├──guatemala-volcano_00000003_post_disaster_target.png
│ ...
│
├── test
│ ├── ...
│ ...
│
├── holdout
│ ├── ...
│ ...
│
├── train.txt # Data name list, recording all the names of training data
├── test.txt # Data name list, recording all the names of testing data
└── holdout.txt # Data name list, recording all the names of holdout data
Before training models, please enter into [changedetection
] folder, which contains all the code for network definitions, training and testing.
cd <project_path>/MambaCD/changedetection
Binary change detection
The following commands show how to train and evaluate MambaBCD-Small on the SYSU dataset:
python script/train_MambaBCD.py --dataset 'SYSU' \
--batch_size 16 \
--crop_size 256 \
--max_iters 320000 \
--model_type MambaBCD_Small \
--model_param_path '<project_path>/MambaCD/changedetection/saved_models' \
--train_dataset_path '<dataset_path>/SYSU/train' \
--train_data_list_path '<dataset_path>/SYSU/train_list.txt' \
--test_dataset_path '<dataset_path>/SYSU/test' \
--test_data_list_path '<dataset_path>/SYSU/test_list.txt'
--cfg '<project_path>/MambaCD/changedetection/configs/vssm1/vssm_small_224.yaml' \
--pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_small_0229_ckpt_epoch_222.pth'
Semantic change detection
The following commands show how to train and evaluate MambaSCD-Small on the SECOND dataset:
python script/train_MambaSCD.py --dataset 'SECOND' \
--batch_size 16 \
--crop_size 256 \
--max_iters 800000 \
--model_type MambaSCD_Small \
--model_param_path '<project_path>/MambaCD/changedetection/saved_models' \
--train_dataset_path '<dataset_path>/SECOND/train' \
--train_data_list_path '<dataset_path>/SECOND/train_list.txt' \
--test_dataset_path '<dataset_path>/SECOND/test' \
--test_data_list_path '<dataset_path>/SECOND/test_list.txt'
--cfg '<project_path>/MambaCD/changedetection/configs/vssm1/vssm_small_224.yaml' \
--pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_small_0229_ckpt_epoch_222.pth'
Building Damge Assessment
The following commands show how to train and evaluate MambaBDA-Small on the xBD dataset:
python script/train_MambaSCD.py --dataset 'xBD' \
--batch_size 16 \
--crop_size 256 \
--max_iters 800000 \
--model_type MambaBDA_Small \
--model_param_path '<project_path>/MambaCD/changedetection/saved_models' \
--train_dataset_path '<dataset_path>/xBD/train' \
--train_data_list_path '<dataset_path>/xBD/train_list.txt' \
--test_dataset_path '<dataset_path>/xBD/test' \
--test_data_list_path '<dataset_path>/xBD/test_list.txt'
--cfg '<project_path>/MambaCD/changedetection/configs/vssm1/vssm_small_224.yaml' \
--pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_small_0229_ckpt_epoch_222.pth'
Before inference, please enter into [changedetection
] folder.
cd <project_path>/MambaCD/changedetection
Binary change detection
The following commands show how to infer binary change maps using trained MambaBCD-Tiny on the LEVIR-CD+ dataset:
python script/infer_MambaBCD.py --dataset 'LEVIR-CD+' \
--model_type 'MambaBCD_Tiny' \
--test_dataset_path '<dataset_path>/LEVIR-CD+/test' \
--test_data_list_path '<dataset_path>/LEVIR-CD+/test_list.txt' \
--cfg '<project_path>/MambaCD/changedetection/configs/vssm1/vssm_tiny_224_0229flex.yaml' \
--pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_tiny_0230_ckpt_epoch_262.pth'
--resume '<saved_model_path>/MambaBCD_Tiny_LEVIRCD+_F1_0.8803.pth'
Semantic change detection The following commands show how to infer semantic change maps using trained MambaSCD-Tiny on the SECOND dataset:
python script/infer_MambaBCD.py --dataset 'SECOND' \
--model_type 'MambaSCD_Tiny' \
--test_dataset_path '<dataset_path>/SECOND/test' \
--test_data_list_path '<dataset_path>/SECOND/test_list.txt' \
--cfg '<project_path>/MambaCD/changedetection/configs/vssm1/vssm_tiny_224_0229flex.yaml' \
--pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_tiny_0230_ckpt_epoch_262.pth'
--resume '<saved_model_path>/[your_trained_model].pth'
-
The encoders for all the above ChangeMamba models are the the VMamba architecture initialized with ImageNet pre-trained weight.
-
Some of comparison methods are not open-sourced. Their accuracy and number of parameters are obtained based on our own implementation.
Method | Overall Accuracy | F1 Score | IoU | Kappa Coefficient | Param | GFLOPs | ckpts |
---|---|---|---|---|---|---|---|
FC-EF | 87.49 | 73.14 | 57.66 | 64.99 | 17.13 | 45.74 | -- |
SNUNet | 97.83 | 74.70 | 59.62 | 73.57 | 10.21 | 176.36 | -- |
DSIFN | 89.59 | 78.80 | 65.02 | 71.92 | 35.73 | 329.03 | -- |
SiamCRNN-101 | 90.77 | 80.44 | 67.28 | 74.40 | 63.44 | 224.30 | -- |
HANet | 89.52 | 77.41 | 63.14 | 70.59 | 2.61 | 70.68 | -- |
CGNet | 91.19 | 79.92 | 66.55 | 74.31 | 33.68 | 329.58 | -- |
TransUNetCD | 90.88 | 80.09 | 66.79 | 74.18 | 66.79 | 74.18 | -- |
SwinSUNet | 91.51 | 81.58 | 68.89 | 76.06 | 39.28 | 43.50 | -- |
ChangeFormer V4 | 90.12 | 78.81 | 65.03 | 72.37 | 33.61 | 852.53 | -- |
BIT-101 | 90.76 | 79.41 | 65.84 | 73.47 | 17.13 | 45.74 | -- |
MambaBCD-Tiny | 91.36 | 81.29 | 68.48 | 75.68 | 17.13 | 45.74 | -- |
MambaBCD-Small | 92.39 | 83.36 | 71.46 | 78.43 | 49.94 | 114.82 | [GDrive][BaiduYun] |
MambaBCD-Base | 92.30 | 83.11 | 71.10 | 78.13 | 84.70 | 179.32 | -- |
Method | Overall Accuracy | F1 Score | IoU | Kappa Coefficient | Param | GFLOPs | ckpts |
---|---|---|---|---|---|---|---|
FC-EF | 97.54 | 70.42 | 54.34 | 69.14 | 17.13 | 45.74 | -- |
SNUNet | 97.83 | 74.70 | 59.62 | 73.57 | 10.21 | 176.36 | -- |
DSIFN | 98.70 | 84.07 | 72.52 | 83.39 | 35.73 | 329.03 | -- |
SiamCRNN-101 | 98.67 | 83.20 | 71.23 | 82.50 | 63.44 | 224.30 | -- |
HANet | 98.22 | 77.56 | 63.34 | 76.63 | 2.61 | 70.68 | -- |
CGNet | 98.63 | 83.68 | 71.94 | 82.97 | 33.68 | 329.58 | -- |
TransUNetCD | 98.66 | 83.63 | 71.86 | 82.93 | 28.37 | 244.54 | -- |
SwinSUNet | 98.92 | 85.60 | 74.82 | 84.98 | 39.28 | 43.50 | -- |
ChangeFormer V4 | 98.01 | 75.87 | 61.12 | 74.83 | 33.61 | 852.53 | -- |
BIT-101 | 98.60 | 82.53 | 70.26 | 81.80 | 17.13 | 45.74 | -- |
MambaBCD-Tiny | 99.03 | 88.04 | 78.63 | 87.53 | 17.13 | 45.74 | [GDrive][BaiduYun] |
MambaBCD-Small | 99.02 | 87.81 | 78.27 | 87.30 | 49.94 | 114.82 | -- |
MambaBCD-Base | 99.06 | 88.39 | 79.20 | 87.91 | 84.70 | 179.32 | -- |
Method | Overall Accuracy | F1 Score | IoU | Kappa Coefficient | Param | GFLOPs | ckpts |
---|---|---|---|---|---|---|---|
FC-EF | 98.87 | 84.89 | 73.74 | 84.30 | 17.13 | 45.74 | -- |
SNUNet | 99.10 | 87.70 | 78.09 | 87.23 | 10.21 | 176.36 | -- |
DSIFN | 99.31 | 89.91 | 81.67 | 89.56 | 35.73 | 329.03 | -- |
SiamCRNN-101 | 99.19 | 89.10 | 80.34 | 88.68 | 63.44 | 224.30 | -- |
HANet | 99.16 | 88.16 | 78.82 | 87.72 | 2.61 | 70.68 | -- |
CGNet | 99.48 | 92.59 | 86.21 | 92.33 | 33.68 | 329.58 | -- |
TransUNetCD | 99.09 | 87.79 | 78.44 | 87.44 | 28.37 | 244.54 | -- |
SwinSUNet | 99.50 | 93.04 | 87.00 | 92.78 | 39.28 | 43.50 | -- |
ChangeFormer V4 | 99.10 | 87.39 | 77.61 | 86.93 | 33.61 | 852.53 | -- |
BIT-101 | 99.27 | 90.04 | 81.88 | 89.66 | 17.13 | 45.74 | -- |
MambaBCD-Tiny | 99.57 | 94.09 | 88.84 | 93.87 | 17.13 | 45.74 | [GDrive][BaiduYun] |
MambaBCD-Small | 99.57 | 94.06 | 88.79 | 93.84 | 49.94 | 114.82 | -- |
MambaBCD-Base | 99.58 | 94.19 | 89.02 | 93.98 | 84.70 | 179.32 | [GDrive]][BaiduYun] |
Method | Overall Accuracy | F1 Score | IoU | SeK | Param | GFLOPs | ckpts |
---|---|---|---|---|---|---|---|
HRSCD-S1 | 45.77 | 38.44 | 62.72 | 5.90 | 3.36 | 8.02 | -- |
HRSCD-S2 | 85.49 | 49.22 | 64.43 | 10.69 | 6.39 | 14.29 | -- |
HRSCD-S3 | 84.62 | 51.62 | 66.33 | 11.97 | 12.77 | 42.67 | -- |
HRSCD-S4 | 86.62 | 58.21 | 71.15 | 18.80 | 13.71 | 43.69 | -- |
ChangeMask | 86.93 | 59.74 | 71.46 | 19.50 | 2.97 | 37.16 | -- |
SSCD-1 | 87.19 | 61.22 | 72.60 | 21.86 | 23.39 | 189.91 | -- |
Bi-SRNet | 87.84 | 62.61 | 73.41 | 23.22 | 19.44 | 63.72 | -- |
TED | 87.39 | 60.34 | 72.79 | 22.17 | 42.16 | 75.79 | -- |
SMNet | 86.68 | 60.34 | 71.95 | 20.29 | 19.44 | 63.72 | -- |
ScanNet | 87.86 | 63.66 | 73.42 | 23.94 | 27.90 | 264.95 | -- |
MambaSCD-Tiny | 88.07 | 63.44 | 73.33 | 23.34 | 21.51 | 73.42 | -- |
MambaSCD-Small | 88.38 | 64.10 | 73.61 | 24.04 | 54.28 | 146.70 | -- |
MambaSCD-Base | 88.12 | 64.03 | 73.68 | 24.11 | 89.99 | 211.55 | -- |
Method | F1_loc | F1_clf | F1_oa | Param | GFLOPs | ckpts |
---|---|---|---|---|---|---|
xView2 Baseline | 80.47 | 3.42 | 26.54 | -- | -- | -- |
Siamese-UNet | 85.92 | 65.58 | 71.68 | -- | -- | -- |
MTF | 83.60 | 70.02 | 74.10 | -- | -- | -- |
ChangeOS-101 | 85.69 | 71.14 | 75.50 | -- | -- | -- |
ChangeOS-101-PPS | 85.69 | 75.44 | 78.52 | -- | -- | -- |
DamFormer | 86.86 | 72.81 | 77.02 | -- | -- | -- |
MambaBDA-Tiny | 87.53 | 78.35 | 81.11 | 19.74 | 59.57 | -- |
MambaBDA-Small | 86.61 | 78.80 | 81.14 | 52.11 | 130.80 | -- |
MambaBDA-Base | 87.38 | 78.84 | 81.41 | 87.76 | 195.43 | -- |
If this code or dataset contributes to your research, please kindly consider citing our paper and give this repo ⭐️ :)
@article{chen2024changemamba,
title={ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model},
author={Hongruixuan Chen and Jian Song and Chengxi Han and Junshi Xia and Naoto Yokoya},
year={2024},
eprint={2404.03425},
archivePrefix={arXiv},
primaryClass={eess.IV}
}
This project is based on VMamba (paper, code), ScanNet (paper, code), xView2 Challenge (paper, code). Thanks for their excellent works!!
For any questions, please feel free to contact us.