Progressive Scale-aware Network for Remote sensing Image Change Captioning

Chenyang Liu, Jiajun Yang, Zipeng Qi, Zhengxia Zou, and Zhenwei Shi*✉

Welcome to our repository!

This repository contains the PyTorch implementation of the paper: "Progressive Scale-aware Network for Remote sensing Image Change Captioning".

For more information, please see our published paper in [IEEE] (Accepted by IGARSS 2023)

Data preparation

Firstly, download the image pairs of LEVIR_CC dataset from the [Repository]. Then preprocess dataset as follows:

python create_input_files.py --karpathy_json_path path/Levir-CC-dataset/LevirCCcaptions.json --image_folder path/Levir-CC-dataset/images 

After that, you can find some resulted files in ./data/. Of course, you can use our provided resulted files directly in [Hugging face].

Train

Make sure you performed the data preparation above. Then, start training as follows:

python ./train.py --encoder_image vit_b_32 --data_folder ./data/ --savepath ./checkpoints/5-times/

Evaluate

You can download our pretrained model in [Hugging face]. Put the model in ./checkpoints/5-times/, then run

python ./eval.py --encoder_image vit_b_32 --data_folder ./data/ --model_path ./checkpoints/5-times/

We recommend training 5 times to get an average score.

Citation:

@INPROCEEDINGS{10283451,
  author={Liu, Chenyang and Yang, Jiajun and Qi, Zipeng and Zou, Zhengxia and Shi, Zhenwei},
  booktitle={IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium}, 
  title={Progressive Scale-Aware Network for Remote Sensing Image Change Captioning}, 
  year={2023},
  volume={},
  number={},
  pages={6668-6671},
  doi={10.1109/IGARSS52108.2023.10283451}}