This is an official implementation of DCIFNet in our TGRS paper DCIFNet.
The code is based on PaddleSeg.
Land cover classification (LCC) based on remote sensing image segmentation is a prominent task of remote sensing data interpretation. The commonly used optical data is susceptible to the weather, so it has the potential to utilize complementary features from the supplementary synthetic aperture radar (SAR) data to enhance segmentation performance. However, current multi-modal segmentation methods focus on the deep fusion of features, which usually ignores the significance of structural consistency information. In order to make use of the mutual correction and information exchange between multi-modal data, we propose DCIFNet, a dual-stream correction-interaction-fusion multi-modal LCC network. Specifically, we design a differential feature correction and enhancement module (DF-CEM) that leverages bidirectional differential features to correct multi-modal features. In addition, for corrected feature pairs, we deploy a parallel attention interaction module (PAIM) to focus on the pixel-level feature correlation and achieve effective information exchange in both channel and spatial dimensions. Through the expert fusion module (EFM), DCIFNet leverages the gate network to attain a flexible and compact feature fusion between multi-modal features. Experimental results show that our method achieves a superior performance compared with other multi-modal fusion segmentation methods on three optical-SAR datasets.
- Requirements
- Python 3.7
- paddlepaddle-gpu 2.4.2
- CUDA 11.7 or higher
- Install all dependencies.
pip install -r requirements.txt
- Xian and Pohang datasets download links DDHRNet
- WHU-RGB-SAR
trainM_list.txt contains optical, sar and label paths, e.g.:
# GF2.jpg GF3.jpg label.png
/workspace/DATA/korea/256_cloud/GF2/6133.jpg /workspace/DATA/korea/256_cloud/GF3/6133.jpg /workspace/DATA/korea/256_cloud/label/6133.png
...bash train.sh
# or
python3.7 train_dual_seg_xian.py > log_ours/xian_cloud_M_20250109_adamw_ours
# ...python3.7 InferBig.pyIf you use DCIFNet in your research, please cite the original paper:
@ARTICLE{11164558,
author={Ren, Bo and Liu, Bo and Wang, Qianfang and Hou, Biao and Yang, Chen and Jiao, Licheng},
journal={IEEE Transactions on Geoscience and Remote Sensing},
title={DCIFNet: Cross-Modal Fusion With Correction and Interaction for Optical–SAR Land Cover Classification},
year={2025},
volume={63},
number={},
pages={1-18},
doi={10.1109/TGRS.2025.3609898}}
This code is released under the Apache License 2.0.