Skip to content

[IEEE TIP 2022] Consistency-Regularized Region-Growing Network for Semantic Segmentation of Urban Scenes with Point-Level Annotations

License

Notifications You must be signed in to change notification settings

YonghaoXu/CRGNet

Repository files navigation

Consistency-Regularized Region-Growing Network for Semantic Segmentation of Urban Scenes with Point-Level Annotations


This research has been conducted at the Institute of Advanced Research in Artificial Intelligence (IARAI).

This is the official PyTorch implementation of the weakly supervised semantic segmentation method for very high-resolution remote sensing image in our paper Consistency-Regularized Region-Growing Network for Semantic Segmentation of Urban Scenes with Point-Level Annotations.

Table of content

  1. Preparation
  2. Usage
  3. Paper
  4. Results
  5. Acknowledgement
  6. License

Preparation

  • Package requirements: The scripts in this repo are tested with torch==1.8 and torchvision==0.9 using a single NVIDIA Tesla V100 GPU.
  • Remote sensing datasets used in this repo:
  • Pretrained models:
  • Data folder structure
    • The data folder is structured as follows:
├── <THE-ROOT-PATH-OF-DATA>/
│   ├── Vaihingen/     
|   |   ├── img/
|   |   ├── gt/
|   |   ├── point/
|   |   |   ├── an1/
|   |   |   ├── an2/
|   |   |   ├── an3/
|   |   |   ├── an4/
│   ├── Zurich/    
|   |   ├── img/
|   |   ├── gt/
|   |   ├── point/
|   |   |   ├── an1/
|   |   |   ├── an2/
|   |   |   ├── an3/
|   |   |   ├── an4/

Usage

  • Pretrain the region-growing network:
CUDA_VISIBLE_DEVICES=0 python CRGNet_Vaihingen.py
CUDA_VISIBLE_DEVICES=0 python CRGNet_Zurich.py

Alternatively, you can download our pretrained models on Vaihingen or Zurich Summer for a quick look.

  • Generate pseudo labels:
CUDA_VISIBLE_DEVICES=0 python GenVaihingen.py
CUDA_VISIBLE_DEVICES=0 python GenZurich.py
  • Finetue the network with pseudo labels:
CUDA_VISIBLE_DEVICES=0 python SelfTrain_Vaihingen.py
CUDA_VISIBLE_DEVICES=0 python SelfTrain_Zurich.py
  • Evaluate the performance on test set:
CUDA_VISIBLE_DEVICES=0 python TestVaihingen.py 
CUDA_VISIBLE_DEVICES=0 python TestZurich.py 

Results

  • Dynamically expanded annotations in different iterations:

  • Example segmentation results of an image in the test set on the Zurich Summer dataset:

  • Performance contribution of each module in CRGNet (reported in mean F1):

Method RG CR ST dCRF Vaihingen Zurich Summer
Baseline 61.63 60.13
+RG 63.89 65.58
+CR 65.48 68.61
+ST 68.07 71.26
+dCRF 70.92 75.68

Paper

Consistency-regularized region-growing network for semantic segmentation of urban scenes with point-level annotations

Please cite the following paper if the code is useful for your research:

@article{crgnet,
  title={Consistency-regularized region-growing network for semantic segmentation of urban scenes with point-level annotations}, 
  author={Xu, Yonghao and Ghamisi, Pedram},
  journal={IEEE Trans. Image Process.},  
  volume={31},
  pages={5038-5051},
  year={2022},
}

Acknowledgement

The authors would like to thank the International Society for Photogrammetry and Remote Sensing (ISPRS), and the German Society for Photogrammetry, Remote Sensing and Geoinformation (DGPF) for providing the Vaihingen dataset, and Dr. Michele Volpi for providing the Zurich Summer dataset.

Lovász-Softmax loss

PyDenseCRF

License

This repo is distributed under MIT License. The code can be used for academic purposes only.

About

[IEEE TIP 2022] Consistency-Regularized Region-Growing Network for Semantic Segmentation of Urban Scenes with Point-Level Annotations

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages