Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation

This is a PyTorch implementation of our group-wise learning framework for weakly supervised semantic segmentation, which is accepted in AAAI-2021. The extended version has been accepted by IEEE TIP.

Group-Wise Learning for Weakly Supervised Semantic Segmentation. [Paper]

Tianfei Zhou, Liulei Li, Xueyi Li, Chunmei Feng, Jianwu Li, and Ling Shao. AAAI 2021/IEEE TIP 2022.

News

Our new paper for weakly supervised semantic segmentation has been accepted to CVPR 2022, with significantly improved performance. Please see RCA.

Prerequisites

We train the model using PyTorch 1.4.0 with four NVIDIA RTX 2080Ti GPU with 11GB memory per card.

• PyTorch 1.4.0

Other minor Python modules can be installed by running

pip install -r requirements.txt

Training

Clone

git clone -- recursive https://github.com/Lixy1997/Group-WSSS

Prepare Dataset

In the paper, we use PASCAL VOC 2012 for training. Here are the steps to prepare the data:

1. Download the PASCAL VOC 2012 dataset.

2. Create soft links:

   cd data; ln -s your/path VOC2012;

Stage #1: Train the classification network for group-wise semantic mining

1. Once the data is prepared, please run python train.py for training the classification network with our default parameters.

   After the network is finished, you can resize the maps to the original image size by

   cd run/pascal
   python vis.py

2. Move the resized maps to the data/VOCdevkit/VOC2012/ folder.

   Put the saliency maps to the data/VOCdevkit/VOC2012/ folder, or you can run DSS model to generate saliency maps by yourself.

3. Generate the pseudo labels of the training set by

   python gen_labels.py

Stage #2: Train the semantic segmentation network

Once the pseudo ground-truths are generated, they are employed to train the semantic segmentation network. We use Deeplab-v2 in all experiments. But most popular FCN-like segmentation networks can be used instead.

Our Results

1. The visualization of CAMs generated by our group-wise semantic mining can be downloaded from Google Drive.

2. The saliency maps used as pseudo labels can be downloaded from Google Drive.

3. The pseudo ground-truths of PASCAL VOC 2012 generated by our model can be downloaded from Google Drive.

4. The segmentation results of val and test sets of PASCAL VOC 2012 dataset can be downloadeded from Google Drive. For reproducing scores of the test set, please submit the results of test set to the official website following the instructions of the official website.

Citation

If you find this work useful for your research, please consider citing the following paper:

@inproceedings{li2021group,
  title={Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation},
  author={Li, Xueyi and Zhou, Tianfei and Li, Jianwu and Zhou, Yi and Zhang, Zhaoxiang},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  pages={1984--1992},
  year={2021}
}

@article{zhou2022group,
  title={Group-Wise Learning for Weakly Supervised Semantic Segmentation},
  author={Zhou, Tianfei and Li, Liulei and Li, Xueyi and Feng, Chun-Mei and Li, Jianwu and Shao, Ling},
  journal={IEEE Transactions on Image Processing},
  year={2022},
  volume={31},
  pages={799-811},
  publisher={IEEE}
}