paper: Weakly Supervised Semantic Segmentation with Boundary Exploration. [pdf]
In this paper, we propose a two-stage framework to tackle semantic segmentation problem with image-level annotation supervision. In the first stage, Attention-pooling CAM is adopted to obtain coarse localization cues, which will be used to synthesize pseudo object boundary labels, in second stage, the prediction of object boundaries will be refined through BENet and direct the propagation of CAM results.
This code heavily depends on the IRN.
- Dataset: PASCAL VOC 2012 & SBD
- Python: 3.6
- Others: python3.6-dev, etc
pip install -r requirement.txtFor pytorch, torchvision, etc, installation command can be easily found with specified setting on official website.Here we use pytorch 1.8.
Specify the VOC dataset path and run the command
python run_sample.py --voc12_root xxxxx- There are some slight differences between the implementation and paper report. In the 'synthesize boundary labels' (make_boundary_label.py), the filter principle will additionally consider the boundary between foregrounds. The formula is revised as follows:
-
The generated pseudo semantic segmentation labels will used to be provide supervision for DeepLab_v1 and DeepLab_v2. There are many considerable implementations in the github, for example: DeepLab-v1, DeepLab-v2, and so on.
-
In previous experiments, the original seed is not set and may cause some bias between reproduction and paper report. Here I report the BES performance in the PASCAL VOC 2012 training set with different seeds.
| seed | report in paper | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| mIoU (w\o dCRF) | 66.4 | 66.2 | 66.5 | 66.2 | 66.4 | 66.4 | 67.9 | 66.0 | 66.7 | 65.7 | 66.6 |
- This paper is inspired by the PSA[1] and IRN[2]. In my view, the boundary is implicitly explored through the prediction of pixels-affinity, BES can be regarded as another implementation.
[1]: Ahn J, Kwak S. Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4981-4990.
[2]: Ahn J, Cho S, Kwak S. Weakly supervised learning of instance segmentation with inter-pixel relations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 2209-2218.
If there is any bug or confusion, I am glad to discuss with you. Sorry for my delayed release, I have spent a long time in TOEFL test and PhD application.