Skip to content
Implement of Multi Co-Attention Network(MCANet)
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
demo_images
doc
libs
result/demo
README.md
demo.py
eval_vis_internet_group.py
eval_vis_msrc_group.py
eval_vis_voc_group.py

README.md

MCANet_CVPR2020

Implementation of Multi Co-Attention Network(MCANet) to solve Object Co-segmentation.

Table of Contents

Installation

  • Clone this repository.
  • Python 2.7
  • PyTorch 0.4.1
  • Packages: numpy,opencv-python,matplotlib,glob
  • Datasets: follow the instructions below

Datasets

Four OCS benchmark datasets are used for evaluating the performance of our proposed MACNet.

VOC10

Collected by Faktor and Irani(paper), this dataset contains 1,037 images of 20 object classes, and it is the most challenging dataset in the experiment due to extreme intra-class variability, subtle foreground/background discrimination, and class imbalance. Download Link.

Internet

Introduced by paper, this dataset encloses images with three common object classes. This dataset raises the particular challenge of noisy outliers, such that images with no common object exist in the dataset. Download Link.

MSRC

This dataset encompasses 14 image groups with 410 images, each group contains common objects drawn from the same class. Download Link.

iCoseg

There are 38 categories in iCoseg dataset with totally 643 images. Each category composes images containing foreground with limited intra-class variability. Download Link.

Model

  • Implementation
  • the codes of resnet101 and ASPP layer are copied from this work
  • download pretrained MCANet weights at here
  • Note: Training code will be released after our submission is accepted.

Demos

Usage: python demo.py

Input with multi-classes demo images, output with grouped images. The result visualization can be seen here.

group 1:

group 2:

group 3:

Evaluation

Metric:

  • Jaccard

Jaccard index reports the ratio of the intersection and union area between the segmented foreground objects and the ground truth.

  • Precision

Precision measures the percentage of correctly segmented object and background pixels.

Usage: python eval_vis_voc_group.py | eval_vis_internet_group.py | eval_vis_msrc_group.py

Performance

  • average
Dataset VOC10 Internet MSRC
Metric P J P J P J
MCANet 93.5 63.7 94.2 74.8 90.1 73.5
  • voc10
Class A.P. Bike. Bird Boat Bottle Bus. Car Cat Chair Cow D.T. Dog Horse M.B. P.S. P.P. Sheep Sofa Train TV
Metric P J P J P J P J P J P J P J P J P J P J P J P J P J P J P J P J P J P J P J P J
MCANet 97.2 77.9 90.7 13.1 96.6 73.1 95.9 71.8 94.6 72.1 94.5 84.2 95.3 82.0 95.0 77.8 89.7 39.2 95.8 78.4 86.6 15.7 95.3 72.1 94.7 73.3 92.9 69.7 93.7 56.1 92.1 51.5 93.7 72.3 88.8 49.5 95.3 79.6 91.6 64.5
  • internet
Class Airplane Car Horse
Metric P J P J P J
MCANet 96.1 73.2 94.7 84.2 91.8 67.1
  • MSRC
Class house plane bike bird dog sign tree chair face flower car sheep cow cat
Metric P J P J P J P J P J P J P J P J P J P J P J P J P J P J
MCANet 90.6 78.1 94.6 74.3 75.4 49.8 96.5 78.5 95.6 82.5 95.0 86.4 66.7 35.4 90.8 67.5 87.7 67.7 91.8 80.3 91.5 81.2 95.4 84.3 94.6 80.7 95.2 82.5
You can’t perform that action at this time.