Deep Extreme Cut (DEXTR)

Visit our project page for accessing the paper, and the pre-computed results.

This repository contains a python3 package implemention of DEXTR written on-top of keras and tensorflow.

Abstract

This paper explores the use of extreme points in an object (left-most, right-most, top, bottom pixels) as input to obtain precise object segmentation for images and videos. We do so by adding an extra channel to the image in the input of a convolutional neural network (CNN), which contains a Gaussian centered in each of the extreme points. The CNN learns to transform this information into a segmentation of an object that matches those extreme points. We demonstrate the usefulness of this approach for guided segmentation (grabcut-style), interactive segmentation, video object segmentation, and dense segmentation annotation. We show that we obtain the most precise results to date, also with less user input, in an extensive and varied selection of benchmarks and datasets.

Installation

1. Clone this repository: git clone https://github.com/jsbroks/dextr-keras/
2. Install dependencies found in the requirements.txt: pip3 install -r requirements.txt
3. Install python package: python3 setup.py install

Pre-trained models

We provide the following DEXTR models, pre-trained on:

• PASCAL + SBD, trained on PASCAL VOC Segmentation train + SBD (10582 images). Achieves mIoU of 91.5% on PASCAL VOC Segmentation val.
• PASCAL, trained on PASCAL VOC Segmentation train (1464 images). Achieves mIoU of 90.5% on PASCAL VOC Segmentation val.
• COCO, trained on COCO train 2014 (82783 images). Achieves mIoU of 87.8% on PASCAL VOC Segmentation val.

Citation

If you use this code, please consider citing the following papers:

@Inproceedings{Man+18,
  Title          = {Deep Extreme Cut: From Extreme Points to Object Segmentation},
  Author         = {K.K. Maninis and S. Caelles and J. Pont-Tuset and L. {Van Gool}},
  Booktitle      = {Computer Vision and Pattern Recognition (CVPR)},
  Year           = {2018}
}

@InProceedings{Pap+17,
  Title          = {Extreme clicking for efficient object annotation},
  Author         = {D.P. Papadopoulos and J. Uijlings and F. Keller and V. Ferrari},
  Booktitle      = {ICCV},
  Year           = {2017}
}

We thank the authors of PSPNet-Keras-tensorflow for making their Keras re-implementation of PSPNet available!

If you encounter any problems please contact us at {kmaninis, scaelles}@vision.ee.ethz.ch.