Exploring the Interactive Guidance for Unified and Effective Image Matting

This repo is the official PyTorch implementation of Exploring the Interactive Guidance for Unified and Effective Image Matting. Since this work is still under review, we only provide the test code and related models, and the full code will be released later.

1. Requirements

• Hardware: 4-8 GPUs (better with >=11G GPU memory)

• Software: PyTorch>=1.0.0, Python3, tensorboardX, and so on.

• to install tensorboardX, you could do as follows:

  pip install tensorboardX
  pip install tensorboard
  

• install python packages: pip install -r requirements.txt

2. Dataset

Composition-1K

Please contact Brian Price (bprice@adobe.com) requesting for the dataset. And following the instructions for preparation.

Synthetic Unified Dataset

Since the dataset is processed from Composition-1K, you can generate it by:

• Acquire Composition-1K as metioned above
• Select SO and ST images from license-free websites, and composit them with SO and ST from Composition-1K to generate NSO and NST images (You need a manual selection to confirm the composited image are natural with multi-objects)

Finally, the folder structure should be:

DATA_ROOT
├── Combined_Dataset
│   ├── Test_set
│   │   │   ├── alpha
│   │   │   ├── fg
│   │   │   ├── bg
│   │   │   ├── trimaps
│   │   │   ├── ImageSets 
├── combined_4classes (the synthetic unified dataset)
│   │   ├── alphas
│   │   ├── images
│   │   │   ├── SO
│   │   │   ├── ST
│   │   │   ├── NSO
│   │   │   ├── NST
│   │   ├── trimaps

3. Model Zoo

Methods	Annotations	Links	MSE	SAD	Grad	Conn	Notes
UIM (box)	Bounding box	gdrive	0.012	38.15	17.90	33.76	trimap-based
UIM (box)	Bounding box	gdrive	0.006	49.85	25.24	43.60	trimap-free
UIM (dextr)	Extreme points	gdrive	0.015	77.25	33.14	59.93	trimap-free
UIM (in_point)	Foreground point	gdrive	0.077	265.87	103.81	142.87	trimap-free
UIM (iog)	FG/BG points	gdrive	0.042	165.92	74.85	92.18	trimap-free
UIM (scribble)	Scribble	gdrive	0.039	139.03	39.55	58.31	trimap-free

*The metrics are all tested on Composition-1K

4. Inference

Download pretrained models and put them under ./pretrained.

Run on one GPU to evaluate the model, the examples are as follow:

• Test on Composition-1K:

cd UIM/
sh seg_matting_tool/test.sh comp1k uim_bbox

• Test on the synthetic unified dataset:

cd UIM/
sh seg_matting_tool/test.sh 4classes uim_bbox

5. License

This project is under the MIT license.