Skip to content
Unseen Object Segmentation in Videos via Transferable Representations, ACCV 2018 (oral)
C++ Python MATLAB Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cache/imagesets
data
datasets
figure
lib
nets Upload files Dec 30, 2018
scripts Update files Dec 30, 2018
utils Upload files Dec 30, 2018
.gitignore
README.md

README.md

TransferSeg

Caffe implementation of our method for transferring knowledge from seen objects in images to unseen objects in videos.
Contact: Yi-Wen Chen (chenyiwena at gmail dot com)

Paper

Unseen Object Segmentation in Videos via Transferable Representations
Yi-Wen Chen, Yi-Hsuan Tsai, Chu-Ya Yang, Yen-Yu Lin and Ming-Hsuan Yang
Asian Conference on Computer Vision (ACCV), 2018 (oral)
Best Student Paper Award Honorable Mention

Please cite our paper if you find it useful for your research.

@inproceedings{Chen_TransferSeg_2018,
  author = {Y.-W. Chen and Y.-H. Tsai and C.-Y. Yang and Y.-Y. Lin and M.-H. Yang},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  title = {Unseen Object Segmentation in Videos via Transferable Representations},
  year = {2018}
}

Installation

git clone https://github.com/wenz116/TransferSeg.git
cd TransferSeg
  • Prepare for MBS
  1. Go to the folder utils/MBS/mex.

  2. Modify the opencv include and lib paths in compile.m/compile_win.m (for Linux/Windows).

  3. Run compile/compile_win in MATLAB (for Linux/Windows).

Dataset

  • Download the PASCAL VOC Dataset as the source image dataset, and put it in the data/PASCAL/VOC2011 folder.

  • Download the DAVIS Dataset as the target video dataset, and put it in the data/DAVIS folder.

Training

  • Download the FCN model pre-trained on PASCAL VOC, and put it in the nets folder.

  • Go to the folder scripts.

  1. Compute optical flow of the input video. Run compute_optical_flow('<VIDEO_NAME>') in MATLAB. The optical flow images will be saved at data/DAVIS/Motion/480p/<VIDEO_NAME>/.

  2. Compute motion prior of the input video via minimum barrier distance. Run get_prior('<VIDEO_NAME>') in MATLAB. The motion prior images will be saved at data/DAVIS/Prior/480p/<VIDEO_NAME>/.

  3. Extract features of each category in PASCAL VOC. The extracted features will be saved at cache/features/, named as features_PASCAL_<CLASS_NAME>_fc7.p.

python get_feature_PASCAL.py <GPU_ID>
  1. Extract features of the input video. The extracted features will be saved at cache/features/, named as features_DAVIS_<VIDEO_NAME>_fc7.p.
python get_feature_DAVIS.py <GPU_ID> <VIDEO_NAME>
  1. Segment mining. The selected segments will be saved at data/DAVIS/Train/480p/<VIDEO_NAME>/.
python get_score.py <GPU_ID> <VIDEO_NAME>
  1. Self learning. The trained models will be saved at output/snapshot/.
./train.sh <GPU_ID> <VIDEO_NAME>

Note

The model and code are available for non-commercial research purposes only.

  • 12/2018: code released
You can’t perform that action at this time.