OSVOS: One-Shot Video Object Segmentation
Architecture of the Deep Neural Network for OSVOS-S
OSVOS is a method that tackles the task of semi-supervised video object segmentation. It is based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one-shot). Experiments on DAVIS 2019 show that OSVOS is faster than currently available techniques and improves the state of the art by a significant margin (79.8% vs 68.0%).
This TensorFlow code is a posteriori implementation of OSVOS and it does not contain the boundary snapping branch. The results published in the paper were obtained using the Caffe version that can be found at OSVOS-caffe.
Expected Output with DAVIS 2019 Test-Dev Dataset with the first frame annotated (Overlay code Not included)
Clone the OSVOS-TensorFlow repository
git clone https://github.com/imdsi/OSVOS.git
Install if necessary the required dependencies:
- Python 2.7, Python 3
- Tensorflow r1.0 or higher (
pip install tensorflow-gpu) along with standard dependencies
- Other python dependencies: PIL (Pillow version), numpy, scipy, matplotlib, six
Download the parent model from here (55 MB) and unzip it under
models/(It should create a folder named 'OSVOS_parent').
All the steps to re-train OSVOS are provided in this repository. In case you would like to test with the pre-trained models, you can download them from here (2.2GB) and unzip them under
models/(It should create a folder for every model).
Demo online training and testing
Edit in file
osvos_demo.pythe 'User defined parameters' (eg. gpu_id, train_model, etc).
It is possible to work with all sequences of DAVIS 2016 just by creating a soft link (
ln -s /path/to/DAVIS/ DAVIS) in the root folder of the project.
Training the parent network (optional)
- All the training sequences of DAVIS 2016 are required to train the parent model, thus download it from here if you don't have it.
- Place the dataset in this repository or create a soft link to it (
ln -s /path/to/DAVIS/ DAVIS) if you have it somewhere else.
- Download the VGG 16 model trained on Imagenet from the TF model zoo from here.
- Place the vgg_16.ckpt file inside
- Edit the 'User defined parameters' (eg. gpu_id) in file
python osvos_parent_demo.py. This step takes 20 hours to train (Titan-X Pascal), and ~15GB for loading data and online data augmentation. Change dataset.py accordingly, to adjust to a less memory-intensive setup.
Have a happy training!
If you encounter any problems with the code, want to report bugs, etc. please contact me at
Check our project page for additional information.