Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Weakly Supervised Affordance Detection

This is the official implementation of our CVPR 2017 paper. And if you like this paper, check out the extension Adaptive Binarization for Weakly Supervised Affordance Segmentation and another approach to Learning Affordance Segmentation from Very Few Examples.

Any bugs or questions, please email sawatzky AT iai DOT uni-bonn DOT de or consult the more detailed Readme.txt.

Installation strongly supervised learning

  1. Download our CAD 120 affordance dataset and the models and store them in deeplabv2_extension/exper/CAD/models/DESIRED_ARCHITECTURE
    strong_object.caffemodel was trained in strongly supervised setup, weak_object.caffemodel was trained in weakly supervised setup on the object split of our CAD 120 affordance dataset. init.caffemodel is pretrained on imagenet for initialisation.

  2. To install our extension, follow the original deeplab installation instructions

Installation weakly supervised learning

For weakly supervised training, also install GrabCut according to the readme.txt in expectation_step/grabcut.

Create folders for the expectation step, i.e. the folder for the fuzzy convnet output during expectation step

mkdir YOUR_PATH_TO_CAD/CAD_release/Convnet_expectation

and the binarized expectation step results

mkdir YOUR_PATH_TO_CAD/CAD_release/weak_segmentation_mat


  1. Adjust the paths in test_release.prototxt.

  2. Run inference.
    YOUR_PATH_TO_DEEPLABV2_EXTENSION/deeplabv2_extension/build/tools/caffe.bin test --model=YOUR_PATH_TO_DEEPLABV2_EXTENSION/deeplabv2_extension/exper/CAD/config/DESIRED_ARCHITECTURE/test_release.prototxt --gpu=0 --weights=YOUR_PATH_TO_DEEPLABV2_EXTENSION/deeplabv2_extension/exper/CAD/models/DESIRED_ARCHITECTURE strong_object.caffemodel --iterations=4605 The predictions are stored as .mat files ending with *blob_0.mat in the folder specified in test_release.protxt, MatWrite layer. Width and height are flipped, as for original deeplab.


Call getMeanIoU_release.m in matlab. First adjust the paths to your setting.

Supervised training

To reproduce our results on the CAD 120 affordance dataset, follow these steps:

  1. Adjust the paths in solver_release.prototxt and train_release.prototxt.

  2. Train your model.
    YOUR_PATH_TO_DEEPLABV2_EXTENSION/deeplabv2_extension/build/tools/caffe.bin train --solver=YOUR_PATH_TO_DEEPLABV2_EXTENSION/deeplabv2_extension/exper/CAD/config/DESIRED_ARCHITECTURE/solver_release.prototxt --gpu=0 --weights=YOUR_PATH_TO_DEEPLABV2_EXTENSION/deeplabv2_extension/exper/CAD/model/DESIRED_ARCHITECTURE/init.caffemodel

Weakly supervised training

  1. Adjust the paths in, expectation.m, solver_release_weak.protxt, train_release.protxt, expectation_release.prototxt, test_release.prototxt.
    Make sure the output folder in expectation_release.prototxt is the same as the input folder in expectation.m

  2. Run weakly supervised training ./

If you find the code useful, please consider citing our paper using the following BibTeX entry.

@InProceedings{Sawatzky_Srikantha_2017_CVPR, author = {Sawatzky, Johann and Srikantha, Abhilash and Gall, Juergen}, title = {Weakly Supervised Affordance Detection.}, booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {July}, year = {2017} }


No description, website, or topics provided.






No releases published


No packages published