Hierarchical Scene Annotator (HSA) - WebApp

The hierarchical scene annotator provides a web interface for machine-assisted groundtruth image labeling. It guides users in annotating an object-part decomposition of a scene by:

• speeding creation of user-specified regions using an adaptive brush that snaps to precomputed superpixels
• supporting drag and drop rearrangement of region hierarchies
• automatically enforcing object-part region containment invariants
• offering interactive visualization of occlusion layering

For more details, see:

[Hierarchical Scene Annotation] (http://vision.caltech.edu/~mmaire/papers/pdf/seg_annotate_bmvc2013.pdf)
Michael Maire, Stella X. Yu, and Pietro Perona
British Machine Vision Conference (BMVC), 2013

The annotator web application consists of a python server backend and a javascript and WebGL based client that runs in any modern browser.

Installation

Run the server (with python version 3.0 or later, on Linux):

$ cd webapp/server/  
$ python server.py  

This starts a locally accessible webapp which can be loaded by a standards compliant WebGL capable browser (the latest versions of Firefox or Chrome should work). Make sure you have WebGL enabled and navigate to:

http://localhost:8082/hsa-app/annotator?image=example

To make the server visible over the network, edit webapp/server/server.py and replace "localhost" with the ip address of the server and restart.

Any image for which .jpg and .seg files exist in the webapp/server/images/ directory can be loaded by replacing "example" in the above URL. User-provided annotations are stored in the same directory with the .ann extension.

Annotation Data

The matlab/ subdirectory contains MATLAB utilities for working with .seg and .ann files.

Generating Segmentations

Before annotating a new image, you'll need to create an oversegmentation and store it as a corresponding .seg file. See matlab/seg_write.m.

While any segmentation algorithm can be used for this task, a good option is to use the Berkeley segmentation engine, available at:

http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/resources.html

The matlab/ucm2seg.m function converts Ultrametric Contour Maps (UCMs) generated by this algorithm into oversegmentations.

Citation

If you make use of this software, please cite the following reference in any publications:

@inproceedings{MYP:BMVC:2013,    
    title     = {Hierarchical Scene Annotation},  
    author    = {Michael Maire and Stella X. Yu and Pietro Perona},  
    booktitle = {British Machine Vision Conference (BMVC)},  
    year      = {2013}  
}

License

Copyright (C) 2012-2013 Michael Maire mmaire@gmail.com

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see http://www.gnu.org/licenses/.

External Dependencies

This project includes code from various external sources. The complete project as well as any code specific to the HSA application may only be distributed under the terms of the GNU Affero General Public License.

Source code for the external dependencies is available from their respective websites as well as independent branches of the hsa-app git repository, as listed below.

bottle

Website: http://bottlepy.org/
GitHub: http://github.com/bottlepy/bottle
License: MIT
Branch: http://github.com/mmaire/hsa-app/tree/extern-bottle

gl-matrix

GitHub: http://github.com/toji/gl-matrix
License: See LICENSE.md in extern-gl-matrix branch
Branch: http://github.com/mmaire/hsa-app/tree/extern-gl-matrix

DataStream.js

GitHub: http://github.com/kig/DataStream.js
Branch: http://github.com/mmaire/hsa-app/tree/extern-datastream

jquery and jquery-ui

Website: http://jquery.org/
License: MIT
Branch http://github.com/mmaire/hsa-app/tree/extern-jquery-dist

jquery-mousewheel

GitHub: http://github.com/brandonaaron/jquery-mousewheel
License: MIT
Branch: http://github.com/mmaire/hsa-app/tree/extern-jquery-mousewheel

Purl

GitHub: http://github.com/allmarkedup/purl
License: MIT
Branch: http://github.com/mmaire/hsa-app/tree/extern-purl

webgl-utils

License: BSD
Branch: http://github.com/mmaire/hsa-app/tree/extern-webgl-utils