No description, website, or topics provided.
C Matlab HTML Python Makefile Objective-C Other
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
code
data
.gitignore
LICENSE
README.md
compile.m
demoDetect.m
model.mat

README.md

3D Wikipedia source code

Here you will find the Matlab and Python source code for automatically labeling objects in 3D models given a reference text, such as Wikipedia.

CITATION:

Bryan C. Russell, Ricardo Martin-Brualla, Daniel J. Butler, Steven M. Seitz, and Luke Zettlemoyer. 3D Wikipedia: Using Online Text to Automatically Label and Navigate Reconstructed Geometry. ACM Transactions on Graphics (SIGGRAPH Asia 2013), Vol. 32, No. 6. PDF | BibTeX | Project page

DOWNLOAD:

You can download a zip file of the source code directly.

Alternatively, you can clone it from GitHub as follows:

$ git clone https://github.com/brussell123/3dwikipedia.git

INSTALLATION:

  1. For the results in the paper we used the Stanford parser. You can download the version we used here:

    http://nlp.stanford.edu/software/stanford-parser-2012-07-09.tgz

    Uncompress the tarball and insert into "./code/LIBS/".

    We found it helpful to raise the memory limit. You can set the memory limit inside "stanford-parser-2012-07-09/lexparser.sh" by changing "-mx1000m" (we set it to be "-mx4000m").

  2. To download images from Google Image Search, be sure to set your user IP address inside the following file before running the scripts:

    ./code/LIBS/google_image_search/userip.txt

    See the following as a reference for the Google Image Search API:

    https://developers.google.com/console/help/#activatingapis

  3. Download the Bundler tarball:

    http://www.cs.cornell.edu/~snavely/bundler/distr/bundler-v0.4-source.tar.gz

    Uncompress the tarball and insert into "./code/LIBS/".

  4. In Matlab, run "compile" to compile all binaries.

RUNNING THE CODE:

  1. You can run the code demo for the Pantheon example by downloading its text, reference panorama, and pre-computed "bundle" struct. Uncompress the tarball and insert into the "./data" directory. For this example you may skip to step 2 below.

    To run on your own data, start by building a 3D model of the site. You will need to download images to build the 3D model, e.g. by querying Flickr for the site name and downloading images through their API. A couple of possibilities for building a sparse 3D point cloud is via VisualSFM (recommended):

    http://ccwu.me/vsfm/

    or Bundler:

    http://www.cs.cornell.edu/~snavely/bundler/

    We provide Matlab scripts to read the sparse point cloud. To read the output from VisualSFM (e.g. "pantheon.nvm"), run the following in Matlab:

    >> addpath ./code;
    >> bundle = ReadNVMFile('pantheon.nvm');

    To read the output from Bundler (e.g. the output lives in a directory "/path/to/pantheon"), run the following in Matlab:

    >> addpath ./code;
    >> bundle = readBundleFile('/path/to/pantheon');
  2. Given the text, reference image, and "bundle" struct, adjust the global variables at the top of the demoDetect.m script and run in Matlab.


Copyright (C) 2014 Intel, University of Washington, Bryan C. Russell, Ricardo Martin-Brualla, Daniel J. Butler, Steven M. Seitz, Luke Zettlemoyer