Skip to content
Code for "Category-Specific Object Reconstruction from a Single Image" CVPR 2015.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
basisShapes
evaluation added error reporting to depth map evaluation May 26, 2015
external/SIRFS removed vlfeat and added script to download it May 26, 2015
main uncommented depth map computation May 26, 2015
nrsfm
sirfsPrior
utils removed vlfeat and added script to download it May 26, 2015
visualize
README.md Update README.md Sep 9, 2017
compile.m
get_params.m
globals.m
setup_data.sh
startup.m

README.md

Category-Specific Object Reconstruction from a Single Image (CVPR 2015)

Abhishek Kar*, Shubham Tulsiani*, Joao Carreira, Jitendra Malik

meanshapes

Datasets and Paths:

You will need PASCAL VOC 2012 and PASCAL 3D+ to run the code. Download them and change the paths in startup.m to reflect your paths.

Compilation:

Run compile.m. Compile the vlfeat library under external/

Data:

$ sh setup_data.sh

The above script will download data for the project, PASCAL VOC 2012 and PASCAL 3D+ and put it under data/. It will also download and extract vlfeat in the external/.

All the required data can be downloaded from here. Unzip in BASE_DIR/data where BASE_DIR is the root directory for the codebase.

The code needs data formatted as in data/pascalData with keypoints and segmentations (either as polygons or binary masks). It also needs keypoint names as in data/partNames and the train/val split in PASCAL as in data/pascalTrainValIds and metadata about keypoints as in data/voc_kp_metadata to align to a canonical frame (here the Pascal 3D frame) and ensure a right handed co-ordinate system.

Train Models:

mainTrain('car','debug',<parameter options>)

This will train basis shape models for a particular class (here 'car') with the experiment id 'debug' that you can use later to visualize/test/evaluate. Look at get_params.m for description of parameters.

Test Models:

jobID = mainTest('car','debug','withKps',<parameter options>)

This will test the model you trained above (with train id 'debug') on the validation set. The test id (here 'withKps') is to enable testing with the same trained model with different test settings (e.g. with or without keypoints, with different parameters, with or without optimizing scale, translation, rotation etc).

Visualizing Results:

visInferredShapes('car', jobID)

Visualize the results for the models above. jobID is returned by mainTest after execution. It is usually 'Test<trainId><testId>'. Use this id to perform all operations on the test data (evaluation/visualization etc).

visNRSFMmodel('car', jobID)

Visualize the result of NRSFM. Shows the predicted 3D keypoints and their convex hull.

visDeformations('car', trainId)

Visualize the learnt deformation bases. Note: use trainId instead of jobID here.

Evaluation:

mainEval('car',jobID);

This will run the mesh and depth map evaluation on the cached meshes and depth maps.

Cached files:

All the files are cached under in 'cachedir' which is usually ./cache. The cached files are named as follows:

  • shapeModelNRSFM.mat - Parameters for the trained NRSFM model. Also contains test_model which contains the estimated parameters on the test set

  • shapeModelOpt<id>.mat - Parameters for the trained basis shape models.

  • statesDir<id> - Directory containing the projection parameters from NRSFM and other info per instance.

  • inferredShapes<id> - Directory containing fitted basis shapes to instances, projection parameters after optimization etc.

  • meshes<id> - Directory containing meshes fitted to each instance. Each file contains 'faces' and 'vertices'. Can be viewed using the showMesh function

  • depthMap<id> - Directory containing meshes rendered into depth maps per instance.

  • sirfs<id> - Directory containing the outputs from SIRFS. Contains albedo, depth, normals, shading and illumination maps.

You can’t perform that action at this time.