Category-Specific Object Reconstruction from a Single Image (CVPR 2015)
Datasets and Paths:
compile.m. Compile the vlfeat library under
$ sh setup_data.sh
The above script will download data for the project, PASCAL VOC 2012 and PASCAL 3D+ and put it under
data/. It will also download and extract
vlfeat in the
All the required data can be downloaded from here. Unzip in
BASE_DIR is the root directory for the codebase.
The code needs data formatted as in
data/pascalData with keypoints and segmentations (either as polygons or binary masks). It also needs keypoint names as in data/partNames and the train/val split in PASCAL as in
data/pascalTrainValIds and metadata about keypoints as in
data/voc_kp_metadata to align to a canonical frame (here the Pascal 3D frame) and ensure a right handed co-ordinate system.
This will train basis shape models for a particular class (here 'car') with the experiment
'debug' that you can use later to visualize/test/evaluate. Look at
for description of parameters.
jobID = mainTest('car','debug','withKps',<parameter options>)
This will test the model you trained above (with train id
'debug') on the validation set. The test id (here
'withKps') is to enable testing with the same trained model with different test settings (e.g. with or without keypoints, with different parameters, with or without optimizing scale, translation, rotation etc).
Visualize the results for the models above.
jobID is returned by
execution. It is usually
'Test<trainId><testId>'. Use this id to perform all
operations on the test data (evaluation/visualization etc).
Visualize the result of NRSFM. Shows the predicted 3D keypoints and their convex hull.
Visualize the learnt deformation bases. Note: use
trainId instead of
This will run the mesh and depth map evaluation on the cached meshes and depth maps.
All the files are cached under in
'cachedir' which is usually
cached files are named as follows:
shapeModelNRSFM.mat- Parameters for the trained NRSFM model. Also contains
test_modelwhich contains the estimated parameters on the test set
shapeModelOpt<id>.mat- Parameters for the trained basis shape models.
statesDir<id>- Directory containing the projection parameters from NRSFM and other info per instance.
inferredShapes<id>- Directory containing fitted basis shapes to instances, projection parameters after optimization etc.
meshes<id>- Directory containing meshes fitted to each instance. Each file contains
'vertices'. Can be viewed using the showMesh function
depthMap<id>- Directory containing meshes rendered into depth maps per instance.
sirfs<id>- Directory containing the outputs from SIRFS. Contains albedo, depth, normals, shading and illumination maps.