Skip to content


Repository files navigation

Shelf-supervised Mesh Prediction in the wild

in CVPR 2021, Yufei Ye, Shubham Tulsiani, Abhinav Gupta

Project Page, Video, Arxiv

We aim to infer 3D shape and pose from a single image and are able to train the system with only image collecitons and segmentation -- no template, camera pose, or multi-view association. The method consists of 2 steps:

  1. Category-level Reconstruction. We first infer a volumetric representation in a canonical frame, along with the camera pose for the input image.
  2. Instance-level Specialization. The coarse volumetric prediction is converted to a mesh-based representation, which is further optimized in the predicted camera frame given the input image.

This code repo is a re-implementation of the paper. The code is developed based on Pytorch 1.3 (Pytorch >=1.5 adds backprop version check which will trigger a runtime error), Pytorch3d 0.2.0, and integrated LPIPS. To voxelize meshes for evaluation, we use util code in Occupancy Net but did not include it in this reimplementation.

Demo: Estimate mesh with our pretrained model

Download pretrained models to weights/

dataset model
OpenImages-50 tar link
Chairs in the wild link
Quadrupeds link
CUB-200-2011 link
python  --checkpoint=weights/wildchair.pth

Similar results should be saved at outputs/

input output shape output shape w/ texture

or for other curated categories:

python  --checkpoint=weights/cub.pth --demo_image examples/cub_0.png
python  --checkpoint=weights/wildchair.pth --demo_image examples/wildchair_0.png
python  --checkpoint=weights/quad.pth --demo_image examples/llama.png

for openimages 50 categories, the following script will reconstruct images under data/demo_images/:



To train your own model, set up dataset following before running

python     --dataset allChair --cfg_file config/pmBigChair.json 

For more training details, please refer to


If you find this work useful, please consider citing:

  title={Shelf-Supervised Mesh Prediction in the Wild},
  author={Ye, Yufei and Tulsiani, Shubham and  Gupta, Abhinav},
  booktitle={Computer Vision and Pattern Recognition (CVPR)}


No description, website, or topics provided.






No releases published


No packages published