Skip to content
/ CASA Public

code for Category-agnostic Skeletal Animal Reconstruction

License

Notifications You must be signed in to change notification settings

Iven-Wu/CASA

Repository files navigation

CASA

This repository contains the official implementation for CASA introduced in the following paper:

CASA: Category-agnostic Skeletal Animal Reconstruction
Yuefan Wu*, Zeyuan Chen*, Shaowei Liu, Zhongzheng Ren, Shenlong Wang
NeurIPS 2022

You can find more visual results and a brief introduction to CASA at our project page.

Environmental Setup

The code is tested in:

  • Python 3.8
  • Pytorch 1.11.0
  • torchvision 0.12.0
  • Cuda 11.3

If you are using Anaconda, the following command can be used to build the environment:

conda env create -f casa.yml
conda activate casa

# install lietorch
git clone --recursive https://github.com/princeton-vl/lietorch.git
cd lietorch
python setup.py install
cd -

# install clip
pip install git+https://github.com/openai/CLIP.git

# install softras
# to compile for different GPU arch, see https://discuss.pytorch.org/t/compiling-pytorch-on-devices-with-different-cuda-capability/106409

pip install -e softras

Overview

We provide instructions for shape optimization on synthetic data.

Preparing Dataset

We use the Planetzoo dataset. If you would like to download the Planetzoo data, please fill out this google form. Then send both the google form and a proof of game purchase of planetzoo to us at casa_planetzoo@googlegroups.com. We will send you the link to download the data.

The dataset should have a structure as follows:

<data_source_name>/
    <animal_name>/

        ├── frame_000001.obj ### ground truth mesh
        ├── frame_000002.obj
        ├── ...
        ├── skeleton ### skeleton
        ├── weight   ### skinning weight
        ├── info    
            ├── 0001.npz ### mask,flow,camera,etc..
            ├── 0002.npz 
            ├── ...
            ├── 0001.png ### renderer images
            ├── 0002.png
            ├── ...

Retrieval

We provide encoded features in ./dataset/embeddings

If you only want to try retrieval part, use the code below.

python clip_retrieve.py 

Optimization

Optimize the shape, skeleton, skinning weight parameters.

python main.py -c config/synthetic.yaml

Acknowledgement

The code is built based on LASR. Thank the authors for sharing their codes!

External repos:

Citation

To cite our paper,

@inproceedings{wu2022casa,
    title={{CASA}: Category-agnostic Skeletal Animal Reconstruction},
    author={Yuefan Wu* and Zeyuan Chen* and Shaowei Liu and Zhongzheng Ren and Shenlong Wang},
    booktitle={NeurIPS},
    year={2022}
}

License

The data is released under Planetzoo Terms of Use, and the code is release under a non-comercial creative commons license.

About

code for Category-agnostic Skeletal Animal Reconstruction

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published