BARC

[Project Page]

Table of Contents

• Description
• Installation
  • Environment
  • Data Preparation
  • Configurations
• Usage
  • Demo
  • Training
  • Inference
• Citation
• License
• Acknowledgments
• Contact

Description

Breed Augmented Regression using Classification (BARC) is a method for dog pose and shape estimation.

Installation

Environment

The code has been tested with CUDA 10.1, CuDNN 7.5, Python 3.7 and PyTorch 1.6.0.

    conda create -n "conda_barc" python=3.7.6 ipython
    conda activate conda_barc
    conda install pytorch==1.6.0 torchvision cudatoolkit=10.1 -c pytorch

To install the remaining dependencies run the following command:

    pip install -r requirements.txt

Data Preparation

All necessary data be downloaded here. A folder named 'checkpoint' contains pretrained models, copy it to the main folder of this project. A folder named 'stanext_related_data.zip' contains information related to the dataset. Please extract the zip file and copy it to data/stanext_related_data.

Download the Stanford Extra image dataset from https://github.com/benjiebob/StanfordExtra and store it in datasets/StanfordExtra_V12.

Your folder structure should look as follows:

folder
├── checkpoint
│   ├── barc_complete
│   └── ...
├── data
│   ├── breed_data
│   ├── smal_data
│   ├── statistics
│   └── stanext_related_data
├── datasets
│   ├── test_image_crops
│   ├── StanfordExtra_V12
│   │   ├── StanExtV12_Images
│   │   └── labels
├── scripts
│   └── ...
├── src
│   └── ...

Configurations

All configuration files can be found in src/configs. You will need to adjust paths in barc_cfg_{train,test,visualization}.yaml and dataset_path_configs.py. If desired you can change the weights for different loss functions used at training time, see barc_loss_weights.json. We do not recommend changing zero-value weights to non-zero values, as most of the unused loss functions were removed.

Usage

Demo

In order to run our pretrained model on new sample images, prepare image crops and put them into the folder datasets/test_image_crops. The crops can have arbitrary rectangular shape, but should show the dog more or less in the center of the image. Please have a look at the provided example image.

    python scripts/visualize.py --workers 12  \
    --model-file-complete barc_complete/model_best.pth.tar \
    --config barc_cfg_visualization.yaml \

Training

You can train a full new model using the following command. Pretrained weights for the stacked hourglass and the 3d pose branch (including the normalizing flow pose prior) will be loaded.

    python scripts/train.py --workers 12 --checkpoint barc_new_v2 \
    --config barc_cfg_train.yaml \
    start \
    --model-file-hg barc_hg_pret/checkpoint.pth.tar \
    --model-file-3d barc_normflow_pret/checkpoint.pth.tar

Inference

In order to reproduce the results as listed in our paper, run the following command:

    python scripts/test.py --workers 12  \
    --model-file-complete barc_complete/model_best.pth.tar \
    --config barc_cfg_test.yaml
    --save-images True

Citation

If you find this Model & Software useful in your research we would kindly ask you to cite:

@inproceedings{BARC:2022,
    title = {BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information},
    author = {Rueegg, Nadine and Zuffi, Silvia and Schindler, Konrad and Black, Michael J.},
    booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
    year = {2022}
}

License

Software Copyright License for non-commercial scientific research purposes. Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use BARC data, model and software, (the "Data & Software"), including 3D meshes, images, videos, textures, software, scripts, and animations. By downloading and/or using the Data & Software (including downloading, cloning, installing, and any other use of the corresponding github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Data & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License.

Acknowledgments

Contact

The code of this repository was implemented by Nadine Rüegg.

For commercial licensing (and all related questions for business applications), please contact ps-licensing@tue.mpg.de.