Skip to content
master
Switch branches/tags
Go to file
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
src
Apr 20, 2020
Apr 20, 2020
Apr 20, 2020

README.md

Mixture Dense Regression for Object Detection and Human Pose Estimation (CVPR 2020)

Distribution of components compared to the ground-truth distribution of object scale

Mixture models are well-established learning approaches that, in computer vision, have mostly been applied to inverse or ill-defined problems. However, they are general-purpose divide-and-conquer techniques, splitting the input space into relatively homogeneous subsets in a data-driven manner. Not only ill-defined but also well-defined complex problems should benefit from them. To this end, we devise a framework for spatial regression using mixture density networks. We realize the framework for object detection and human pose estimation. For both tasks, a mixture model yields higher accuracy and divides the input space into interpretable modes. For object detection, mixture components focus on object scale, with the distribution of components closely following that of ground truth the object scale. This practically alleviates the need for multi-scale testing, providing a superior speed-accuracy trade-off. For human pose estimation, a mixture model divides the data based on viewpoint and uncertainty -- namely, front and back views, with back view imposing higher uncertainty. We conduct experiments on the MS COCO dataset and do not face any mode collapse.

For questions, please contact me at ali.varamesh@kuleuven.be.

Installation

1- Fiest use mixturedense.yml to reproduce the exact Anaconda environment that we have used for our experiments:

conda env create -f mixturedense.yml

To activate the environment:

source activate mixturedense

2- Install COCOAPI

3- Compile deformable convolutional conda env create -f environment.yml(from DCNv2).

cd src/lib/models/networks/DCNv2
./make.sh

Train

To train models from scratch, first, organize the dataset in the following order and see sample commands at experiments

Dataset preparation

For training and evaluation download the MS COCO dataset and organize it as indicated below:

COCO

  • Download the images (2017 Train, 2017 Val, 2017 Test) from coco website.
  • Download annotation files (2017 train/val and test image info) from coco website.
  • Place the data (or create symlinks) to make the data folder like:
optdata_dir
|-- coco
    |-- annotations
        |   |-- instances_train2017.json
        |   |-- instances_val2017.json
        |   |-- person_keypoints_train2017.json
        |   |-- person_keypoints_val2017.json
        |   |-- image_info_test-dev2017.json
    |-- train2017
    |-- val2017
    |-- test2017ll 

Test

To test the models for detction and pose estimation on a images (stored in a directory) use the inference_ctdet.py and inference_pose.py scripts, respectively

Pretrained models

Detection_DLA34_1x_MDN_3

Detection_DLA34_1x_CenterNet

Detection_HG_3x_CenterNet

Detection_HG_3x_MDN_3

Keypoints_HG_3x_CenterNet

Keypoints_HG_3x_MDN_3

Detection_DLA34_CenterNet_COCO_10percent_Train_Resolution_768_100_epochs

Detection_DLA34_MDN_3_COCO_10percent_Train_Resolution_768_100_epochs

License

This repo is released under the MIT License. We have forked parts of the codebase from other repositories; please refer to the corresponding repositories for their licenses' details.

Citation

@article{varamesh2019mixture,
  title={Mixture Dense Regression for Object Detection and Human Pose Estimation},
  author={Varamesh, Ali and Tuytelaars, Tinne},
  journal={arXiv preprint arXiv:1912.00821},
  year={2019}
}

Acknoledgement

Our repo is forked from the amazing codebase of the Object as Points paper

About

Mixture Dense Regression for Object Detection and Human Pose Estimation

Resources

License

Releases

No releases published

Packages

No packages published