Skip to content
This repository has been archived by the owner on Jan 16, 2024. It is now read-only.
/ pod_compare Public archive

Code for our paper titled: "A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving"

License

Notifications You must be signed in to change notification settings

asharakeh/pod_compare

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving

This repository is the official implementation of A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving.

Requirements

Software Support:

Name Supported Versions
Ubuntu 18.04, 20.04
Python 3.7 ,3.8
CUDA 10.1 ,10.2, 11.0
Cudnn 7.6.5 , 8.0.1
PyTorch 1.5 , 1.5.1, 1.6

To install requirements virtualenv and virtualenvwrapper should be available on the target machine.

Virtual Environment Creation:

# Clone repo
git clone https://github.com/asharakeh/pod_compare.git
cd pod_compare

# Create python virtual env
mkvirtualenv pod_compare

# Add library path to virtual env
add2virtualenv src

# Install requirements
cat requirements.txt | xargs -n 1 -L 1 pip install

Datasets

Download the Berkeley Deep Drive (BDD) Object Detection Dataset here. The BDD dataset should have the following structure:

 └── BDD_DATASET_ROOT
     ├── info
     |   └── 100k
     |       ├── train
     |       └── val
     ├── labels
     └── images
            ├── 10K
            └── 100K
                ├── test
                ├── train
                └── val

Download the KITTI Object Detection Dataset here. The KITTI dataset should have the following structure:

└── KITTI_DATASET_ROOT
    ├── object
        ├── training    <-- 7481 train data
        |   ├── image_2
        |   ├── calib
        |   └── label_2
        └── testing     <-- 7580 test data
               ├── image_2
               └── calib

Download the Lyft Object Detection Dataset here. The Lyft dataset needs to be converted to KITTI format first using the official Lyft dataset API. The Lyft dataset should have the following structure:

└── LYFT_DATASET_ROOT
    └── training
        ├── image_2
        └── label_2

For all three datasets, labels need to be converted to COCO format. To do so, run the following:

python src/core/datasets convert_bdd_to_coco.py --dataset-dir /path/to/bdd/dataset/root
python src/core/datasets convert_kitti_to_coco.py --dataset-dir /path/to/kitti/dataset/root
python src/core/datasets convert_lyft_to_coco.py --dataset-dir /path/to/lyft/dataset/root

If the script to convert BDD labels to COCO format does not work, please use these pre-converted labels.

Training

To train the model(s) in the paper, run this command:

python src/train_net.py
--num-gpus 2
--dataset-dir /path/to/bdd/dataset/root
--config-file BDD-Detection/retinanet/name_of_config.yaml
--random-seed xx
--resume

Evaluation

For running inference and evaluation of a model, run the following code:

python src/apply_net.py --dataset-dir /path/to/test/dataset/root --test-dataset test_dataset_name --config-file BDD-Detection/retinanet/name_of_config.yaml --inference-config Inference/name_of_inference_config.yaml

--test-dataset can be one of bdd_val, kitti_val, or lyft_val. --dataset-dir corresponds to the root directory of the dataset used.

Evaluation code will run inference on the test dataset and then will generate mAP, Negative Log Likelihood, Calibration Error, and Minimum Uncertainty Error results. If only evaluation of metrics is required, add --eval-only to the above code snippet.

We provide a list of config combinations that generate the architectures used in our paper:

Method Name Config File Inference Config File Pretrained Model
Baseline RetinaNet retinanet_R_50_FPN_1x.yaml standard_nms.yaml retinanet_R_50_FPN_1x.pth
Loss Attenuation retinanet_R_50_FPN_1x_reg_cls_var.yaml standard_nms.yaml retinanet_R_50_FPN_1x_reg_cls_var.pth
Loss Attenuation + Dropout retinanet_R_50_FPN_1x_reg_cls_var_dropout.yaml mc_dropout_ensembles_pre_nms.yaml retinanet_R_50_FPN_1x_reg_cls_var_dropout.pth
BayesOD retinanet_R_50_FPN_1x_reg_cls_var.yaml bayes_od.yaml retinanet_R_50_FPN_1x_reg_cls_var.pth
BayesOD + Dropout retinanet_R_50_FPN_1x_reg_cls_var_dropout.yaml bayes_od_mc_dropout.yaml retinanet_R_50_FPN_1x_reg_cls_var_dropout.pth
Pre-NMS Ensembles retinanet_R_50_FPN_1x_reg_cls_var.yaml ensembles_pre_nms.yaml -
Post-NMS Ensembles retinanet_R_50_FPN_1x_reg_cls_var.yaml ensembles_post_nms.yaml -
Black Box retinanet_R_50_FPN_1x_dropout.yaml mc_dropout_ensembles_post_nms.yaml retinanet_R_50_FPN_1x_dropout.pth
Output Redundancy retinanet_R_50_FPN_1x.yaml anchor_statistics.yaml retinanet_R_50_FPN_1x.pth

Ensemble methods require multiple independent training runs using different random seeds. To do so, run the training code while adding random-seed xx. We test with 5 runs using seed values of 0, 1000, 2000, 3000, and 4000 in our paper.

For evaluating with PDQ, please use the official PDQ code for COCO data available here.

Citation

If you use this code, please cite our paper:

@misc{feng2020review,
title={A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving}, 
author={Di Feng and Ali Harakeh and Steven Waslander and Klaus Dietmayer},
year={2020},
eprint={2011.10671},
archivePrefix={arXiv},
primaryClass={cs.CV}}

License

This code is released under the Apache 2.0 License.

About

Code for our paper titled: "A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages