Explainable Object-induced Action Decision for Autonomous Vehicles

The repo for our cvpr 2020 paper. We used maskrcnn benchmark for bounding box extraction. The project page is also available here.

Proposed architecture.

 

Installation

Step 1

Clone this repo.

git clone https://github.com/Twizwei/bddoia_project

Step 2

Install maskrcnn benchmark. Follow the instructions here to install maskrcnn benchmark.  

Dataset

Download the our dataset BDD-OIA and then extract it.

To run our model for last frames, it is better to establish symbolic link.

cd ./maskrcnn/maskrcnn-benchmark
mkdir datasets
ln -s dir_to_lastframe datasets

BDD-OIA also contains data for videos.

 

Training and evaluation

First we need to train Faster RCNN on BDD100K. The pretrained Faster RCNN is available here

To train the model, run

python ./maskrcnn/maskrcnn-benchmark/action_prediction/train.py --batch size 2 --num_epoch 50 --initLR 0.001 --gtroot "root-to-action-gt" --reasonroot "root-to-explanation-gt" MODEL.SIDE True MODEL.ROI_HEADS.SCORE_THRESH 0.4 MODEL.PREDICTOR_NUM 1 OUTPUT_DIR "output-directory" MODEL.META_ARCHITECTURE "Baseline1"

Training configurations can be found in train.py and maskrcnn/maskrcnn-benchmark/maskrcnn_benchmark/config/defaults/py .

 

To evaluate the model, run

python ./maskrcnn/maskrcnn-benchmark/action_prediction/test.py --batch size 2 --gtroot "root-to-action-gt" --reasonroot "root-to-explanation-gt" WEIGHT "weights-dir" MODEL.SIDE True MODEL.ROI_HEADS.SCORE_THRESH 0.4 MODEL.PREDICTOR_NUM 1 OUTPUT_DIR "output-directory" MODEL.META_ARCHITECTURE "Baseline1"

A pretrained model net_Final.pth is also available here.

Acknowledgement

 Mask R-CNN Benchmark