OccupancyM3D

Note that this is an initial version, this repository needs to be further cleaned and refactored.

paper link: [2305.15694] Learning Occupancy for Monocular 3D Object Detection

Installation

We employ the design and framework of OpenPCDet, and follows the base setup in CaDDN, thanks for their great work!

Therefore, please follow the installation steps in OpenPCDet.

Getting Started

First, please follow the KITTI data file generation in CaDDN. You can find "depth_caddn" in CaDDN, and "lidar_depth" in Google Drive.

We do not use the heavy deeplibV3 backbone. Instead, we use the pre-trained dla34 backbone from DD3D. Note that the authors slightly modify the model. We re-map the key and provide it at Google Drive.

Then, train the model:

cd OccupancyM3D/OpenPCDet/tools

CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --rdzv_endpoint=localhost:6400 --nproc_per_node=4  train.py --launcher pytorch  --cfg_file ./cfgs/kitti_models/OccupancyM3D.yaml \
  --sync_bn --workers 4 \
  --num_epochs_to_eval 20  \
  --extra_tag OccupancyM3D

Eval the model using the pre-trained model:

CUDA_VISIBLE_DEVICES=0 python test.py --cfg_file ./cfgs/kitti_models/OccupancyM3D.yaml \
  --extra_tag val \
  --batch_size 2 --workers 4 \
  --ckpt $pre-trained-model-path

Pretrained Model

To ease the usage, we provide the pre-trained model at: Google Drive

Here we give the comparison.

Models	Car@BEV IoU=0.7			Car@3D IoU=0.7
	Easy	Mod	Hard	Easy	Mod	Hard
original paper	35.72	26.60	23.68	26.87	19.96	17.15
this repo	36.26	26.25	23.22	28.64	19.84	17.77

Visualization Results

Some good cases and bad cases (marked using arrows)

KITTI results

Waymo results