BEVHeight++: Toward Robust Visual Centric 3D Object Detection

Lei Yang · Tao Tang · Jun Li · Kun Yuan · Peng Chen · Li Wang · Yi Huang · Lei Li · Xinyu Zhang · Kaicheng Yu

BEVHeight++ is a new vision-based 3D object detector specially designed for both roadside and vihicle-side scenarios. On popular 3D detection benchmarks of roadside cameras, BEVHeight++ surpasses all previous vision-centric methods by a significant margin. In terms of the ego-vehicle scenario, our BEVHeight++ also possesses superior over depth-only methods.

News

• [2024/09/06] Both arXiv and codebase are released!

Incoming

• Release the pretrained models

Table of Contents

1. Getting Started
2. Acknowledgment
3. Citation

Getting Started

• Installation
• Prepare Dataset

Train BEVHeight with 8 GPUs

python [EXP_PATH] --amp_backend native -b 8 --gpus 8

Eval BEVHeight with 8 GPUs

python [EXP_PATH] --ckpt_path [CKPT_PATH] -e -b 8 --gpus 8

Experimental Results

• DAIR-V2X-I Dataset

• Rope3D Dataset

• KITTI Dataset

Method	AP|3D			AP|BEV			config	model ckpt
	Easy	Mod.	Hard	Easy	Mod.	Hard
BEVDepth	10.69	7.31	5.88	35.14	23.22	19.33	config	\
BEVHeight	10.61	6.97	5.51	34.58	22.05	17.96	config	\
BEVHeight++	11.37	8.06	6.35	36.81	25.49	20.81	config	model_ckpt

• KITTI-360 Dataset

Method	AP3D (IoU=0.5)			AP3D (IoU=0.25)			config	model ckpt
	AP (Lrg)	AP (Car)	mAP	AP (Lrg)	AP (Car)	mAP
BEVDepth	2.17	42.01	22.09	30.52	59.84	45.18	config	model_ckpt
BEVHeight	1.78	41.76	21.77	33.02	56.69	44.85	config	model_ckpt
BEVHeight++	2.35	46.84	24.59	30.72	65.78	48.25	config	model_ckpt

• Waymo Dataset

• BEVDepth: config, model_ckpt
• BEVHeight: config, model_ckpt
• BEVHeight++: config, model_ckpt

IoU3D	Difficulty	Method	AP3D				APH3D
			All	0-30	30-50	50- $\infty$	All	0-30	30-50	50- $\infty$
0.7	Level_1	BEVDepth	2.86	7.51	1.13	0.12	2.84	7.45	1.11	0.12
		BEVHeight	2.62	6.78	1.35	0.08	2.60	6.71	1.33	0.08
		BEVHeight++	3.10	8.04	1.41	0.13	3.07	7.96	1.40	0.13
0.7	Level_2	BEVDepth	2.68	7.50	1.09	0.10	2.66	7.43	1.08	0.10
		BEVHeight	2.46	6.77	1.30	0.07	2.44	6.70	1.29	0.07
		BEVHeight++	2.91	8.03	1.36	0.12	2.88	7.95	1.35	0.11
0.5	Level_1	BEVDepth	13.22	31.67	6.01	1.43	13.05	31.27	5.93	1.41
		BEVHeight	12.73	29.87	5.90	1.33	12.55	29.45	5.83	1.30
		BEVHeight++	14.00	32.59	6.31	1.86	13.81	32.16	6.25	1.83
0.5	Level_2	BEVDepth	12.41	31.61	5.81	1.25	12.25	31.21	5.73	1.23
		BEVHeight	11.95	29.81	5.70	1.16	11.78	29.40	5.63	1.14
		BEVHeight++	13.14	32.53	6.10	1.62	12.97	32.10	6.04	1.59

Acknowledgment

This project is not possible without the following codebases.

• BEVHeight
• BEVDepth
• DAIR-V2X
• pypcd

Citation

If you use BEVHeight in your research, please cite our work by using the following BibTeX entry:

@article{yang2023bevheight++,
  title={Bevheight++: Toward robust visual centric 3d object detection},
  author={Yang, Lei and Tang, Tao and Li, Jun and Chen, Peng and Yuan, Kun and Wang, Li and Huang, Yi and Zhang, Xinyu and Yu, Kaicheng},
  journal={arXiv preprint arXiv:2309.16179},
  year={2023}
}