RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception

This is the official implementation of CVPR2024 paper. "RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception". Ruiyang Hao^*, Siqi Fan^*, Yingru Dai, Zhenlin Zhang, Chenxi Li, Yuntian Wang, Haibao Yu, Wenxian Yang, Jirui Yuan, Zaiqing Nie^†

Overview

• Data Download
• Data Conversion
• Quick Start
• Benchmark
• Citation
• Acknowledgment

Data Download

Please check the bottom of this page website to download the data. As shown in the figure bellow.

After downloading the data, please put the data in the following structure:

├── RCooper
│   ├── calib
|      |── lidar2cam
|      |── lidar2world
│   ├── data
|      |── folders named specific scene index
│   ├── labels
|      |── folders named specific scene index
│   ├── original_label
|      |── folders named specific scene index

Data Conversion

To facilitate the research of cooperative perception methods on RCooper. We provide the format converter from RCooper to other popular public cooperative perception datasets. After the conversion, researchers can directly employ the methods using several opensourced frameworks.

We now support the following conversions:

• DAIR-V2X
• V2V4Real
• OPV2V

RCooper to V2V4Real

Setup the dataset path in codes/dataset_convertor/converter_config.py, and complete the conversion.

cd codes/dataset_converter
python rcooper2vvreal.py

RCooper to OPV2V

Setup the dataset path in codes/dataset_convertor/converter_config.py, and complete the conversion.

cd codes/dataset_converter
python rcooper2opv2v.py

RCooper to DAIR-V2X

Setup the dataset path in codes/dataset_convertor/converter_config.py, and complete the conversion.

cd codes/dataset_converter
python rcooper2dair.py

Quick Start

For detection training & inference, you can find instructions in docs/corridor_scene or docs/intersection_scene in detail. (Notes: you may need to set PYTHONPATH to call modified codes other than the pip-installed ones.)

For Tracking, you can find instructions in docs/tracking.md in detail.

All the checkpoints are released in link in the tabels below, you can save them in codes/ckpts/.

Benchmark

Results of Cooperative 3D object detection for corridor scenes

Method	AP@0.3	AP@0.5	AP@0.7	Download Link
No Fusion	40.0	29.2	11.1	url
Late Fusion	44.5	29.9	10.8	url
Early Fusion	69.8	54.7	30.3	url
AttFuse	62.7	51.6	32.1	url
F-Cooper	65.9	55.8	36.1	url
Where2Comm	67.1	55.6	34.3	url
CoBEVT	67.6	57.2	36.2	url

Results of Cooperative 3D object detection for intersection scenes

Method	AP@0.3	AP@0.5	AP@0.7	Download Link
No Fusion	58.1	44.1	23.8	url
Late Fusion	65.1	47.6	24.4	url
Early Fusion	50.0	33.9	18.3	url
AttFuse	45.5	40.9	27.9	url
F-Cooper	49.5	32.0	12.9	url
Where2Comm	50.5	42.2	29.9	url
CoBEVT	53.5	45.6	32.6	url

Results of Cooperative tracking for corridor scenes

Method	AMOTA(↑)	AMOTP(↑)	sAMOTA(↑)	MOTA(↑)	MT(↑)	ML(↓)
No Fusion	8.28	22.74	34.05	23.89	17.34	42.71
Late Fusion	9.60	25.77	35.64	24.75	24.37	42.96
Early Fusion	23.78	38.18	59.16	44.30	53.02	12.81
AttFuse	21.75	35.31	57.43	44.50	45.73	22.86
F-Cooper	22.47	35.54	58.49	45.94	47.74	22.11
Where2Comm	22.55	36.21	59.60	46.11	50.00	19.60
CoBEVT	21.54	35.69	53.85	47.32	47.24	18.09

Results of Cooperative tracking for corridor scenes

Method	AMOTA(↑)	AMOTP(↑)	sAMOTA(↑)	MOTA(↑)	MT(↑)	ML(↓)
No Fusion	18.11	39.71	58.29	49.16	35.32	41.64
Late Fusion	21.57	43.40	63.02	50.58	42.75	34.20
Early Fusion	21.38	47.71	62.93	50.15	36.80	42.75
AttFuse	11.84	36.63	46.92	39.32	29.00	53.90
F-Cooper	-4.86	14.71	0.00	-45.66	11.52	50.56
Where2Comm	14.21	38.48	50.97	42.27	29.00	45.72
CoBEVT	14.82	38.71	49.04	44.67	33.83	35.69

Citation

If you find RCooper useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@inproceedings{hao2024rcooper,
  title={RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception},
  author={Hao, Ruiyang and Fan, Siqi and Dai, Yingru and Zhang, Zhenlin and Li, Chenxi and Wang, Yuntian and Yu, Haibao and Yang, Wenxian and Jirui, Yuan and Nie, Zaiqing},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}

Acknowledgment

• V2V4Real
• OpenCOOD
• AB3DMOT
• DAIR-V2X

Sincere appreciation for their great contributions.