Xinhao Liu*, Moonjun Gong*, Qi Fang, Haoyu Xie, Yiming Li, Hang Zhao, Chen Feng
- [2023/10]: OCFBech-nuScenes is available on Hugging Face.
- [2023/10]: The preprint version is available on arXiv. The OCFBench dataset is available on Hugging Face.
Scene completion and forecasting are two popular perception problems in research for mobile agents like autonomous vehicles. Existing approaches treat the two problems in isolation, resulting in a separate perception of the two aspects. In this paper, we introduce a novel LiDAR perception task of Occupancy Completion and Forecasting (OCF) in the context of autonomous driving to unify these aspects into a cohesive framework. This task requires new algorithms to address three challenges altogether: (1) sparse-to-dense reconstruction, (2) partial-to-complete hallucination, and (3) 3D-to-4D prediction. To enable supervision and evaluation, we curate a large-scale dataset termed OCFBench from public autonomous driving datasets. We analyze the performance of closely related existing baseline models and our own ones on our dataset. We envision that this research will inspire and call for further investigation in this evolving and crucial area of 4D perception.
The code is tested with Python 3.9, Pytorch 2.0.1, and CUDA 11.8. Please install dependencies by
conda create -n occ4cast python=3.9
conda activate occ4cast
pip install -r requirements.txt
Please refer to our Hugging Face page for documentation and download.
To train the model, please modify the configurations in baselines/run_train.sh and run
cd baselines
bash run_train.sh
- Add Waymo dataset.
- Add nuScenes dataset.
- Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting, CVPR 2023
- SSCBench: Monocular 3D Semantic Scene Completion Benchmark in Street Views, arXiv 2023
- Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving, arXiv 2023
If you find our work useful in your research, please consider citing:
@article{Liu2023occ4cast,
title={LiDAR-based 4D Occupancy Completion and Forecasting},
author={Xinhao Liu and Moonjun Gong and Qi Fang and Haoyu Xie and Yiming Li and Hang Zhao and Chen Feng},
journal={arXiv preprint arXiv:2310.11239},
year={2023}
}