Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking

Paper | Supplementary | Talk | 2024 Challenge | Leaderboard

NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking

Daniel Dauner^1,2, Marcel Hallgarten^1,5, Tianyu Li³, Xinshuo Weng⁴, Zhiyu Huang^4,6, Zetong Yang³
Hongyang Li³, Igor Gilitschenski^7,8, Boris Ivanovic⁴, Marco Pavone^4,9, Andreas Geiger^1,2, and Kashyap Chitta^1,2

¹University of Tübingen, ²Tübingen AI Center, ³OpenDriveLab at Shanghai AI Lab, ⁴NVIDIA Research
⁵Robert Bosch GmbH, ⁶Nanyang Technological University, ⁷University of Toronto, ⁸Vector Institute, ⁹Stanford University

Advances in Neural Information Processing Systems (NeurIPS), 2024
Track on Datasets and Benchmarks

Highlights

🔥 NAVSIM gathers simulation-based metrics (such as progress and time to collision) for end-to-end driving by unrolling simplified bird's eye view abstractions of scenes for a short simulation horizon. It operates under the condition that the policy has no influence on the environment, which enables efficient, open-loop metric computation while being better aligned with closed-loop evaluations than traditional displacement errors.

Table of Contents

1. Highlights
2. Getting started
3. Changelog
4. License and citation
5. Other resources

Getting started

• Download and installation
• Understanding and creating agents
• Understanding the data format and classes
• Dataset splits vs. filtered training / test splits
• Understanding the PDM Score
• Submitting to the Leaderboard

(back to top)

Changelog

• [2024/09/03] NAVSIM v1.1 release
  • Leaderboard for navtest on Hugging Face
  • Release of baseline checkpoints on Hugging Face
  • Updated docs for submission and paper
  • Code refactoring, formatting, minor fixes
• [2024/04/21] NAVSIM v1.0 release (official devkit version for AGC 2024)
  • Parallelization of metric caching / evaluation
  • Adds Transfuser baseline (see agents)
  • Adds standardized training and test filtered splits (see splits)
  • Visualization tools (see tutorial_visualization.ipynb)
• [2024/04/03] NAVSIM v0.4 release
  • Support for test phase frames of competition
  • Download script for trainval
  • Egostatus MLP Agent and training pipeline
• [2024/03/25] NAVSIM v0.3 release (official devkit version for warm-up phase)
  • Adds code for Leaderboard submission
• [2024/03/11] NAVSIM v0.2 release
  • Easier installation and download
  • mini and test data split integration
  • Privileged Human agent
• [2024/02/20] NAVSIM v0.1 release (initial demo)
  • OpenScene-mini sensor blobs and annotation logs
  • Naive ConstantVelocity agent

(back to top)

License and citation

All assets and code in this repository are under the Apache 2.0 license unless specified otherwise. The datasets (including nuPlan and OpenScene) inherit their own distribution licenses. Please consider citing our paper and project if they help your research.

@inproceedings{Dauner2024NEURIPS, 
	author = {Daniel Dauner and Marcel Hallgarten and Tianyu Li and Xinshuo Weng and Zhiyu Huang and Zetong Yang and Hongyang Li and Igor Gilitschenski and Boris Ivanovic and Marco Pavone and Andreas Geiger and Kashyap Chitta}, 
	title = {NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking}, 
	booktitle = {Advances in Neural Information Processing Systems (NeurIPS)}, 
	year = {2024}, 
}

@misc{Contributors2024navsim,
    title={NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking},
    author={NAVSIM Contributors},
    howpublished={\url{https://github.com/autonomousvision/navsim}},
    year={2024}
} 

(back to top)

Other resources

• SLEDGE | tuPlan garage | CARLA garage | Survey on E2EAD
• PlanT | KING | TransFuser | NEAT

(back to top)