Skip to content
The devkit of the nuScenes dataset.
Branch: master
Clone or download
holger-nutonomy Detection eval results format and readme (#115)
* Added meta data to submission and rewording

* Overhauled readme and comments

* Make it explicit that 0.5s time window is only relevant at test time

* Modified eval code for new result format

* Renamed weighted_sum to nd_score

* Clarification on maximum time window

* Added option to use test set with annotations for eval server

* Also writing meta data to output file

* Write only a subset of examples to disk, create example folder

* Make sure test GT is not visualized and plot 10 examples by default.

* Print metrics to stdout

* Standardize number format

* Write more user-friendly metric names to stdout

* Refactoring: Moved writing to disk/stdout one level up, renamed run() to evaluate(), made main a class method

* Option to not render curves

* Fixed return type and docstring

* Save mean_dist_aps to directly show them in the leaderboard
Latest commit 21771d6 Apr 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
python-sdk Detection eval results format and readme (#115) Apr 12, 2019
setup Export 2d annotations (#97) Apr 10, 2019
.gitignore Added instructions and cleaned up code Sep 14, 2018
README.md Add paper references (#101) Mar 27, 2019
instructions.md
license.txt
schema.md Release v1.0.0 (#64) Mar 25, 2019

README.md

nuScenes devkit

Welcome to the devkit of the nuScenes dataset.

Overview

Changelog

  • Mar. 26, 2019: Full dataset, paper, & devkit v1.0.0 released. Support dropped for teaser data.
  • Dec. 20, 2018: Initial evaluation code released. Devkit folders restructured.
  • Nov. 21, 2018: RADAR filtering and multi sweep aggregation.
  • Oct. 4, 2018: Code to parse RADAR data released.
  • Sep. 12, 2018: Devkit for teaser dataset released.

Dataset download

To download nuScenes you need to go to the Download page, create an account and agree to the nuScenes Terms of Use. After logging in you will see multiple archives. For the devkit to work you will need to download all archives. Please unpack the archives to the /data/sets/nuscenes folder *without* overwriting folders that occur in multiple archives. Eventually you should have the following folder structure:

/data/sets/nuscenes
    samples	-	Sensor data for keyframes.
    sweeps	-	Sensor data for intermediate frames.
    maps	-	Large image files (~500 Gigapixel) that depict the drivable surface and sidewalks in the scene.
    v1.0-*	-	JSON tables that include all the meta data and annotations. Each split (trainval, test, mini) is provided in a separate folder.

If you want to use another folder, specify the dataroot parameter of the NuScenes class (see tutorial).

Devkit setup

The devkit is tested for Python 3.6 and Python 3.7. To install python, please check here.

Our devkit is available and can be installed via pip:

pip install nuscenes-devkit

If you don't have pip, please check here to install pip.

For an advanced installation, see installation for detailed instructions.

Tutorial

To get started with the nuScenes devkit, please run the tutorial as an IPython notebook:

jupyter notebook $HOME/nuscenes-devkit/python-sdk/tutorial.ipynb

In case you want to avoid downloading and setting up the data, you can also take a look at the rendered notebook on nuScenes.org. To learn more about the dataset, go to nuScenes.org or take a look at the database schema and annotator instructions. The nuScenes paper provides detailed analysis of the dataset.

Object detection task

For instructions related to the object detection task (results format, classes and evaluation metrics), please refer to this readme.

Backward compatibility

  • Mar. 26, 2019: With the full dataset release we drop support for the code and data of the teaser release. Several changes to the map table and map files break backward compatibility.
  • Dec. 20, 2018: We restructured the nuscenes-devkit code, which breaks backward compatibility. The new structure has a top-level package nuscenes which contains packages eval, export and utils. Therefore, existing imports from nuscenes_utils should be replaced by nuscenes.nuscenes.

Citation

Please use the following citation when referencing nuScenes:

@article{nuscenes2019,
  title={nuScenes: A multimodal dataset for autonomous driving},
  author={Holger Caesar and Varun Bankiti and Alex H. Lang and Sourabh Vora and 
          Venice Erin Liong and Qiang Xu and Anush Krishnan and Yu Pan and 
          Giancarlo Baldan and Oscar Beijbom},
  journal={arXiv preprint arXiv:1903.11027},
  year={2019}
}

You can’t perform that action at this time.