Colab example for running simulation with ScenarioNet:
Colab example for reading established ScenarioNet dataset:
In this example, we will show you how to convert a small batch of Waymo scenarios into the internal Scenario Description. After that, the scenarios will be loaded to MetaDrive simulator for closed-loop simulation. First of all, please install MetaDrive and ScenarioNet following these steps :ref:`installation`.
For any dataset, the first step after installing ScenarioNet is to install the corresponding official toolkit as we need to use it to parse the original data and convert to our internal scenario description. For Waymo data, we already have the parser in ScenarioNet so just install the TensorFlow and Protobuf via:
pip install tensorflow==2.11.0 conda install protobuf==3.20
Note
You may fail to install protobuf
if using pip install protobuf==3.20
. If so, install via conda install protobuf=3.20
.
For other datasets like nuPlan and nuScenes, you need to setup nuplan-devkit and nuscenes-devkit respectively. Guidance on how to setup these datasets and connect them with ScenarioNet can be found at :ref:`datasets`.
Access the Waymo motion data at Google Cloud.
Download one tfrecord scenario file from waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s
.
In this tutorial, we only use the first file training_20s.tfrecord-00000-of-01000
.
Just click the download button |:arrow_down:| on the right side to download it.
And place the downloaded tfrecord file to a folder. Let's call it exp_waymo
and the structure is like this:
exp_waymo ├──training_20s.tfrecord-00000-of-01000
Note
For building database from all scenarios, install gsutil
and use this command:
gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s" .
Likewise, place all downloaded tfrecord files to the same folder.
Run the following command to extract scenarios in exp_waymo
to exp_converted
:
python -m scenarionet.convert_waymo -d /path/to/exp_converted/ --raw_data_path /path/to/exp_waymo --num_files=1
Note
When running python -m
, make sure the directory you are at doesn't contain a folder called scenarionet
.
Otherwise, the running may fail. For more details about the command, use python -m scenarionet.convert_waymo -h
Now all extracted scenarios will be placed in exp_converted
directory.
If we list the directory with ll
command, the structure will be like:
exp_converted ├──exp_converted_0 ├──exp_converted_1 ├──exp_converted_2 ├──exp_converted_3 ├──exp_converted_4 ├──exp_converted_5 ├──exp_converted_6 ├──exp_converted_7 ├──dataset_mapping.pkl ├──dataset_summary.pkl
This is because we use 8 workers to extract the scenarios, and thus the converted scenarios will be stored in 8 subfolders.
If we go check exp_converted_0
, we will see the structure is like:
├──sd_waymo_v1.2_2085c5cffcd4727b.pkl ├──sd_waymo_v1.2_27997d88023ff2a2.pkl ├──sd_waymo_v1.2_3ece8d267ce5847c.pkl ├──sd_waymo_v1.2_53e9adfdac0eb822.pkl ├──sd_waymo_v1.2_8e40ffb80dd2f541.pkl ├──sd_waymo_v1.2_df72c5dc77a73ed6.pkl ├──sd_waymo_v1.2_f1f6068fabe77dc8.pkl ├──dataset_mapping.pkl ├──dataset_summary.pkl
Therefore, the subfolder produced by each worker is actually where the converted scenarios are placed.
To aggregate the scenarios produced by all workers, the exp_converted/dataset_mapping.pkl
stores the mapping
from scenario_id to the path of the target scenario file relative to exp_converted
.
As a result, we can get all scenarios produced by 8 workers by loading the database exp_converted.
Several basic operations are available and allow us to split, merge, move, and check the databases.
First of all, let's check how many scenarios are included in this database built from training_20s.tfrecord-00000-of-01000
:
python -m scenarionet.num -d /path/to/exp_converted/
It will show that there are totally 61 scenarios. For machine learning applications, we usually want to split training/test sets. To this end, we can use the following command to build the training set:
python -m scenarionet.split --from /path/to/exp_converted/ --to /path/to/exp_train --num_scenarios 40
Again, use the following commands to build the test set:
python -m scenarionet.split --from /path/toexp_converted/ --to /path/to/exp_test --num_scenarios 21 --start_index 40
We add the start_index
argument to select the last 21 scenarios as the test set.
To ensure that no overlap exists, we can run this command:
python -m scenarionet.check_overlap --d_1 /path/to/exp_train/ --d_2 /path/to/exp_test/
It will report No overlapping in two database!.
Now, let's suppose that the /exp_train/
and /exp_test/
are two databases built
from different source and we want to merge them into a larger one.
This can be achieved by:
python -m scenarionet.merge --from /path/to/exp_train/ /path/to/exp_test -d /path/to/exp_merged
Let's check if the merged database is the same as the original one:
python -m scenarionet.check_overlap --d_1 /path/to/exp_merged/ --d_2 /path/to/exp_converted
It will show there are 61 overlapped scenarios. Congratulations! Now you are already familiar with some common operations. More operations and details is available at :ref:`operations`.
The database can be loaded to MetaDrive simulator for scenario replay or closed-loop simulation.
First of all, let's replay scenarios in the exp_converted
database:
python -m scenarionet.sim -d /path/to/exp_converted --render 2D
By adding --render 3D
flag, we can use 3D renderer:
python -m scenarionet.sim -d /path/to/exp_converted --render 3D
Note
--render advanced
enables the advanced deferred rendering pipeline,
but an advanced GPU better than RTX 2060 is required.