This topic describes how to manage the "CADC" dataset.
"CADC" is a fusion dataset with 8 sensors<advanced_features/fusion_dataset/fusion_dataset_structure:sensor> including 7 cameras<tensorbay.sensor.sensor.Camera> and 1 lidar<tensorbay.sensor.sensor.Lidar> , and has reference/label_format:Box3D type of labels on the point cloud data. (Fig. %s <example-cadc>). See this page for more details about this dataset.
The preview of a point cloud from "CADC" with Box3D labels.
First of all, create a GAS client.
../../../../docs/code/CADC.py
Then, create a fusion dataset client by passing the fusion dataset name and is_fusion argument to the GAS client.
../../../../docs/code/CADC.py
To check if you have created "CADC" fusion dataset, you can list all your available datasets. See this page <features/dataset_management:Read Dataset> for details.
The datasets listed here include both datasets <reference/dataset_structure:dataset> and fusion datasets <advanced_features/fusion_dataset/fusion_dataset_structure:fusion dataset>.
../../../../docs/code/CADC.py
Now we describe how to organize the "CADC" fusion dataset by the ~tensorbay.dataset.dataset.FusionDataset instance before uploading it to TensorBay. It takes the following steps to organize "CADC".
The first step is to write the reference/dataset_structure:Catalog. Catalog is a json file contains all label information of one dataset. See this page <reference/dataset_structure:Catalog> for more details. The only annotation type for "CADC" is reference/label_format:Box3D, and there are 10 reference/label_format:Category types and 9 reference/label_format:Attributes types.
../../../../tensorbay/opendataset/CADC/catalog.json
Note
The annotations for "CADC" have tracking information, hence the value of isTracking should be set as True.
The second step is to write the reference/glossary:Dataloader. The reference/glossary:Dataloader function of "CADC" is to manage all the files and annotations of "CADC" into a ~tensorbay.dataset.dataset.FusionDataset instance. The code block <cadc-dataloader> below displays the "CADC" dataloader.
../../../../tensorbay/opendataset/CADC/loader.py
To load a fusion dataset, we first need to create an instance of ~tensorbay.dataset.dataset.FusionDataset.(L75)
Note that after creating the advanced_features/fusion_dataset/fusion_dataset_structure:fusion dataset, you need to set the is_continuous attribute of ~tensorbay.dataset.DatasetBase.notes to True,(L76) since the frames<advanced_features/fusion_dataset/fusion_dataset_structure:frame> in each advanced_features/fusion_dataset/fusion_dataset_structure:fusion segment is time-continuous.
Same as dataset, you also need to load the reference/dataset_structure:catalog.(L77) The catalog file "catalog.json" is in the same directory with dataloader file.
In this example, we create fusion segments by dataset.create_segment(SEGMENT_NAME).(L86) We manage the data under the subfolder(L33) of the date folder(L32) into a fusion segment and combine two folder names to form a segment name, which is to ensure that frames in each segment are continuous.
After constructing the fusion segment, the sensors<advanced_features/fusion_dataset/fusion_dataset_structure:sensor> corresponding to different data should be added to the fusion segment.(L87)
In "CADC" , there is a need for projection, so we need not only the name for each sensor, but also the calibration parameters.
And to manage all the ~tensorbay.sensor.sensor.Sensors (L81, L183) corresponding to different data, the parameters from calibration files are extracted.
~tensorbay.sensor.sensor.Lidar sensor only has ~tensorbay.sensor.sensor.Sensor.extrinsics, here we regard the lidar as the origin of the point cloud 3D coordinate system, and set the extrinsics as defaults(L189).
To keep the projection relationship between sensors, we set the transform from the camera 3D coordinate system to the lidar 3D coordinate system as ~tensorbay.sensor.sensor.Camera extrinsics(L205).
Besides ~tensorbay.sensor.sensor.Sensor.extrinsics, ~tensorbay.sensor.sensor.Camera sensor also has ~tensorbay.sensor.sensor.Camera.intrinsics, which are used to project 3D points to 2D pixels.
The intrinsics consist of two parts, ~tensorbay.sensor.intrinsics.CameraMatrix and ~tensorbay.sensor.intrinsics.DistortionCoefficients.(L208-L211)
After adding the sensors to the fusion segments, the frames should be added into the continuous segment in order(L96).
Each frame contains the data corresponding to each sensor, and each data should be added to the frame under the key of sensor name(L147).
In fusion datasets, it is common that not all data have labels. In "CADC", only point cloud files(Lidar data) have reference/label_format:Box3D type of labels(L145). See this page <reference/label_format:Box3D> for more details about Box3D annotation details.
Note
The CADC dataloader <cadc-dataloader> above uses relative import(L16-L19). However, when you write your own dataloader you should use regular import. And when you want to contribute your own dataloader, remember to use relative import.
Optionally, the organized dataset can be visualized by Pharos, which is a TensorBay SDK plug-in. This step can help users to check whether the dataset is correctly organized. Please see features/visualization:Visualization for more details.
After you finish the reference/glossary:Dataloader and organize the "CADC" into a ~tensorbay.dataset.dataset.FusionDataset instance, you can upload it to TensorBay for sharing, reuse, etc.
../../../../docs/code/CADC.py
Remember to execute the commit step after uploading. If needed, you can re-upload and commit again. Please see this page <features/version_control:Version Control> for more details about version control.
Note
Commit operation can also be done on our GAS Platform.
Now you can read "CADC" dataset from TensorBay.
../../../../docs/code/CADC.py
In reference/dataset_structure:Dataset "CADC", there are lots of FusionSegments <reference/dataset_structure:Segment>: 2018_03_06/0001, 2018_03_07/0001, ...
You can get the segment names by list them all.
../../../../docs/code/CADC.py
You can get a segment by passing the required segment name.
../../../../docs/code/CADC.py
Note
If the reference/dataset_structure:Segment or advanced_features/fusion_dataset/fusion_dataset_structure:fusion segment is created without given name, then its name will be "".
In the 2018_03_06/0001 advanced_features/fusion_dataset/fusion_dataset_structure:fusion segment, there are several sensors<advanced_features/fusion_dataset/fusion_dataset_structure:sensor>. You can get all the sensors by accessing the sensors<tensorbay.sensor.sensor.Sensors> of the ~tensorbay.dataset.dataset.FusionSegment.
../../../../docs/code/CADC.py
In each advanced_features/fusion_dataset/fusion_dataset_structure:fusion segment, there are a sequence of frames<advanced_features/fusion_dataset/fusion_dataset_structure:frame>. You can get one by index.
../../../../docs/code/CADC.py
In each advanced_features/fusion_dataset/fusion_dataset_structure:frame, there are several data<advanced_features/fusion_dataset/fusion_dataset_structure:data in fusion dataset> corresponding to different sensors. You can get each data by the corresponding sensor name.
../../../../docs/code/CADC.py
In "CADC", only data<advanced_features/fusion_dataset/fusion_dataset_structure:data in fusion dataset> under ~tensorbay.sensor.sensor.Lidar has a sequence of reference/label_format:Box3D annotations. You can get one by index.
../../../../docs/code/CADC.py
There is only one label type in "CADC" dataset, which is box3d. The information stored in reference/label_format:Category is one of the category names in "categories" list of catalog.json <cadc-catalog>. The information stored in reference/label_format:Attributes is some of the attributes in "attributes" list of catalog.json <cadc-catalog>.
See this page <reference/label_format:Box3D> for more details about the structure of Box3D.
To delete "CADC", run the following code:
../../../../docs/code/CADC.py