VCD is a metadata format designed to enable the description of scene information, particularly efficient for discrete data series, such as image or point-cloud sequences from sensor data. Originally, VCD focused on video content data, but has been extended to provide structures to describe, potentially, any type of information of a scene.
VCD is defined as a structure of data, and as such, can be represented as a JSON Schema, or a Google's Protocol Buffer proto file.
The syntax(see vcd_schema_json-v4.3.0.json), as a JSON Schema file, contains the full description of the VCD structure.
More details can be found at the project's website: https://vcd.vicomtech.org
Using pip (Python >3.8)):
pip install vcd
VCD can be also used cloning this repository. And adding the files to a location the Python environment recognizes. You can also use the provided setup.py file to install if from the source:
pip uninstall vcd
python setup.py build
python setup.py install
NPM packages can be used
npm install vcd
VCD Python API exposes functions to load, create, manipulate and serialize VCD content. Samples and use cases can be found in the test folder( see tests).
As a basic example, VCD can be used in a Python script as follows:
import vcd.core as core
import vcd.types as types
# Create a VCD instance
myVCD = core.VCD()
# Add Objects, Actions, etc.
uid = myVCD.add_object(name='someName', semantic_type='#Pedestrian')
myVCD.add_object_data(uid=uid, object_data=types.bbox(name="head", val=[0, 0, 100, 200]))
...
# Serialize
myVCD.stringify(pretty=False, validation=True)The API contains useful functions that ensures the produced content is compliant with the syntax. Nevertheless, the VCD class allows the user to access directly the content, in the form of a Python dictionary.
import vcd.core as core
import vcd.types as types
# Load a VCD file
myVCD = core.vcd('./tests/etc/vcd430_test_kitti_tracking_0_actions.json')
# Access data directly
metadata = myVCD.data['vcd']['metadata']
# Modify data directly
myVCD['vcd']['objects'][3]['type'] = "#Car"
...
# Serialize
stringified_vcd = myVCD.stringify(pretty=False, validation=True)This can be useful some times, but it is not recommended, as the dictionary may deviate from a valid VCD syntax. To check if a content is valid, the API exposes a validation function:
# Validate
myVCD.validate(stringified_vcd)This validation function is optionally called when saving to JSON files.
The Typescript API follows entirely the Python API, and thus core.py and vcd.core.ts are mostly equivalent. The testing scripts in Typescript and in Python use the same base JSON files.
See examples in nodejs/src/_tests_
VCD is defined as a syntax, and as such, different versions imply differences in the syntax or data structure. In addition, each version has a dedicated library version compatible with it.
Last version is VCD 4.3.0.
Main changes at VCD 4.3.0 from VCD 4.2.0 are:
- Integrated SCL (Scene Configuration Library) into VCD
- Automatic drawing functions for multi-sensor set-ups (e.g. topview)
- Improved API functions for offline VCD edition
- Added Typescript API for web applications
- Common set of test files for Python and Typescript APIs
- Simplified Relations construction
- Preliminar work on Ontology connection with Neo4j
Main changes at VCD 4.2.1 from VCD 4.1.0 are:
- Improved Frame-message creation
- Enhanced API for adding Relations and RDFs
- Added examples for semantic labeling
- General bug fixing and better frame interval management
Main changes at VCD 4.1.0 from VCD 4.0.0 are:
- Enhanced JSON schema self-documentation
- Explicit definition of timestamping and sync information
- Explicit definition of intrinsics parameters for cameras
- Explicit definition of extrinsics parameters for stream (as 4x4 pose matrices)
- Explicit definition of odometry entries
- Reviewed samples and converters
Main changes at VCD 4.0.0 from VCD 3.3.0 are:
- Python API
- Mixed-mode instead of Element-wise or Frame-wise mode: 'Objects', 'Actions', contain the static data, while 'Frames' contain the dynamic part with pointers to the static data
- Elements can contain multiple frameIntervals, instead of just one. This allows to manage "gaps".
- The API has been simplified, and only VCD objects can be created
- The concept of ObjectDataContainer has disappeared, now all information is within "ObjectData" structures
- Frames are listed in the JSON as a dict(), not as an array. Keys are the frame nums.
- Elements are listed in the JSON as a dict(), not as an array. Keys are the uid of elements.
- Timestamp information is now optional for the user, who can insert it as frameProperties (along with intrinsics)
- Relations are timeless: are completely defined by the rdf.subjects and rdf.objects
- All fields in JSON are lowercase, e.g. 'vcd', 'objects'.
- Stringify_frame can be executed to create messages asking for dynamic-only or full (static plus dynamic) information
VCD has evolved as follows:
- VCD 1.0 (2013)
- VCD 2.0 (2014)
- Integrated into Viulib library (module viulib_evaluation)
- Element-wise and Frame-wise modes
- XML and JSON serialization via ASL library
- VCD 3.0 (2018)
- Independent C++ library
- Element-wise and Frame-wise modes
- Multi-sensor support
- JSON serialization via ASL library
- Pixel-wise loss-less compression modes
- Comparison routines
- VCD 4.0 (2019)
- Python library
- Element and Frame-wise mode simultaneously
- Multi-sensor and multi-interval
- Native Python JSON serialization
- Google's Protocol Buffer serialization
- Object data 'num' for single numbers, 'vec' for arrays of numbers
- VCD 4.1-3 (2020)
- Explicit definition of intrinsics, extrinsics and odometry
- Enhanced timestamping and sync information
- Enhanced semantics management (RDF triplets)
- Integrated SCL and complex calibration set-ups
- Drawing functions
- Preliminar work on Ontology and Neo4j connection
VCD has been used in the following projects: Cloud-LSVA, VI-DAS, inLane, P-REACT, EWISA, Viulib, begirale, SmaCS, HEADSTART.
Along with the development of VCD, we are participating in the definition of the incoming labeling standard for the automotive sector: ASAM OpenLABEL.
https://www.asam.net/project-detail/scenario-storage-and-labelling/
VCD 4.3.0 is shaped to be compliant with the format defined in OpenLABEL concept paper. As the standard evolves into a standardisation project, VCD will evolve as well to become the first labeling toolset compliant with the standard.
Vicomtech created VCD in 2013, and since, has maintained VCD syntax and libraries. Developments of VCD were supported and funded by the European Commission (EC) Horizon 2020 programme (project [Cloud-LSVA] (http://cloud-lsva.eu), grant agreement 688099).
VCD was registered at the "Registro territorial de la propiedad intelectual de la comunidad autonoma del Pais Vasco", under number 55-354-17, by the Basque Governement, at 2017/07/07.
Main developers:
- Marcos Nieto - mnieto@vicomtech.org
- Orti Senderos - osenderos@vicomtech.org
Contributors: Thanks to Peter Leskovsky, Mikel Garcia, Gonzalo Pierola, Stefano Masneri, Lorena Garcia, Itziar Urbieta, Andoni Mujika and many others in Vicomtech.
Copyright (c) 2020 Vicomtech
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Part of the work carried out to make VCD a reality has been published in Elsevier SoftwareX journal. If you find VCD useful and want to cite it in your publications, please use the following citation (the paper pdf can be accessed here).
M. Nieto, O. Senderos, and O. Otaegui, “Boosting AI applications: Labeling format for complex datasets,” SoftwareX, 2021, p. 100653, vol. 13 (https://doi.org/10.1016/j.softx.2020.100653).