- Peter Gagliardi, Cesium
- Sean Lilley, Cesium
- Sam Suhag, Cesium
- Don McCurdy, Independent
- Marco Hutter, Cesium
- Bao Tran, Cesium
- Samuel Vargas, Cesium
- Patrick Cozzi, Cesium
Draft
Written against the glTF 2.0 specification.
- Overview
- Schema Definitions
- Metadata Storage
- Assigning Metadata
- Optional vs. Required
- Schema
- Revision History
This extension defines a means of storing structured metadata within a glTF 2.0 asset. The key concepts of this extension are the definition of a schema that describes the structure of the metadata, and methods for storing and associating metadata with different entities within the asset.
The metadata schema definition includes a JSON representation of the schema structure, suitable for being stored inside a glTF asset. The storage formats that are defined in this extension allow storing metadata in standard glTF vertex attributes, or in the channels of a texture. Both representations are compact binary storage formats that are appropriate for efficiently transmitting large quantities of metadata.
The schema definition and the storage formats in this extension are implementations of the 3D Metadata Specification. This specification should be considered a normative reference for definitions and requirements. This document provides inline definitions of terms where appropriate.
Disambiguation: glTF has other methods of storing details that could similarly be described as metadata or properties, including
KHR_xmp_json_ld
, Extras, and Extensions. While those methods associate data with discrete objects in a glTF asset — nodes, materials, etc. —EXT_structural_metadata
is uniquely suited for properties of more granular conceptual features in subregions composed of vertices or texels.
Data types and meanings of properties are provided by a schema, as defined in the 3D Metadata Specification and summarized below.
Defined in schema.schema.json.
Top-level definitions for the structure and data types of properties. The schema provides a set of classes and enums the asset can reference.
A schema may be embedded in the extension directly or referenced externally with the schemaUri
property. Multiple glTF assets may refer to the same external schema to avoid duplication. A schema is defined by an EXT_structural_metadata
extension attached to the glTF root object.
Example: A simple schema defining enums and classes.
{ "extensions": { "EXT_structural_metadata": { "schema": { "id": "schema_001", "name": "Schema 001", "description": "An example schema.", "version": "3.5.1", "enums": { ... }, "classes": { ... } } } } }
Defined in class.schema.json.
Template for metadata entities. Classes provide a list of property definitions. Instances of a class can be created from property values that conform to the class's property definitions.
Classes are defined as entries in the schema.classes
dictionary, indexed by class ID. Class IDs must be alphanumeric identifiers matching the regular expression ^[a-zA-Z_][a-zA-Z0-9_]*$
.
Example: A "Tree" class, which might describe a table of tree measurements taken in a park. Property definitions are abbreviated here, and introduced in the next section.
{ "extensions": { "EXT_structural_metadata": { "schema": { "classes": { "tree": { "name": "Tree", "description": "Woody, perennial plant.", "properties": { "species": { ... }, "age": { ... }, "height": { ... }, "diameter": { ... } } } } } } } }
Defined in class.property.schema.json.
Class properties are defined abstractly in a class. The class is instantiated with specific values conforming to these properties. Class properties support a richer variety of data types than glTF accessors or GPU shading languages allow. Details about the supported types can be found in the 3D Metadata Specification.
Class properties are defined as entries in the class.properties
dictionary, indexed by property ID. Property IDs must be alphanumeric identifiers matching the regular expression ^[a-zA-Z_][a-zA-Z0-9_]*$
.
Example: A "Tree" class, which might describe a table of tree measurements taken in a park. Properties include species, height, and diameter of each tree, as well as the number of birds observed in its branches.
{ "extensions": { "EXT_structural_metadata": { "schema": { "classes": { "tree": { "name": "Tree", "description": "Woody, perennial plant.", "properties": { "species": { "description": "Type of tree.", "type": "ENUM", "enumType": "speciesEnum", "required": true }, "age": { "description": "The age of the tree, in years", "type": "SCALAR", "componentType": "UINT8", "required": true }, "height": { "description": "Height of tree measured from ground level, in meters.", "type": "SCALAR", "componentType": "FLOAT32" }, "diameter": { "description": "Diameter at trunk base, in meters.", "type": "SCALAR", "componentType": "FLOAT32" } } } } } } } }
Defined in enum.schema.json.
Set of categorical types, defined as (name, value)
pairs. Enum properties use an enum as their type.
Enums are defined as entries in the schema.enums
dictionary, indexed by an enum ID. Enum IDs must be alphanumeric identifiers matching the regular expression ^[a-zA-Z_][a-zA-Z0-9_]*$
.
Example: A "Species" enum defining types of trees. An "Unspecified" enum value is optional, but when provided as the
noData
value for a property (see: 3D Metadata → No Data Values) may be helpful to identify missing data.{ "extensions": { "EXT_structural_metadata": { "schema": { "enums": { "speciesEnum": { "name": "Species", "description": "An example enum for tree species.", "values": [ {"name": "Unspecified", "value": 0}, {"name": "Oak", "value": 1}, {"name": "Pine", "value": 2}, {"name": "Maple", "value": 3} ] } } } } } }
Defined in enum.value.schema.json.
Pairs of (name, value)
entries representing possible values of an enum property.
Enum values are defined as entries in the enum.values
array. Duplicate names or duplicate integer values are not allowed.
The classes defined in the schema are templates describing the data types and meanings of properties. An instance of such a metadata class is referred to as a metadata entity, and can be created from a set of values that conform to the structure of the class. This extension defines different ways of storing large amounts of property values inside a glTF asset, in compact binary forms:
- Property Tables store property values as parallel arrays in a column-based binary layout, using standard glTF buffer views. These tables can be accessed with a row index, and allow associating complex, structured metadata with arbitrary types with entities of a glTF asset on different levels of granularity.
- Property Attributes associate vertex attributes of a mesh primitive with a particular metadata class.
- Property Textures store property values in channels of a texture, suitable for very high-frequency data mapped to less-detailed 3D surfaces.
Each storage type refers to a metadata class, and contains a dictionary of properties
. Each of these properties corresponds to one property of the metadata class and defines how the actual property data is stored. These property storage definitions may override the minimum
and maximum
values and the offset
and scale
from the property definition in the class, to account for the actual range of values that is stored for each property.
The following sections describe these storage formats in more detail.
Defined in propertyTable.schema.json.
Each property table defines a specified number (count
) of metadata entities conforming to a particular class (class
), with property values stored as parallel arrays in a column-based binary layout. Property tables support a richer variety of data types than glTF accessors or GPU shading languages allow, and are suitable for datasets that can be expressed in a tabular layout.
Property tables are defined as entries in the propertyTables
array of the root-level EXT_structural_metadata
extension, and may be referenced by assigning metadata to glTF elements, or by other extensions.
The property table may provide value arrays for only a subset of the properties of its class, but class properties marked required: true
must not be omitted. Each property value array given by the property table must be defined by a class property with the same alphanumeric property ID, with values matching the data type of the class property.
Example: A
tree_survey_2021-09-29
property table, implementing thetree
class defined in earlier examples. The table contains observations for 10 trees, with each property defined by a buffer view containing 10 values.The glTF asset contains multiple objects (trees) that are associated with unique identifiers. These identifiers can be assigned to the objects in different ways. For example, using the
EXT_mesh_features
extension, theEXT_instance_features
extension, or an application-specific mechanism for identifying the objects. The identifiers then serve as an index into the property table row that stores the property values for the properties that are defined in thetree
class.{ "extensions": { "EXT_structural_metadata": { "schema": { ... }, "propertyTables": [{ "name": "tree_survey_2021-09-29", "class": "tree", "count": 10, "properties": { "species": { "values": 0, }, "age": { "values": 1 }, "height": { "values": 2 }, // "diameter" is not required and has been omitted from this table. } }] } } }
Property arrays are stored in glTF buffer views and use the binary encoding defined in the Binary Table Format section of the 3D Metadata Specification.
As in the core glTF specification, values of NaN
, +Infinity
, and -Infinity
are never allowed.
Each buffer view byteOffset
must be aligned to a multiple of its component size.
Implementation note: Authoring tools may choose to align all buffer views to 8-byte boundaries for consistency, but client implementations should only depend on 8-byte alignment for buffer views containing 64-bit component types.
Defined in propertyAttribute.schema.json.
Property attributes associate vertex attributes of a mesh primitive with a metadata class. They refer to a certain class from the schema definition via their class
property, and contain a properties
dictionary that defines a set of properties that conform to this class. Each property refers to an attribute that may be stored in a mesh primitive.
The property types that are supported via property attributes are therefore restricted to the types that are supported by standard glTF accessors. These types are a strict subset of the types that are supported with the schema definitions in this extension.
Example:
An example of a property attribute that represents information about the movement of points in a point cloud.
The schema defines a class called
movement
. It has adirection
property that is a normalized 3D float vector for the movement direction, and amagnitude
property that describes the magnitude of the movement.The top-level
propertyAttributes
array contains a property attribute that refers to this class. Themovement
anddirection
properties of the class are associated with attributes called_DIRECTION
and_MAGNITUDE
.The mesh primitive defines (non-indexed) vertices with primitive mode 0, and thus, represents a simple point cloud, with the positions of the points being stored in the
POSITION
attribute. Additionally, it defines vertex attributes_DIRECTION
and_MAGNITUDE
, which contain the data for the properties from the property attribute.Top-level extension object:
{ "extensions": { "EXT_structural_metadata": { "schema": { "classes": { "movement": { "name": "movement", "description": "The movement of points in a point cloud", "properties": { "direction": { "description": "The movement direction, as a normalized 3D vector", "type": "VEC3", "componentType": "FLOAT32", "required": true }, "magnitude": { "description": "The magnitude of the movement", "type": "SCALAR", "componentType": "FLOAT32", "required": true } } } } }, "propertyAttributes": [{ "class": "movement", "properties": { "direction": { "attribute": "_DIRECTION", }, "magnitude": { "attribute": "_MAGNITUDE", } } }] } } }Primitive
{ "primitives": [ { "mode:": 0, "attributes": { "POSITION": 0, "_DIRECTION": 1, "_MAGNITUDE": 2, }, "extensions": { "EXT_structural_metadata": { "propertyAttributes": [0] } } } ] }
Defined in propertyTexture.schema.json.
Property textures use texture channels to store property values conforming to a particular class (identified by ID class
), with those values accessed directly by texture coordinates. Property textures are especially useful when texture mapping high frequency data to less detailed 3D surfaces. Unlike textures used in glTF materials, property textures are not necessarily visible in a rendered scene. Like property tables, property textures are implementations of the 3D Metadata Specification.
Property textures are defined as entries in the propertyTextures
array of the root-level EXT_structural_metadata
extension, and may be referenced by extensions on primitive objects.
A property texture may provide channels for only a subset of the properties of its class, but class properties marked required: true
must not be omitted.
Example: Property texture implementing a "wall" class, with property values stored in a glTF texture at index 0 and indexed by
TEXCOORD_0
. Each property of the"wall"
class is stored in one channel of the texture.Class and property texture
{ "extensions": { "EXT_structural_metadata": { "schema": { "classes": { "wall": { "name": "Wall Temperature vs. Insulation", "properties": { "insideTemperature": { "name": "Inside Temperature", "type": "SCALAR", "componentType": "UINT8" }, "outsideTemperature": { "name": "Outside Temperature", "type": "SCALAR", "componentType": "UINT8" }, "insulation": { "name": "Insulation Thickness", "type": "SCALAR", "componentType": "UINT8", "normalized": true } } } } }, "propertyTextures": [ { "class": "wall", "properties": { "insideTemperature": { "index": 0, "texCoord": 0, "channels": [0] }, "outsideTemperature": { "index": 0, "texCoord": 0, "channels": [1] }, "insulation": { "index": 0, "texCoord": 0, "channels": [2] } } } ] } } }Primitive
{ "primitives": [ { "attributes": { "POSITION": 0, "TEXCOORD_0": 1 }, "indices": 2, "material": 0, "extensions": { "EXT_structural_metadata": { "propertyTextures": [0] } } } ] }
Each property that is defined in the propertyTexture
object extends the glTF textureInfo
object. The texCoord
specifies a texture coordinate set in the referring primitive. The index
is the index of the glTF texture object that stores the actual data. Additionally, each property specifies an array of channels
, which are the indices of the texture channels providing data for the respective property. Channels of an RGBA
texture are numbered 0–3 respectively.
Texture filtering must be 9728
(NEAREST), 9729
(LINEAR), or undefined, for any texture object referenced as a property texture. Texture values must be encoded with a linear transfer function.
Example: A property texture for 2D wind velocity samples. The "speed" property values are stored in the red channel. The "direction" property values are stored as a unit-length vector, with X/Y components in the green and blue channels. Both properties are indexed by UV coordinates in a
TEXCOORD_0
attribute.// Root EXT_structural_metadata extension: { "propertyTextures": [ { "class": "wind", "properties": { "speed": { "index": 0, "texCoord": 0, "channels": [0] }, "direction": { "index": 0, "texCoord": 0, "channels": [1, 2] } } } ] }
Multiple channels of a property texture can be used to represent individual bytes of larger data types. The values from the selected channels represent the bytes of the actual property value, in little-endian order.
Implementation note: Specialized texture formats may allow additional channels, or channels with a higher number of bits per channel. The usage of such texture formats for property textures has to be defined by additional extensions.
Certain property types cannot be encoded in property textures. For example, variable-length arrays or strings are not supported. Enum values may be encoded as integer values according to their enum value type (see Enum). For other property types, the number of channels that are selected must match the number of bytes of the property type.
Example:
If a property is defined to store (single)
FLOAT32
components, then these values can be stored in the 4 channels of a property texture. The raw bits of the property value can be computed asvec4<uint8> rgba = texture(sampler, coordinates); uint8 byte0 = rgba[channels[0]]; uint8 byte1 = rgba[channels[1]]; uint8 byte2 = rgba[channels[2]]; uint8 byte3 = rgba[channels[3]]; uint32 rawBits = byte0 | (byte1 << 8) | (byte2 << 16) | (byte3 << 24); float32 value = uintBitsToFloat(rawBits);
If a property has the type
VEC2
withUIN16
components, or an array with a fixed length of 2 andUINT16
components, then the respective property can be represented with 4 channels as well:vec4<uint8> rgba = texture(sampler, coordinates); uint8 byte0 = rgba[channels[0]]; uint8 byte1 = rgba[channels[1]]; uint8 byte2 = rgba[channels[2]]; uint8 byte3 = rgba[channels[3]]; value[0] = byte0 | (byte1 << 8); value[1] = byte2 | (byte3 << 8);
Property values are stored in a compact binary tabular format described in the 3D Metadata Specification, with each property table array occupying a glTF buffer view. EXT_structural_metadata
imposes 8-byte binary data alignment requirements on an asset, allowing support for 64-bit data types while remaining compatible with the 4-byte alignments in the core glTF specification:
- GLB-stored
JSON
chunk must be padded with trailingSpace
characters (0x20
) to 8-byte boundary. - GLB-stored
BIN
chunk must be padded with trailing zeroes (0x00
) to 8-byte boundary.
As a result, byte length of the BIN
chunk may be up to 7 bytes larger than JSON-defined buffer.byteLength
to satisfy alignment requirements.
Defined in EXT_structural_metadata.schema.json.
When property values are stored in a Property Table, then the entries of this table may be referenced from within the glTF asset: Each node
of the glTF asset can contain an EXT_structural_metadata
object that defines the source of the metadata values for this node. It contains the propertyTable
, which is the index of the property table in the array of property tables of the root-level EXT_structural_metadata
extension object, and the index
, which is the index of the row in this table that contains the metadata values for the respective node.
Example:
An example of metadata that is assigned to a node. It associates the given node with the metadata values that are stored in row 4 of the property table with index 1, referring to the array of property tables that are defined in the top-level
EXT_structural_metadata
extension object.{ "extensions": { "EXT_structural_metadata": { "schema": { ... }, "propertyTables": [ ... ] } }, "nodes": [ ... { "name": "A node with metadata", "EXT_structural_metadata": { "propertyTable": 1, "index": 4 } } ] }
This extension is optional, meaning it should be placed in the extensionsUsed
list, but not in the extensionsRequired
list.
The revision history of this extension can be found in the common revision history of the 3D Tiles Next extensions.