Skip to content

Latest commit

 

History

History
2914 lines (2322 loc) · 93.3 KB

Specification.adoc

File metadata and controls

2914 lines (2322 loc) · 93.3 KB

glTF 2.0 Interactivity Extension Specification

Table of Contents
Note
Note

Khronos posts the AsciiDoc source of the glTF specification to enable community feedback and remixing under CC-BY 4.0. Published versions of the Specification are located in the glTF Registry.

1. Foreword

Copyright 2013-2023 The Khronos Group Inc.

This specification is protected by copyright laws and contains material proprietary to Khronos. Except as described by these terms, it or any components may not be reproduced, republished, distributed, transmitted, displayed, broadcast, or otherwise exploited in any manner without the express prior written permission of Khronos.

This specification has been created under the Khronos Intellectual Property Rights Policy, which is Attachment A of the Khronos Group Membership Agreement available at https://www.khronos.org/files/member_agreement.pdf. Khronos grants a conditional copyright license to use and reproduce the unmodified specification for any purpose, without fee or royalty, EXCEPT no licenses to any patent, trademark or other intellectual property rights are granted under these terms. Parties desiring to implement the specification and make use of Khronos trademarks in relation to that implementation, and receive reciprocal patent license protection under the Khronos IP Policy must become Adopters under the process defined by Khronos for this specification; see https://www.khronos.org/conformance/adopters/file-format-adopter-program.

Some parts of this Specification are non-normative through being explicitly identified as purely informative, and do not define requirements necessary for compliance and so are outside the Scope of this Specification.

Where this Specification includes normative references to external documents, only the specifically identified sections and functionality of those external documents are in Scope. Requirements defined by external documents not created by Khronos may contain contributions from non-members of Khronos not covered by the Khronos Intellectual Property Rights Policy.

Khronos makes no, and expressly disclaims any, representations or warranties, express or implied, regarding this specification, including, without limitation: merchantability, fitness for a particular purpose, non-infringement of any intellectual property, correctness, accuracy, completeness, timeliness, and reliability. Under no circumstances will Khronos, or any of its Promoters, Contributors or Members, or their respective partners, officers, directors, employees, agents or representatives be liable for any damages, whether direct, indirect, special or consequential damages for lost revenues, lost profits, or otherwise, arising from or in connection with these materials.

Khronos® and Vulkan® are registered trademarks, and ANARI™, WebGL™, glTF™, NNEF™, OpenVX™, SPIR™, SPIR‑V™, SYCL™, OpenVG™ and 3D Commerce™ are trademarks of The Khronos Group Inc. OpenXR™ is a trademark owned by The Khronos Group Inc. and is registered as a trademark in China, the European Union, Japan and the United Kingdom. OpenCL™ is a trademark of Apple Inc. and OpenGL® is a registered trademark and the OpenGL ES™ and OpenGL SC™ logos are trademarks of Hewlett Packard Enterprise used under license by Khronos. ASTC is a trademark of ARM Holdings PLC. All other product names, trademarks, and/or company names are used solely for identification and belong to their respective owners.

2. Introduction

2.1. General

This document, referred to as the “glTF Interactivity Extension Specification” or just the “Specification” hereafter, describes the KHR_interactivity glTF extension.

This extension aims to enhance glTF 2.0 by adding the ability to encode behavior and interactivity in 3D assets.

Note
Note

This specification is for single user experiences only and does not deal with any of the complexity involved in multi-user networked experiences.

2.2. Document Conventions

The glTF Interactivity Extension Specification is intended for use by both implementers of the asset exporters or converters (e.g., digital content creation tools) and application developers seeking to import or load interactive glTF assets, forming a basis for interoperability between these parties.

Specification text can address either party; typically, the intended audience can be inferred from context, though some sections are defined to address only one of these parties.

Any requirements, prohibitions, recommendations, or options defined by normative terminology are imposed only on the audience of that text.

2.2.1. Normative Terminology and References

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in BCP 14.

These key words are highlighted in the specification for clarity.

References to external documents are considered normative if the Specification uses any of the normative terms defined in this section to refer to them or their requirements, either as a whole or in part.

2.2.2. Informative Language

Some language in the specification is purely informative, intended to give background or suggestions to implementers or developers.

If an entire chapter or section contains only informative language, its title is suffixed with “(Informative)”. If not designated as informative, all chapters, sections, and appendices in this document are normative.

All Notes, Implementation notes, and Examples are purely informative.

2.2.3. Technical Terminology

TBD

2.2.4. Normative References

The following documents are referenced by normative sections of the specification:

2.2.4.1. External Specifications

2.3. Motivation and Design Goals (Informative)

glTF 2.0 assets are widely used in various industries, including automotive, e-commerce, and gaming. There is a growing demand for adding logic and behavior to glTF assets, particularly in the metaverse. This extension aims to fulfill this demand by providing a portable, easily implementable, safe, and visually accessible solution for adding behavior to glTF assets. The extension is inspired by visual scripting features of leading game engines and aims to deliver a minimum meaningful and extensible feature set.

2.3.1. What Is a Behavior Graph?

A behavior graph is a series of interconnected nodes that represent behaviors and interactions in a 3D asset. It can respond to events and cause changes in the asset’s appearance and behavior.

2.3.2. What Problems Can They Solve?

Behavior graphs offer a flexible and multi-functional approach to encoding behavior, making them useful for various applications. For instance, they can be used to create smart assets with behavior and interactions, AR experiences with user interactions, and immersive game levels with dynamic assets and objectives.

2.3.3. What Do They Not Solve?

Behavior graphs are not designed to handle UI presentation or arbitrary scripting. Creating a 3D UI using behavior graphs would be complex, not portable, and not accessible. Similarly, arbitrary scripting is challenging to make safe, portable across platforms, and has a vast surface area.

2.3.4. Comparison with Trigger-Action Lists

Behavior graphs and trigger-action lists are the two common models for representing and executing behaviors in the digital world. Common 3D experience commerce tools use trigger-action lists, while behavior graphs are typically used by high-end game engines. In this section, we will explore the differences and similarities between these two models, and explain why glTF chose to adopt behavior graphs.

Behavior graphs and trigger-action lists share common features, such as being safe and sandboxed, offering limited execution models controlled by the viewer, and both supporting the “trigger” and “action” node categories. However, there are also significant differences between the two models. Trigger-action lists lack “Queries”, “Logic”, and “Control Flow” nodes, meaning that sophisticated behavior based on queries, logic, or control flow branches is not possible. This lack of functionality greatly affects the ability to create complex behavior and control structures and rules out the implementation of advanced control flow structures in the future.

On the other hand, behavior graphs are a superset of trigger-action lists, meaning that the former can support everything that trigger-action lists can, and more. Behavior graphs support “Queries”, “Logic” and “Control Flow” nodes, making them more expressive and capable of creating more sophisticated behaviors. This makes behavior graphs the preferred method of choice for high-end game engines, as it offers an identical safety model as trigger-action lists while being more expressive.

2.3.5. Turing Completeness

The execution model and node choices for this extension mean that it is Turing-complete. This means that an implementation of this can execute any computation and it is also hard to predict if it will run forever (e.g. halt or not.)

While this may present security implications, it is not a major hindrance and can be safely mitigated so that any implementation does not become susceptible to denial of services by badly behaving behavior graphs, whether intention or not.

The main way to mitigate the risk of non-halting behavior graphs is to limit the amount of time given to them for execution, both in terms of individual time slice as well as overall execution time.

2.3.6. Implementation Limitations

There will be limitations in engines and devices to such as:

  • Number of nodes in the graph

  • Number of variables

  • Number of custom events

  • Number of concurrent pending events/async nodes

  • Number of nodes executed per time slice

  • Speed of graph execution

These limitations are not defined in this specification.

3. Concepts

3.1. Graphs

A behavior graph is a JSON object containing nodes. It MAY also contain custom variables and custom events.

Behavior graphs are directed graphs with no directed cycles.

3.2. Nodes and Sockets

A node is a JSON object, which represents an executable item. Each node is defined by its type and a set of sockets. There are four kinds of sockets.

Output value sockets represent data initialized by the node or produced during its execution. For example, it could be results of math operations or parts of the node’s internal state. Accessing these sockets either triggers computing the return value on the fly by executing the node or returns a value based on the node’s internal state. Exact behavior depends on the node’s type.

Input value sockets represent data accessed during the node’s execution. For example, it could be arguments of math operations or execution parameters such as iteration count for loop nodes or duration for time-related nodes. These sockets MUST either be given a static value in the node object or connected to an output value socket of a different node. The node MAY access its input value sockets multiple times during the execution. The runtime MUST guarantee that all input value sockets have defined values when the node execution starts.

Output flow sockets represent “function pointers” that the node will call to advance the graph execution. For example, bodies and branches of flow control nodes are output flow sockets that drive further execution when certain condition are fulfilled. Output flow sockets MAY be unconnected; in such a case graph execution proceeds as if such sockets are no-ops.

Input flow sockets represent “methods” that could be called on the node. For example, flow control nodes (such as loops and conditions) usually have an in input flow socket that starts node’s execution. Additional operations MAY also be defined such as reset for nodes having an internal state.

Nodes MAY be configurable through static properties collectively called “node’s configuration” that MAY affect the node’s behavior and the number of its sockets, such as the number of cases for a switch-case control flow node.

Input and output value sockets have associated data types, e.g., floats, integers, booleans, etc.

Node’s sockets and configurations are defined by its type. Node types follow domain/operation naming pattern.

A node is executed when its input flow socket is reached by or when one of its output value sockets is requested by another node. Usually, the node executes its dependencies (if any), its own logic, and any number (including zero) of outgoing flow sockets.

3.3. Types

All value sockets and configurations are strictly typed.

Implementations of this extension MUST support the following type signatures.

3.3.1. Value Types

The following types are supported for value sockets.

bool

a boolean value

float

a double precision IEEE-754 floating-point scalar value

float2

a two-component vector of float values

float3

a three-component vector of float values

float4

a four-component vector of float values

float4x4

a 4x4 matrix of float values

int

a two’s complement 32-bit signed integer scalar value

3.3.2. Configuration Types

Node configurations MAY use all of the value types and these two additional types.

int[]

an array of int values

string

a UTF-8 string value

3.4. Custom Events

A behavior graph MAY define custom events for interacting with external execution environments and/or creating asynchronous loops. Although semantics of custom events is application-specific, their declarations MUST include value socket type information to ensure graph’s type safety.

3.5. Custom Variables

A behavior graph MAY define custom variables. A variable MAY be declared simultaneously with its initial value, otherwise the variable MUST be initialized to the type-specific default.

Custom variables MUST retain their values until the graph execution is terminated.

3.6. Node Types

3.6.1. Math Nodes

In this section, floatN is a placeholder for any of float, float2, float3, float4, or float4x4 types. All value sockets of floatN types have the same type within a node.

3.6.1.1. Constants
3.6.1.1.1. E

Type

math/e

Euler’s number

Output value sockets

float value

2.718281828459045

3.6.1.1.2. Pi

Type

math/pi

Ratio of a circle’s circumference to its diameter

Output value sockets

float value

3.141592653589793

3.6.1.1.3. Infinity

Type

math/inf

Positive infinity

Output value sockets

float value

Infinity

3.6.1.1.4. Not a Number

Type

math/nan

Not a Number

Output value sockets

float value

NaN

3.6.1.2. Arithmetic Nodes

These all operate component-wise. The description is per component.

If any input value is NaN, the output value is also NaN.

3.6.1.2.1. Absolute Value

Type

math/abs

Absolute value operation

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\(\begin{cases} -a & \text{if } a \lt 0 \\ 0 & \text{if } a = 0 \\ a & \text{if } a \gt 0 \end{cases}\)

3.6.1.2.2. Sign

Type

math/sign

Sign operation

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\(\begin{cases} -1 & \text{if } a \lt 0 \\ a & \text{if } a = \pm0 \\ +1 & \text{if } a \gt 0 \end{cases}\)

3.6.1.2.3. Truncate

Type

math/trunc

Truncate operation

Input value sockets

floatN a

Argument

Output value sockets

floatN value

Integer value equal to the nearest integer to \$a\$ whose absolute value is not larger than the absolute value of \$a\$

If the argument is infinity, it is returned unchanged.

3.6.1.2.4. Floor

Type

math/floor

Floor operation

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\$floor(a)\$, value equal to the nearest integer that is less than or equal to \$a\$

If the argument is infinity, it is returned unchanged.

3.6.1.2.5. Ceil

Type

math/ceil

Ceil operation

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\$ceil(a)\$, value equal to the nearest integer that is greater than or equal to \$a\$

If the argument is infinity, it is returned unchanged.

3.6.1.2.6. Fraction

Type

math/fract

Fractional operation

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\$a - floor(a)\$

3.6.1.2.7. Negation

Type

math/neg

Negation operation

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\$-a\$

3.6.1.2.8. Addition

Type

math/add

Addition operation

Input value sockets

floatN a

First addend

floatN b

Second addend

Output value sockets

floatN value

Sum, \$a + b\$

3.6.1.2.9. Subtraction

Type

math/sub

Subtraction operation

Input value sockets

floatN a

Minuend

floatN b

Subtrahend

Output value sockets

floatN value

Difference, \$a - b\$

3.6.1.2.10. Multiplication

Type

math/mul

Multiplication operation

Input value sockets

floatN a

First factor

floatN b

Second factor

Output value sockets

floatN value

Product, \$a * b\$

Note
Note

For float4x4 arguments, this operation performs per-component multiplication.

3.6.1.2.11. Division

Type

math/div

Division operation

Input value sockets

floatN a

Dividend

floatN b

Divisor

Output value sockets

floatN value

Quotient, \$a / b\$

3.6.1.2.12. Remainder

Type

math/rem

Remainder operation

Input value sockets

floatN a

Dividend

floatN b

Divisor

Output value sockets

floatN value

\(\begin{cases} \mathit{NaN} & \text{if } a = \pm \infty \text{ or } b = \pm 0 \\ a & \text{if } a \ne \pm \infty \text{ and } b = \pm \infty \\ a - (b \cdot trunc(\frac{a}{b})) & \text{otherwise} \end{cases}\)

3.6.1.2.13. Minimum

Type

math/min

Minimum operation

Input value sockets

floatN a

First argument

floatN b

Second argument

Output value sockets

floatN value

Smallest of the arguments

For the purposes of this node, negative zero is less than positive zero.

Note
Note

This is implementable in ECMAScript via the following expression:

Math.min(a, b)
3.6.1.2.14. Maximum

Type

math/max

Maximum operation

Input value sockets

floatN a

First argument

floatN b

Second argument

Output value sockets

floatN value

Largest of the arguments

For the purposes of this node, negative zero is less than positive zero.

Note
Note

This is implementable in ECMAScript via the following expression:

Math.max(a, b)
3.6.1.2.15. Clamp

Type

math/clamp

Clamp operation

Input value sockets

floatN a

Value to clamp

floatN b

First boundary

floatN c

Second boundary

Output value sockets

floatN value

\(min(max(a, min(b, c)), max(b, c))\)

This node relies on math/min and math/max nodes defined above.

Note
Note

This operation correctly handles a case when \$b\$ is greater than \$c\$.

3.6.1.2.16. Saturate

Type

math/saturate

Saturate operation

Input value sockets

floatN a

Value to saturate

Output value sockets

floatN value

\(min(max(a, 0), 1)\)

3.6.1.2.17. Interpolate

Type

math/mix

Linear interpolation operation

Input value sockets

floatN a

Interpolated value at \$0\$

floatN b

Interpolated value at \$1\$

floatN c

Unclamped interpolation coefficient

Output value sockets

floatN value

\$(1 - c) * a + c * b\$

3.6.1.3. Comparison Nodes

If any input value is NaN, the output value is false.

3.6.1.3.1. Equality

Type

math/eq

Equality operation

Input value sockets

floatN a

First argument

floatN b

Second argument

Output value sockets

bool value

True if the input arguments are equal, per-component; false otherwise

3.6.1.3.2. Less Than

Type

math/lt

Less than operation

Input value sockets

float a

First argument

float b

Second argument

Output value sockets

bool value

True if \$a < b\$; false otherwise

3.6.1.3.3. Less Than Or Equal To

Type

math/le

Less than or equal to operation

Input value sockets

float a

First argument

float b

Second argument

Output value sockets

bool value

True if \$a <= b\$; false otherwise

3.6.1.3.4. Greater Than

Type

math/gt

Greater than operation

Input value sockets

float a

First argument

float b

Second argument

Output value sockets

bool value

True if \$a > b\$; false otherwise

3.6.1.3.5. Greater Than Or Equal To

Type

math/ge

Greater than or equal operation

Input value sockets

float a

First argument

float b

Second argument

Output value sockets

bool value

True if \$a >= b\$; false otherwise

3.6.1.4. Special Nodes
3.6.1.4.1. Is Not a Number

Type

math/isnan

Not a Number check operation

Input value sockets

float a

Argument

Output value sockets

bool value

True if \$a\$ is NaN; false otherwise

3.6.1.4.2. Is Infinity

Type

math/isinf

Infinity check operation

Input value sockets

float a

Argument

Output value sockets

bool value

True if \$a\$ is positive or negative infinity; false otherwise

3.6.1.4.3. Select

Type

math/select

Conditional selection operation

Input value sockets

bool condition

Value selecting the value returned

T a

Positive selection option

T b

Negative selection option

Output value sockets

T value

\$a\$ if the the condition input value is true; \$b\$ otherwise

The type T represents any type. It MUST be the same for the output value socket and the input value sockets \$a\$ and \$b\$, otherwise the node is invalid.

3.6.1.5. Angle and Trigonometry Nodes

Node parameters specified as angle are assumed to be in units of radians.

These all operate component-wise. The description is per component.

If any input value is NaN, the output value is also NaN.

3.6.1.5.1. Degrees-To-Radians

Type

math/rad

Converts degrees to radians

Input value sockets

floatN a

Value in degrees

Output value sockets

floatN value

\$a * pi / 180\$

3.6.1.5.2. Radians-To-Degrees

Type

math/deg

Converts radians to degrees

Input value sockets

floatN a

Value in radians

Output value sockets

floatN value

\$a * 180 / pi\$

3.6.1.5.3. Sine

Type

math/sin

Sine function

Input value sockets

floatN a

Angle

Output value sockets

floatN value

\(\begin{cases} sin(a) & \text{if } a \ne \pm\infty \\ \mathit{NaN} & \text{if } a = \pm\infty \end{cases}\)

3.6.1.5.4. Cosine

Type

math/cos

Cosine function

Input value sockets

floatN a

Angle

Output value sockets

floatN value

\(\begin{cases} cos(a) & \text{if } a \ne \pm\infty \\ \mathit{NaN} & \text{if } a = \pm\infty \end{cases}\)

3.6.1.5.5. Tangent

Type

math/tan

Tangent function

Input value sockets

floatN a

Angle

Output value sockets

floatN value

\(\begin{cases} tan(a) & \text{if } a \ne \pm\infty \\ \mathit{NaN} & \text{if } a = \pm\infty \end{cases}\)

Note
Note

Since \$a\$ cannot exactly represent \(\pm\frac{\pi}{2}\), this function does not return infinity. The closest representable argument values would likely produce \(\pm16331239353195370\).

3.6.1.5.6. Arcsine

Type

math/asin

Arcsine function

Input value sockets

floatN a

Sine value

Output value sockets

floatN value

\(\begin{cases} arcsin(a) \in [-\frac{\pi}{2}; \frac{\pi}{2}] & \text{if } |a| \le 1 \\ \mathit{NaN} & \text{if } |a| \gt 1 \end{cases}\)

3.6.1.5.7. Arccosine

Type

math/acos

Arccosine function

Input value sockets

floatN a

Cosine value

Output value sockets

floatN value

\(\begin{cases} arccos(a) \in [0; \pi] & \text{if } |a| \le 1 \\ \mathit{NaN} & \text{if } |a| \gt 1 \end{cases}\)

3.6.1.5.8. Arctangent

Type

math/atan

Arctangent function

Input value sockets

floatN a

Tangent value

Output value sockets

floatN value

\(arctan(a) \in [-\frac{\pi}{2}; \frac{\pi}{2}]\)

3.6.1.5.9. Arctangent 2

Type

math/atan2

Arctangent 2 function

Input value sockets

floatN a

Y coordinate

floatN b

X coordinate

Output value sockets

floatN value

Angle between the positive X-axis and the vector from the \$(0, 0)\$ origin to the \$(X, Y)\$ point on a 2D plane

Zero and infinity argument values are handled according to ECMA-262 or IEEE 754 standards.

3.6.1.6. Hyperbolic Nodes

These all operate component-wise. The description is per component.

If any input value is NaN, the output value is also NaN.

3.6.1.6.1. Hyperbolic Sine

Type

math/sinh

Hyperbolic sine function

Input value sockets

floatN a

Hyperbolic angle value

Output value sockets

floatN value

\(\dfrac{e^a-e^{-a}}{2}\)

3.6.1.6.2. Hyperbolic Cosine

Type

math/cosh

Hyperbolic cosine function

Input value sockets

floatN a

Hyperbolic angle value

Output value sockets

floatN value

\(\dfrac{e^a+e^{-a}}{2}\)

3.6.1.6.3. Hyperbolic Tangent

Type

math/tanh

Hyperbolic tangent function

Input value sockets

floatN a

Hyperbolic angle value

Output value sockets

floatN value

\(\dfrac{e^a-e^{-a}}{e^a+e^{-a}}\)

3.6.1.6.4. Inverse Hyperbolic Sine

Type

math/asinh

Inverse hyperbolic sine function

Input value sockets

floatN a

Hyperbolic sine value

Output value sockets

floatN value

\(ln(a+\sqrt{a^2+1})\)

3.6.1.6.5. Inverse Hyperbolic Cosine

Type

math/acosh

Inverse hyperbolic cosine function

Input value sockets

floatN a

Hyperbolic cosine value

Output value sockets

floatN value

\(\begin{cases} ln(a+\sqrt{a^2-1}) & \text{if } a \ge 1 \\ \mathit{NaN} & \text{if } a \lt 1 \end{cases}\)

3.6.1.6.6. Inverse Hyperbolic Tangent

Type

math/atanh

Inverse hyperbolic tangent function

Input value sockets

floatN a

Hyperbolic tangent value

Output value sockets

floatN value

\(\begin{cases} \dfrac{1}{2}ln\dfrac{1+a}{1-a} & \text{if } |a| \lt 1 \\ \pm\infty & \text{if } a = \pm1 \\ \mathit{NaN} & \text{if } |a| \gt 1 \end{cases}\)

3.6.1.7. Exponential Nodes

These all operate component-wise. The description is per component.

If any input value is NaN, the output value is also NaN.

3.6.1.7.1. Exponent

Type

math/exp

Exponent function

Input value sockets

floatN a

Power value

Output value sockets

floatN value

\$e^a\$

3.6.1.7.2. Natural Logarithm

Type

math/log

Natural logarithm function

Input value sockets

floatN a

Argument value

Output value sockets

floatN value

\(\begin{cases} ln(a) & \text{if } a \gt 0 \\ -\infty & \text{if } a = 0 \\ \mathit{NaN} & \text{if } a \lt 0 \end{cases}\)

3.6.1.7.3. Base-2 Logarithm

Type

math/log2

Base-2 logarithm function

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\(\begin{cases} log_2(a) & \text{if } a \gt 0 \\ -\infty & \text{if } a = 0 \\ \mathit{NaN} & \text{if } a \lt 0 \end{cases}\)

3.6.1.7.4. Base-10 Logarithm

Type

math/log10

Base-10 logarithm function

Input value sockets

floatN a

Argument

Output value sockets

floatN value

\(\begin{cases} log_{10}(a) & \text{if } a \gt 0 \\ -\infty & \text{if } a = 0 \\ \mathit{NaN} & \text{if } a \lt 0 \end{cases}\)

3.6.1.7.5. Square Root

Type

math/sqrt

Square root function

Input value sockets

floatN a

Radicand

Output value sockets

floatN value

\(\begin{cases} \sqrt{a} & \text{if } a \ge 0 \\ \mathit{NaN} & \text{if } a \lt 0 \end{cases}\)

3.6.1.7.6. Cube Root

Type

math/cbrt

Cube root function

Input value sockets

floatN a

Radicand

Output value sockets

floatN value

\(\sqrt[3]{a}\)

3.6.1.7.7. Power

Type

math/pow

Power function

Input value sockets

floatN a

Base

floatN b

Exponent

Output value sockets

floatN value

\$a^b\$

Zero and infinity argument values are handled according to the ECMA-262 standard.

3.6.1.8. Vector Nodes

If any input value is NaN, the output value is also NaN.

3.6.1.8.1. Length

Type

math/length

Vector length

Input value sockets

float{2|3|4} a

Vector

Output value sockets

float value

Length of \$a\$, e.g., \$sqrt(a_x^2 + a_y^2)\$ for float2

3.6.1.8.2. Normalize

Type

math/normalize

Vector normalization

Input value sockets

float{2|3|4} a

Vector

Output value sockets

floatN value

Vector in the same direction as \$a\$ but with a unit length, e.g., \$a/sqrt(a_x^2 + a_y^2)\$ for float2

3.6.1.8.3. Dot Product

Type

math/dot

Dot product

Input value sockets

float{2|3|4} a

First vector

float{2|3|4} b

Second vector of the same type as \$a\$

Output value sockets

float value

Sum of per-component products of \$a\$ and \$b\$, e.g., \$a_x * b_x + a_y * b_y\$ for float2

3.6.1.8.4. Cross Product

Type

math/cross

Cross product

Input value sockets

float3 a

Vector

float3 b

Vector

Output value sockets

float3 value

Cross product of \$a\$ and \$b\$, i.e., \$(a_y * b_z - a_z * b_y, a_z * b_x - a_x * b_z, a_x * b_y - a_y * b_x)\$

3.6.1.8.5. Rotate 2D

Type

math/rotate2d

2D rotation

Input value sockets

float2 a

Vector to rotate

float b

Angle in radians

Output value sockets

float2 value

Vector \$a\$ rotated counter-clockwise by \$b\$

3.6.1.8.6. Rotate 3D

Type

math/rotate3d

3D rotation

Input value sockets

float3 a

Vector to rotate

float3 b

Vector representing an axis to rotate around

float c

Angle in radians

Output value sockets

float3 value

Vector \$a\$ rotated around vector \$b\$ counter-clockwise by \$c\$

If the vector \$b\$ is not unit, rotation results may be undefined.

3.6.1.8.7. Transform

Type

math/transform

Vector transformation

Input value sockets

float4 a

Vector to transform

float4x4 b

Transformation matrix

Output value sockets

float4 value

Transformed vector

3.6.1.8.8. Combine

Type

math/combine2

Combine two floats into a two-component vector

Input value sockets

float a

First component

float b

Second component

Output value sockets

float2 value

Vector

Type

math/combine3

Combine three floats into a three-component vector

Input value sockets

float a

First component

float b

Second component

float c

Third component

Output value sockets

float3 value

Vector

Type

math/combine4

Combine four floats into a four-component vector

Input value sockets

float a

First component

float b

Second component

float c

Third component

float d

Fourth component

Output value sockets

float4 value

Vector

3.6.1.8.9. Extract

Type

math/extract2

Extract two floats from a two-component vector

Input value sockets

float2 a

Vector

Output value sockets

float 0

First component

float 1

Second component

Type

math/extract3

Extract three floats from a three-component vector

Input value sockets

float3 a

Vector

Output value sockets

float 0

First component

float 1

Second component

float 2

Third component

Type

math/extract4

Extract four floats from a four-component vector

Input value sockets

float4 a

Vector

Output value sockets

float 0

First component

float 1

Second component

float 2

Third component

float 3

Fourth component

3.6.1.9. Matrix Nodes
3.6.1.9.1. Transpose

Type

math/transpose

Transpose operation

Input value sockets

float4x4 a

Matrix to transpose

Output value sockets

float4x4 value

Matrix that is the transpose of \$a\$

3.6.1.9.2. Determinant

Type

math/determinant

Dot product

Input value sockets

float4x4 a

Matrix

Output value sockets

float value

Determinant of \$a\$

3.6.1.9.3. Inverse

Type

math/inverse

Inverse operation

Input value sockets

float4x4 a

Matrix to inverse

Output value sockets

float4x4 value

Matrix that is the inverse of \$a\$

3.6.1.9.4. Multiplication

Type

math/matmul

Matrix multiplication operation

Input value sockets

float4x4 a

First matrix

float4x4 b

Second matrix

Output value sockets

float4x4 value

Matrix product

3.6.1.9.5. Combine

Type

math/combine4x4

Combine 16 floats into a 4x4 matrix

Input value sockets

float a

First row, first column element

float b

Second row, first column element

float c

Third row, first column element

float d

Fourth row, first column element

float e

First row, second column element

float f

Second row, second column element

float g

Third row, second column element

float h

Fourth row, second column element

float i

First row, third column element

float j

Second row, third column element

float k

Third row, third column element

float l

Fourth row, third column element

float m

First row, fourth column element

float n

Second row, fourth column element

float o

Third row, fourth column element

float p

Fourth row, fourth column element

Output value sockets

float4x4 value

Matrix

3.6.1.9.6. Extract

Type

math/extract4x4

Extract 16 floats from a 4x4 matrix

Input value sockets

float4x4 a

Matrix

Output value sockets

float 0

First row, first column element

float 1

Second row, first column element

float 2

Third row, first column element

float 3

Fourth row, first column element

float 4

First row, second column element

float 5

Second row, second column element

float 6

Third row, second column element

float 7

Fourth row, second column element

float 8

First row, third column element

float 9

Second row, third column element

float 10

Third row, third column element

float 11

Fourth row, third column element

float 12

First row, fourth column element

float 13

Second row, fourth column element

float 14

Third row, fourth column element

float 15

Fourth row, fourth column element

3.6.1.10. Integer Arithmetic Nodes

All inputs to these nodes are two’s complement 32-bit signed integers.

3.6.1.10.1. Absolute Value

Type

math/abs

Absolute value operation

Input value sockets

int a

Argument

Output value sockets

int value

\(\begin{cases} -a & \text{if } a \lt 0 \\ a & \text{if } a \ge 0 \end{cases}\)

As this node is defined in terms of the negation node (see below), the absolute value of -2147483648 is -2147483648.

Note
Note

This is implementable in ECMAScript via the following expression:

Math.abs(a)|0
3.6.1.10.2. Sign

Type

math/sign

Sign operation

Input value sockets

int a

Argument

Output value sockets

int value

\(\begin{cases} -1 & \text{if } a \lt 0 \\ 0 & \text{if } a = 0 \\ +1 & \text{if } a \gt 0 \end{cases}\)

3.6.1.10.3. Negation

Type

math/neg

Negation operation

Input value sockets

int a

Argument

Output value sockets

int value

\$-a\$

Negating -2147483648 MUST return -2147483648.

Note
Note

This is implementable in ECMAScript via the following expression:

(-a)|0
3.6.1.10.4. Addition

Type

math/add

Addition operation

Input value sockets

int a

First addend

int b

Second addend

Output value sockets

int value

Sum, \$a + b\$

Arithmetic overflow MUST wrap around, for example:

2147483647 + 1 == -2147483648
Note
Note

This is implementable in ECMAScript via the following expression:

(a + b)|0
3.6.1.10.5. Subtraction

Type

math/sub

Subtraction operation

Input value sockets

int a

Minuend

int b

Subtrahend

Output value sockets

int value

Difference, \$a - b\$

Arithmetic overflow MUST wrap around, for example:

-2147483648 - 1 == 2147483647
Note
Note

This is implementable in ECMAScript via the following expression:

(a - b)|0
3.6.1.10.6. Multiplication

Type

math/mul

Multiplication operation

Input value sockets

int a

First factor

int b

Second factor

Output value sockets

int value

Product, \$a * b\$

Arithmetic overflow MUST wrap around, for example:

 2147483647 * 2147483647 == 1

-2147483648 * (-1)       == -2147483648
Note
Note

This is implementable in ECMAScript via the following expression:

Math.imul(a, b)
3.6.1.10.7. Division

Type

math/div

Division operation

Input value sockets

int a

Dividend

int b

Divisor

Output value sockets

int value

\(\begin{cases} \frac{a}{b} & \text{if } b \ne 0 \\ 0 & \text{if } b = 0 \end{cases}\)

The quotient MUST be truncated towards zero.

Arithmetic overflow is defined as follows:

-2147483648 / (-1) == -2147483648
Note
Note

This is implementable in ECMAScript via the following expression:

(a / b)|0
3.6.1.10.8. Remainder

Type

math/rem

Remainder operation

Input value sockets

int a

Dividend

int b

Divisor

Output value sockets

int value

\(\begin{cases} a - (b \cdot trunc(\frac{a}{b})) & \text{if } b \ne 0 \\ 0 & \text{if } b = 0 \end{cases}\)

Note
Note

This is implementable in ECMAScript via the following expression:

(a % b)|0
3.6.1.10.9. Minimum

Type

math/min

Minimum operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

int value

Smallest of the arguments

3.6.1.10.10. Maximum

Type

math/max

Maximum operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

int value

Largest of the arguments

3.6.1.10.11. Clamp

Type

math/clamp

Clamp operation

Input value sockets

int a

Value to clamp

int b

First boundary

int c

Second boundary

Output value sockets

int value

\(min(max(a, min(b, c)), max(b, c))\)

Note
Note

This operation correctly handles a case when \$b\$ is greater than \$c\$.

3.6.1.11. Integer Comparison Nodes

All inputs to these nodes are two’s complement 32-bit signed integers.

3.6.1.11.1. Equality

Type

math/eq

Equality operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

bool value

True if the input arguments are equal; false otherwise

3.6.1.11.2. Less Than

Type

math/lt

Less than operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

bool value

True if \$a < b\$; false otherwise

3.6.1.11.3. Less Than Or Equal To

Type

math/le

Less than or equal to operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

bool value

True if \$a <= b\$; false otherwise

3.6.1.11.4. Greater Than

Type

math/gt

Greater than operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

bool value

True if \$a > b\$; false otherwise

3.6.1.11.5. Greater Than Or Equal To

Type

math/ge

Greater than or equal operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

bool value

True if \$a >= b\$; false otherwise

3.6.1.12. Integer Bitwise Nodes

All inputs to these nodes are two’s complement 32-bit signed integers.

3.6.1.12.1. Bitwise NOT

Type

math/not

Bitwise NOT operation

Input value sockets

int a

Argument

Output value sockets

int value

~a

3.6.1.12.2. Bitwise AND

Type

math/and

Bitwise AND operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

int value

a & b

3.6.1.12.3. Bitwise OR

Type

math/or

Bitwise OR operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

int value

a | b

3.6.1.12.4. Bitwise XOR

Type

math/xor

Bitwise XOR operation

Input value sockets

int a

First argument

int b

Second argument

Output value sockets

int value

a ^ b

3.6.1.12.5. Right Shift

Type

math/asr

Right Shift

Input value sockets

int a

Value to be shifted

int b

Number of bits to shift by

Output value sockets

int value

a >> b

Only the lowest 5 bits of \$b\$ are considered, i.e., its effective range is [0, 31]. The result MUST be truncated to 32 bits and interpreted as a two’s complement signed integer. The most significant bit of \$a\$ MUST be propagated.

3.6.1.12.6. Left Shift

Type

math/lsl

Left Shift

Input value sockets

int a

Value to be shifted

int b

Number of bits to shift by

Output value sockets

int value

a << b

Only the lowest 5 bits of \$b\$ are considered, i.e., its effective range is [0, 31]. The result MUST be truncated to 32 bits and interpreted as a two’s complement signed integer.

3.6.1.12.7. Count Leading Zeros

Type

math/clz

Count leading zeros operation

Input value sockets

int a

Argument

Output value sockets

int value

Number of leading zero bits in \$a\$

If \$a\$ is 0, the operation returns 32; if \$a\$ is negative, the operation returns 0.

Note
Note

This is implementable in ECMAScript via the following expression:

Math.clz32(a)
3.6.1.12.8. Count Trailing Zeros

Type

math/ctz

Count trailing zeros operation

Input value sockets

int a

Argument

Output value sockets

int value

Number of trailing zero bits in \$a\$

If \$a\$ is 0, the operation returns 32.

Note
Note

This is implementable in ECMAScript via the following expression:

a ? (31 - Math.clz32(a & -a)) : 32;
3.6.1.12.9. Count One Bits

Type

math/popcnt

Count set bits operation

Input value sockets

int a

Argument

Output value sockets

int value

Number of set bits in \$a\$

If \$a\$ is 0, the operation returns 0; if \$a\$ is -1, the operation returns 32.

3.6.1.13. Boolean Arithmetic Nodes
3.6.1.13.1. Equality

Type

math/eq

Equality operation

Input value sockets

bool a

First argument

bool b

Second argument

Output value sockets

bool value

True if and only if both \$a\$ and \$b\$ have the same value; false otherwise

3.6.1.13.2. Boolean NOT

Type

math/not

Boolean NOT operation

Input value sockets

bool a

Argument

Output value sockets

bool value

True if \$a\$ is false; false if \$a\$ is true

3.6.1.13.3. Boolean AND

Type

math/and

Boolean AND operation

Input value sockets

bool a

First argument

bool b

Second argument

Output value sockets

bool value

True if and only if both \$a\$ and \$b\$ are true; false otherwise

3.6.1.13.4. Boolean OR

Type

math/or

Boolean OR operation

Input value sockets

bool a

First argument

bool b

Second argument

Output value sockets

bool value

False if and only if both \$a\$ and \$b\$ are false; true otherwise

3.6.1.13.5. Boolean XOR

Type

math/xor

Boolean XOR operation

Input value sockets

bool a

First argument

bool b

Second argument

Output value sockets

bool value

True if and only if \$a\$ is not equal to \$b\$; false otherwise

3.6.2. Type Conversion Nodes

3.6.2.1. Boolean Conversion Nodes
3.6.2.1.1. Boolean to Integer

Type

type/boolToInt

Boolean to integer conversion

Input value sockets

bool a

Argument

Output value sockets

int value

\$1\$ if \$a\$ is true; \$0\$ otherwise

Note
Note

This is implementable in ECMAScript via the following expression:

a|0
3.6.2.1.2. Boolean to Float

Type

type/boolToFloat

Boolean to float conversion

Input value sockets

bool a

Argument

Output value sockets

float value

\$1\$ if \$a\$ is true; \$0\$ otherwise

Note
Note

This is implementable in ECMAScript via the following expression:

+a
3.6.2.2. Integer Conversion Nodes
3.6.2.2.1. Integer to Boolean

Type

type/intToBool

Integer to boolean conversion

Input value sockets

int a

Argument

Output value sockets

bool value

True if \$a\$ is not equal to zero; false otherwise

Note
Note

This is implementable in ECMAScript via the following expression:

!!a
3.6.2.2.2. Integer to Float

Type

type/intToFloat

Integer to float conversion

Input value sockets

int a

Argument

Output value sockets

float value

Floating-point value equal to \$a\$

Since floating-point values have double precision, this conversion MUST be lossless.

This operation MUST NOT produce negative zero.

Note
Note

This operation is no-op in ECMAScript.

3.6.2.3. Float Conversion Nodes
3.6.2.3.1. Float to Boolean

Type

type/floatToBool

Float to boolean conversion

Input value sockets

float a

Argument

Output value sockets

bool value

False if \$a\$ is NaN or equal to zero; true otherwise

Note
Note

This is implementable in ECMAScript via the following expression:

!!a
3.6.2.3.2. Float to Integer

Type

type/floatToInt

Float to integer conversion

Input value sockets

float a

Argument

Output value sockets

int value

Integer value produced as described below

  1. If the \$a\$ input value is zero, infinite, or NaN, return zero and skip the next steps.

  2. Let \$t\$ be \$a\$ with its fractional part removed by rounding towards zero.

  3. Let \$k\$ be a value of the same sign as \$t\$ such that its absolute value is less than \$2^32\$ and \$k\$ is equal to \$t - q * 2^32\$ for some integer \$q\$.

  4. If \$k\$ is greater than or equal to \$2^31\$, return \$k - 2^32\$; otherwise return \$k\$.

Note
Note

This is implementable in ECMAScript via the following expression:

a|0

3.6.3. Control Flow Nodes

3.6.3.1. Sync Nodes
3.6.3.1.1. Sequence

Type

flow/sequence

Sequentially activate all connected output flows

Input flow sockets

in

The entry flow into this node

Output flow sockets

<name>

One or more output flows; their names are purely informative

This node has no internal state.

When the in input flow is activated, all output flows are activated one by one in the order they are defined in JSON.

3.6.3.1.2. Branch

Type

flow/branch

Branch the execution flow based on a condition

Input flow sockets

in

The entry flow into this node

Input value sockets

bool condition

Value selecting the branch taken

Output flow sockets

true

The flow to be activated if the condition input value is true

false

The flow to be activated if the condition input value is false

This node has no internal state.

The condition input value is evaluated each time the node is executed.

3.6.3.1.3. Switch

Type

flow/switch

Conditionally route the execution flow to one of the outputs

Configuration

int[] cases

The cases on which to perform the switch on; values MUST be unique 32-bit signed integers; at least one value MUST be present

Input flow sockets

in

The entry flow into this node

Input value sockets

int selection

The value on which the switch operates

Output flow sockets

<case>

The output flow, case is an integer decimal number present in the cases configuration array

default

The output flow used when the selection input value is not present in the cases configuration array

The node has one or more <case> output flow sockets named as decimal integers equal to the elements of the cases configuration array. Encoded as JSON strings, these output flow socket names MUST contain only decimal integers (ASCII characters 0x30 …​ 0x39) and optionally a leading minus sign (ASCII character 0x2D); other characters and leading zeros are not allowed.

For example, if the cases configuration array is [-1, 0, 1], the output socket names are exactly "-1", "0", and "1".

This node has no internal state.

When the in input flow is activated:

  1. Evaluate the selection input value.

  2. If the cases configuration array does not contain the selection input value:

    1. activate the default output flow if it is connected.

  3. If the cases configuration array contains the selection input value:

    1. activate the output flow named <selection> if it is connected.

3.6.3.1.4. While Loop

Type

flow/while

Repeatedly activate the output flow based on a condition

Input flow sockets

in

The entry flow into this node

Input value sockets

bool condition

Loop condition

Output flow sockets

loopBody

The flow to be activated while the condition input value is true

completed

The flow to be activated once the condition input value is false

This node has no internal state.

When the in input flow is activated:

  1. Evaluate the condition. The condition MUST NOT statically evaluate to true, otherwise the node is invalid.

  2. If the condition is true,

    1. activate the loopBody output flow;

    2. after completion of the loopBody output flow, goto step 1.

  3. If the condition is false,

    1. activate the completed output flow.

3.6.3.1.5. For Loop

Type

flow/for

Repeatedly activate the output flow based on an incrementing index value

Configuration

int initialIndex

The index value before the loop starts

Input flow sockets

in

The entry flow into this node

Input value sockets

int startIndex

The start index of the loop

int endIndex

The end index of the loop

Output flow sockets

loopBody

The flow to be activated if the index value is less than the endIndex input value

completed

The flow to be activated if the index value is greater than or equal to the endIndex input value

Output value sockets

int index

The current index value if the node has ever been activated, initialIndex otherwise

The internal state of this node consists of one 32-bit signed integer value index initialized to initialIndex.

When the in input flow is activated:

  1. Evaluate the startIndex input value.

  2. Set index to startIndex.

  3. Evaluate the endIndex input value.

  4. If index is less than the endIndex input value,

    1. activate the loopBody output flow;

    2. after completion of the loopBody output flow, increment the index value by 1;

    3. goto step 3.

  5. If the index value is greater than or equal to the endIndex input value,

    1. activate the completed output flow.

3.6.3.1.6. Do N

Type

flow/doN

Activate the output flow no more than N times

Input flow sockets

in

The entry flow into this node

reset

When this flow is activated, the currentCount value is reset to 0

Input value sockets

int n

Maximum number of times the out output flow is activated

Output flow sockets

out

The flow to be activated if the currentCount value is less than the n input value

Output value sockets

int currentCount

The current execution count

The internal state of this node consists of one 32-bit signed integer value currentCount initialized to 0.

When the reset input flow is activated:

  1. Reset currentCount to 0.

When the in input flow is activated:

  1. Evaluate the n input value.

  2. If currentCount is less than n,

    1. increment currentCount by 1;

    2. activate the out output flow.

3.6.3.1.7. Multi Gate

Type

flow/multiGate

Route the execution flow to one of the outputs sequentially or randomly

Configuration

bool isRandom

If set to true, output flows are executed in random order, picking a random not used output flow each time until all are done

bool isLoop

If set to true, the outputs will repeat in a loop continuously after all are done

Input flow sockets

in

The entry flow into this node

reset

When this flow is activated, the lastIndex value is reset to -1 and all outputs are marked as not used

Output flow sockets

<name>

One or more output flows; their names are purely informative

Output value sockets

int lastIndex

The index of the last used output; -1 if the node has not been activated

The internal state of this node consists of one 32-bit signed integer value lastIndex initialized to -1 and an array of booleans with all values initialized to false representing used output flows. The size of the boolean array is equal to the number of output flows.

The output flows are ordered as they are defined in JSON.

When the reset input flow is activated:

  1. Reset the lastIndex value to -1.

  2. Mark all output flows as not used in the boolean array.

When the in input flow is activated:

  1. If the isRandom configuration value is false,

    1. let i be the smallest not used output flow index according to the boolean array or -1 if all output flows are marked as used.

  2. If the isRandom configuration value is true,

    1. let i be a random not used output flow index according to the boolean array or -1 if all output flows are marked as used.

  3. If i is greater than -1,

    1. mark the output flow with index i as used in the boolean array;

    2. set the lastIndex value to i;

    3. activate the output flow with index i.

  4. If i is equal to -1 and the isLoop configuration value is true,

    1. mark all output flows as not used in the boolean array;

    2. if the isRandom configuration value is false,

      1. set i to 0;

    3. if the isRandom configuration value is true,

      1. set i to a random output flow index;

    4. mark the output flow with index i as used in the boolean array;

    5. set the lastIndex value to i;

    6. activate the output flow with index i.

3.6.3.1.8. Wait All

Type

flow/waitAll

Activate the output flow when all input flows have been activated at least once.

Configuration

int inputFlows

The number of input flows, MUST be greater than 0 and less than or equal to 2147483647

Input flow sockets

<i>

The i-th input flow, i is a non-negative integer decimal number less than the inputFlows configuration value

reset

When this flow is activated, all input flows are marked as unused

Output flow sockets

out

The flow to be activated after every input flow activation except the last missing input

completed

The flow to be activated when the last missing input flow is activated

Output value sockets

int remainingInputs

The number of not yet activated input flows

The node has one or more input flow sockets named as sequential non-negative decimal integers depending on the inputFlows configuration value. Encoded as JSON strings, these input flow socket names contain only decimal integers (ASCII characters 0x30 …​ 0x39); other characters and leading zeros are not allowed.

For example, if inputFlows is 3, the input socket names are "0", "1", and "2" exactly.

The internal state of this node consists of one 32-bit signed integer value remainingInputs initialized to the number of connected input flows not including the reset input flow and a map of booleans with all values initialized to false representing used input flow sockets. The size of the boolean map is equal to the initial remainingInputs value.

When the reset input flow is activated:

  1. Reset remainingInputs to the number of connected input flows not including the reset input flow.

  2. Mark all connected input flows as not used in the boolean map.

When any of the <i> input flows is activated:

  1. If the <i>-th input flow is not marked as used in the boolean map:

    1. mark the <i>-th input flow as used in the boolean map.

    2. decrement the remainingInputs value by 1;

  2. If the remainingInputs value is zero:

    1. activate the completed output flow.

  3. If the remainingInputs value is not zero:

    1. activate the out output flow.

3.6.3.1.9. Throttle

Type

flow/throttle

Activate the output flow unless it has been activated less than a certain time ago

Input flow sockets

in

The entry flow into this node

reset

When this flow is activated, the output flow throttling state is reset

Input value sockets

float duration

The time, in seconds, to wait after an output flow activation before allowing subsequent output flow activations

Output flow sockets

out

The flow to be activated if the output flow is not currently throttled

err

The flow to be activated if the duration input value is negative, infinite, or NaN

Output value sockets

float lastRemainingTime

The remaining throttling time, in seconds, at the moment of the last valid activation of the input flow or NaN if the input flow has never been activated with a valid duration input value

The internal state of this node consists of an uninitialized timestamp value of an implementation-defined high-precision time type and a floating-point lastRemainingTime value initialized to NaN.

When the reset input flow is activated:

  1. Reset the lastRemainingTime value to NaN.

When the in input flow is activated:

  1. Evaluate the duration input value.

  2. If the duration input value is NaN, infinite, negative, or not convertible into an implementation-specific time type used for the internal timestamp value,

    1. activate the err output flow and skip the next steps.

  3. If the lastRemainingTime value is not NaN:

    1. Let elapsed be a non-negative difference, in seconds, between the timestamp and the current time.

    2. If the duration input value is less than or equal to the elapsed value,

      1. set the timestamp value to the current time;

      2. set the lastRemainingTime value to zero;

      3. activate the out output flow.

    3. If the duration input value is greater than the elapsed value,

      1. set the lastRemainingTime value to the positive difference, in seconds, between the duration and elapsed values.

  4. If the lastRemainingTime value is NaN,

    1. set the timestamp value to the current time;

    2. set the lastRemainingTime value to zero;

    3. activate the out output flow.

3.6.3.2. Delay Nodes
3.6.3.2.1. Set Delay

Type

flow/setDelay

Schedule the output flow activation after a certain delay

Input flow sockets

in

The entry flow into this node

cancel

When this flow is activated, all delayed activations scheduled by this node are cancelled

Input value sockets

float duration

The duration, in seconds, to delay the completed output flow activation

Output value sockets

int lastDelayIndex

The delay index assigned during the last successful node execution

Output flow sockets

out

The flow to be activated if the duration value is valid

err

The flow to be activated if the duration value is invalid

completed

The flow to be activated after the delay

The internal state of this node consists of an integer lastDelayIndex value initialized to -1 and a dynamic array of activation indices scheduled by the node. This array is initially empty and its maximum size is implementation-specific.

The internal state of an execution graph having one or more flow/setDelay nodes includes a dynamic array of activation indices scheduled from all such nodes. This array is initially empty and its maximum size is implementation-specific.

Implementations MUST be aware of their effective limit on the maximum supported duration input value to avoid any implicit behavior changes, e.g., due to numeric overflows; exceeding such value MUST lead to the err output flow activation as described below.

When the in input flow is activated:

  1. Evaluate the duration input value.

  2. If the duration input value is NaN, infinite, negative, or not convertible into an implementation-specific time type,

    1. activate the err output flow and skip the next steps.

  3. If scheduling a new activation exceeds any implementation-specific limit,

    1. activate the err output flow and skip the next steps.

  4. Let activationTime be an implementation-defined high-precision time value equal to the sum of the current time value and the duration input value converted to the same time type.

  5. If activationTime is not valid according to implementation-specific validation rules, e.g., it exceeds an internal threshold value,

    1. activate the err output flow and skip the next steps.

  6. Set lastDelayIndex to a positive value representing the delayed flow activation being scheduled. This value MUST be unique across all previous activations of all flow/setDelay nodes of the graph.

  7. Push the value of lastDelayIndex to the graph and node arrays of activation indices.

  8. Schedule the following actions at the activationTime time:

    1. Removal of the activation index value from both arrays of activation indices.

    2. Activation of the completed output flow.

  9. Activate the out output flow.

When the cancel input flow is activated:

  1. Set the lastDelayIndex value to -1.

  2. For each activation index value in the node’s array of activation indices:

    1. Remove this activation index value from the node’s and the graph’s arrays of activation indices.

    2. Cancel the corresponding scheduled activation.

3.6.3.2.2. Cancel Delay

Type

flow/cancelDelay

Cancel a previously scheduled output flow activation

Input flow sockets

in

The entry flow into this node

Input value sockets

int delayIndex

The index value of the scheduled activation to be cancelled

Output flow sockets

out

The flow to be activated after executing this node

This node has no internal state but its execution MAY affect internal states of other nodes and the graph.

When the in input flow is activated:

  1. Evaluate the delayIndex input value.

  2. Remove this activation index value from all arrays of activation indices if it exists.

  3. Cancel the corresponding scheduled activation if it exists.

  4. Activate the out output flow.

Non-positive or not existing delay index values MUST NOT cause any runtime errors.

3.6.4. State Manipulation Nodes

3.6.4.1. Custom Variable Access
3.6.4.1.1. Variable Get

Type

variable/get

Get a custom variable value

Configuration

int variable

The custom variable index

Output value sockets

T value

The custom variable value

This node gets a custom variable value using the variable index provided by the variable configuration value. The type T is determined by the referenced variable. The variable index MUST be non-negative and less than the total number of custom variables, otherwise the node is invalid.

This node has no internal state.

3.6.4.1.2. Variable Set

Type

variable/set

Set a custom variable value

Configuration

int variable

The custom variable index

Input flow sockets

in

The entry flow into this node

Input value sockets

T value

The new variable value

Output flow sockets

out

The flow to be activated after the value is set

This node sets a custom variable value using the variable index provided by the variable configuration value and the value input value. The type T is determined by the referenced variable. The variable index MUST be non-negative and less than the total number of custom variables, otherwise the node is invalid.

This node has no internal state.

When the in input flow is activated:

  1. Evaluate the value input value.

  2. Set the custom variable with the variable configuration value index to the value input value.

  3. Activate the out output flow.

3.6.4.2. Object Model Access
3.6.4.2.1. Pointer Get

Type

pointer/get

Get an object model property value

Configuration

string pointer

The JSON pointer or JSON pointer template

Input value sockets

int <segment>

The JSON pointer template path segment to be substituted at runtime

Output value sockets

T value

The resolved property value

bool isValid

True if the property value can be resolved, false otherwise

This node gets a glTF Asset Object Model value using the provided JSON pointer. The type T is determined by the pointer string according to the glTF Asset Object Model Specification. If the type of the pointer cannot be statically determined, the node is invalid. Pointers containing extras properties are out of scope of this specification but MAY be supported by implementations.

Note
Examples

The pointer configuration value "/nodes/0/scale" is valid; its output value socket is of float3 type.

The pointer configuration value "/myProperty" is invalid because this path is not defined.

The pointer string MAY be a template pointer string, i.e., it MAY contain path segments substituted at runtime using the input values. All input values MUST be of int type. Path segments, if used, MUST substitute only array indices in the pointer templates as listed in the glTF Asset Object Model Specification.

Note
Example

If the pointer configuration value is "/nodes/{myId}/scale", the node has the myId input value socket, which value denotes the node index.

This node has no internal state.

If the pointer or the pointer template with all its substitutions applied can be resolved, the value output value is the resolved property value and the isValid output value is true.

If the pointer or the pointer template with all its substitutions applied cannot be resolved, the value output value is the default value for its type and the isValid output value is false.

Note
Note

Unresolvable pointers include those with negative or out-of-bounds array indices and/or non-existent JSON objects.

3.6.4.2.2. Pointer Set

Type

pointer/set

Set an object model property value

Configuration

string pointer

The JSON pointer or JSON pointer template

Input flow sockets

in

The entry flow into this node

Input value sockets

int <segment>

The JSON pointer template path segment to be substituted at runtime

T value

The new property value

Output flow sockets

out

The flow to be activated if the JSON pointer can be resolved

err

The flow to be activated if the JSON pointer cannot be resolved

This node sets a glTF Asset Object Model value using the provided JSON pointer. The type T is determined by the pointer string according to the glTF Asset Object Model Specification. If the type of the pointer cannot be statically determined, the node is invalid. Pointers containing extras properties are out of scope of this specification but MAY be supported by implementations.

The pointer string MAY be a template pointer string, i.e., it MAY contain path segments substituted at runtime using the input values. All input values used for path segment substitutions MUST be of int type. Path segments, if used, MUST substitute only array indices in the pointer templates as listed in the glTF Asset Object Model Specification.

If the value input value is not valid for the resolved property, the effective property value becomes implementation-defined and subsequent pointer/get evaluations of the property MAY return any value of the corresponding type until the property is updated with a valid value.

This node has no internal state.

When the in input flow is activated:

  1. Evaluate all input values.

  2. If the pointer or the pointer template with all its substitutions applied can be resolved,

    1. if the pointer interpolation state dynamic array (defined below) contains an entry with the same resolved JSON Pointer value, remove it from the array;

    2. set the resolved property to the value input value;

    3. activate the out output flow.

  3. If the pointer or the pointer template with all its substitutions applied cannot be resolved,

    1. activate the err output flow.

3.6.4.2.3. Pointer Interpolate

Type

pointer/interpolate

Interpolate an object model property value

Configuration

string pointer

The JSON pointer or JSON pointer template

Input flow sockets

in

The entry flow into this node

Input value sockets

int <segment>

The JSON pointer template path segment to be substituted at runtime

T value

The target property value

float duration

The time, in seconds, in which the property SHOULD reach the target value

float2 p1

Control point P1

float2 p2

Control point P2

Output flow sockets

out

The flow to be activated if the JSON pointer can be resolved and the input values are valid

err

The flow to be activated if the JSON pointer cannot be resolved or the input values are invalid

done

The flow to be activated when the property reaches the target value

This node interpolates and updates the specified glTF Asset Object Model property multiple times over the specified duration using the provided JSON pointer. The type T is determined by the pointer string according to the glTF Asset Object Model Specification. If the type of the pointer cannot be statically determined or if it is integer or boolean, the node is invalid. Pointers containing extras properties are out of scope of this specification but MAY be supported by implementations.

The pointer string MAY be a template pointer string, i.e., it MAY contain path segments substituted at runtime using the input values. All input values used for path segment substitutions MUST be of int type. Path segments, if used, MUST substitute only array indices in the pointer templates as listed in the glTF Asset Object Model Specification.

If the value input value or any intermediate interpolated value are not valid for the resolved property, the effective property value becomes implementation-defined and subsequent pointer/get evaluations of the property MAY return any value of the corresponding type until the property is updated with a valid value.

This node has no internal state.

When a node of this type is used in the behavior graph, the global graph state includes an implementation-defined pointer interpolation state dynamic array each element of which contains the following data:

  • The resolved JSON Pointer to the Object Model property being interpolated

  • Implementation-defined high precision timestamp value representing the interpolation start time

  • Interpolation duration value converted to the implementation-defined high precision time type

  • Object Model property value at the time of the successful node activation

  • Information needed for cubic Bézier spline evaluation derived from the node’s input values

  • Target property value

  • Implementation-specific pointer to the done output flow of the node that has added this entry

This array is initially empty and its maximum size is implementation-specific.

When the in input flow is activated:

  1. Evaluate all input values.

  2. If the pointer or the pointer template with all its substitutions applied cannot be resolved,

    1. activate the err output flow and skip the next steps.

  3. If the duration input value is NaN, infinite, negative, or not convertible into an implementation-specific time type used for the internal interpolation start time value,

    1. activate the err output flow and skip the next steps.

  4. If any component of the p1 or p2 input values is NaN or infinite or if any of the first components of these input values is negative or greater than 1,

    1. activate the err output flow and skip the next steps.

  5. If starting a new pointer interpolation exceeds any implementation-specific limit,

    1. activate the err output flow and skip the next steps.

  6. If the pointer interpolation state dynamic array contains an entry with the same resolved JSON Pointer value,

    1. remove it from the array.

  7. Using the implicitly-defined end points \$P_0 (0, 0)\$ and \$P_3 (1, 1)\$ together with the control points \$P_1\$ and \$P_2\$ provided via the input values construct a cubic Bézier easing function for the \$[0, 1\$] input range.

  8. Add a new entry to the pointer interpolation state dynamic array filling it with the required information based on the evaluated input values.

  9. Activate the out output flow.

On each asset animation update, for each entry in the pointer interpolation state dynamic array:

  1. Compute the current input progress position t as the time passed since the interpolation start divided by the interpolation’s duration.

  2. If t is less than or equal to zero

    1. skip the next steps.

  3. If t is NaN or greater than or equal to 1,

    1. set the Object Model property to the target value;

    2. remove the current entry from the pointer interpolation state dynamic array;

    3. activate the done output flow linked to the current entry

    4. skip the next steps.

  4. Using the cubic Bézier spline information, compute the output progress position q based on the t value. This step implies that \(t \in [0; 1]\).

  5. Set the linked Object Model property to the new value computed as a linear interpolation between the original and the target property values using the output progress position q as the interpolation coefficient.

If the Object Model property is a quaternion, spherical linear interpolation expression SHOULD be used.

Intermediate output progress values MAY be less than zero or greater than one.

3.6.4.3. Animation Control Nodes
3.6.4.3.1. Animation Play

Type

animation/start

Start playing an animation

Input flow sockets

in

The entry flow into this node

Input value sockets

int animation

Animation index

float startTime

Start time

float endTime

End time

float speed

Speed multiplier

Output flow sockets

out

The flow to be activated if the input values are valid

err

The flow to be activated if any of the input values is invalid

completed

The flow to be activated after the animation ends

This node starts playing an animation using the specified input values.

For the purposes of the Animation Control Nodes the concept of glTF animations is extended to unambiguously map any requested input timestamp \$r\$ to the effective input timestamp \$t\$ present in the glTF animation data as follows.

  1. Let \$T\$ be the maximum value of all animation sampler input accessors of the animation. Then, the stored animation data defines the animated property values for all effective input timestamps in the \$[0, T\$] range.

  2. Let \$r\$ be a scalar value on a timeline infinite in both directions, from negative infinity to positive infinity.

  3. If \$T\$ is not equal to zero, let \$s\$ be the current iteration number computed as follows: \(s=\begin{cases} \left\lceil \dfrac{r-T}{T} \right\rceil & \text{if } r \gt 0 \\ \left\lfloor \dfrac{r}{T} \right\rfloor & \text{if } r \le 0 \\ \end{cases}\)

  4. Now for each requested input timestamp \$r\$, the corresponding effective input timestamp is \(t=\begin{cases} r - s * T & \text{if } T \ne 0 \\ 0 & \text{if } T=0 \\ \end{cases}\)

This node has no internal state.

When a node of this type is used in the behavior graph, the global graph state includes an implementation-defined animation state dynamic array each element of which contains the following data:

  • Animation index

  • Start time value

  • End time value

  • Stop time value (see animation/stopAt)

  • Speed value

  • Implementation-specific creation timestamp value associated with the system time when this entry was added

  • Implementation-specific end completion pointer to the completed output flow of the node that has added this entry

  • Implementation-specific stop completion pointer to the completed output flow of the node that has scheduled its stopping (see animation/stopAt)

This array is initially empty; its maximum size is implementation-specific.

When the in input flow is activated:

  1. Evaluate all input values.

  2. If the animation input value is negative or greater than or equal to the number of glTF animations in the asset,

    1. activate the err output flow and skip the next steps.

  3. If the startTime or endTime input values are NaN or if the startTime input value is infinite,

    1. activate the err output flow and skip the next steps.

  4. If the speed input value is NaN, infinite, or less than or equal to zero,

    1. activate the err output flow and skip the next steps.

  5. If starting a new animation exceeds any implementation-specific limit,

    1. activate the err output flow and skip the next steps.

  6. If the animation state dynamic array contains an entry with the same animation index,

    1. remove it from the array; the previously set completed flows MUST NOT be activated.

  7. Add a new entry to the animation state dynamic array filling it with the required information based on the evaluated input values. The stop time value MUST be set to the end time value and the stop completion pointer MUST be set to null.

  8. Activate the out output flow.

On each asset animation update, for each entry in the animation state dynamic array:

  1. TBD

3.6.4.3.2. Animation Stop

Type

animation/stop

Immediately stop a playing animation

Input flow sockets

in

The entry flow into this node

Input value sockets

int animation

Animation index

Output flow sockets

out

The flow to be activated if the animation index is valid

err

The flow to be activated if the animation index is invalid

This node stops a playing animation.

This node has no internal state.

When the in input flow is activated:

  1. Evaluate all input values.

  2. If the animation input value is negative or greater than or equal to the number of glTF animations in the asset,

    1. activate the err output flow and skip the next steps.

  3. If the animation state dynamic array exists and contains an entry with the same animation index,

    1. remove it from the array and stop the playing animation. The animated properties MUST keep their current values and the previously associated completed flows MUST NOT be activated.

  4. Activate the out output flow.

3.6.4.3.3. Animation Stop At

Type

animation/stopAt

Schedule stopping a playing animation

Input flow sockets

in

The entry flow into this node

Input value sockets

int animation

Animation index

float stopTime

Stop time

Output flow sockets

out

The flow to be activated if the input values are valid

err

The flow to be activated if any of the input values is invalid

completed

The flow to be activated after the animation stops

This node stops a playing animation.

This node has no internal state.

When the in input flow is activated:

  1. Evaluate all input values.

  2. If the animation input value is negative or greater than or equal to the number of glTF animations in the asset,

    1. activate the err output flow and skip the next steps.

  3. If the stopTime input value is NaN,

    1. activate the err output flow and skip the next steps.

  4. If the animation state dynamic array exists and does contain an entry with the same animation index,

    1. update the entry’s stop completion pointer to the completed output flow of this node;

    2. update the entry’s stop time to the stopTime input value.

  5. Activate the out output flow.

3.6.5. Event Nodes

3.6.5.1. Lifecycle Event Nodes
3.6.5.1.1. On Start

Type

event/onStart

Start event

Output flow sockets

out

The flow to be activated when the start event happens

This node is activated when all glTF asset resources are loaded and ready for rendering and interactions.

This node has no internal state.

3.6.5.1.2. On Tick

Type

event/onTick

Tick event

Output value sockets

float timeSinceStart

Relative time in seconds since the graph execution start

float timeSinceLastTick

Relative time in seconds since the last tick occurred

Output flow sockets

out

The flow to be activated when the tick event happens

This node is activated when a tick occurs. There will be at most one tick per rendered frame, which SHOULD align with frame time, but there are no guarantees of time elapsed between ticks.

The internal state of this node consists of two floating-point time values initialized to NaN. They MUST be set to their effective values before the out output flow is activated.

3.6.5.2. Custom Event Nodes
3.6.5.2.1. Receive

Type

event/receive

Receive a custom event

Configuration

int event

The custom event index

Output value sockets

<custom>

Output values defined by the custom event

This node is activated when a custom event specified by the event configuration value occurs. The types, names, and semantics of the output value sockets are defined by the custom event index.

The event configuration value MUST be non-negative and less than the total number of custom event definitions, otherwise the node is invalid.

This node has no internal state.

3.6.5.2.2. Send

Type

event/send

Send a custom event

Configuration

int event

The custom event index

Input flow sockets

in

The entry flow into this node

Input value sockets

<custom>

Input values defined by the custom event

Output flow sockets

out

The flow to be activated after sending the event

err

The flow to be activated if any of the input values is invalid

This node sends a custom event specified by the event configuration value. The types, names, and validation rules of the input value sockets are defined by the custom event index.

The event configuration value MUST be non-negative and less than the total number of custom event definitions, otherwise the node is invalid.

This node has no internal state.

When the in input flow is activated:

  1. Evaluate all input values.

  2. If any of them is invalid, activate the err output flow and skip the next step.

  3. Activate the out output flow.

4. JSON Syntax

4.1. General

A KHR_interactivity extension object is added to the root-level extensions property. It contains four arrays corresponding to four interactivity concepts: types, events, variables, and nodes. As with the core glTF spec, if a JSON array is empty, it MUST be omitted from the asset.

{
  "asset": {
    "version": "2.0"
  },
  "extensionsUsed": ["KHR_interactivity"],
  "extensions": {
    "KHR_interactivity": {
      "types": [
        //
      ],
      "events": [
        //
      ],
      "variables": [
        //
      ],
      "nodes": [
        //
      ]
    }
  }
}

4.2. Types

The types array defines mappings between graph-local type indices and the recognized type signatures.

The following example defines type 0 as float2, type 1 as int, and type 2 as float:

"types": [
  {
    "signature": "float2"
  },
  {
    "signature": "int"
  },
  {
    "signature": "float"
  }
]

The signature value MUST be one of the value types defined in this extension specification or "custom". In the latter case, the type semantic MUST be provided by an additional extension.

Non-custom signature MUST NOT appear more than once.

4.3. Events

The events array defines external identifiers and value socket types for custom events.

The following example defines a custom “checkout” event with an external identifier and one value socket:

"events": [
  {
    "id": "checkout",
    "values": [
      {
        "id": "variant",
        "type": 1
      }
    ]
  }
]

The event ID value is an application-specific event identifier recognized by the execution environment. If the id property is undefined, the event is considered internal to the graph.

The values array defines IDs and type indices of the sockets associated with the event. If the array is undefined, the event has no associated value sockets.

4.4. Variables

The variables array defines custom variables with their types and optional initialization values.

The following example defines a custom variable with its initial value:

"variables": [
  {
    "type": 0,
    "value": [0.5,  0.5]
  }
]

The type value defines the index of the variable type.

The value array, if present, defines the initial variable value. The following table defines array size and default values for all value types defined in this extension.

Type Array size Default value

bool

1

Boolean false

float

1

Floating-point zero

float2

2

Two floating-point zeros

float3

3

Three floating-point zeros

float4

4

Four floating-point zeros

float4x4

16

Sixteen floating-point zeros

int

1

Integer zero

If the variable type is custom, the value property is defined by the extension defining the custom type.

4.5. Nodes

The nodes array defines the behavior graph.

Each element of the nodes array represents a node instance, i.e., it specifies node’s type, configuration, sources of input value sockets, and pointers of the output flow sockets.

Input value sockets MAY have inline constant values; in this case, the value socket type MUST be defined.

Inline values and configurations use JSON arrays similarly to the initial variable values.

The following example instantiates a math/add node that has both its input value sockets filled with inline integer values.

"nodes": [
  {
    "type": "math/add",
    "values": [
      {
        "id": "a",
        "value": [1],
        "type": 1
      },
      {
        "id": "b",
        "value": [2],
        "type": 1
      }
    ]
  }
]

The following example instantiates three nodes. The math/sub node has both its input value sockets connected to output value sockets of two other nodes: math/pi and math/e.

"nodes": [
  {
    "type": "math/pi"
  },
  {
    "type": "math/e"
  },
  {
    "type": "math/sub",
    "values": [
      {
        "id": "a",
        "node": 0,
        "socket": "value"
      },
      {
        "id": "b",
        "node": 1,
        "socket": "value"
      }
    ]
  }
]

The following example instantiates two nodes. The variable/set node sets a custom variable with index 0 when the start event happens.

"variables": [
  {
    "type": 1
  }
],
"nodes": [
  {
    "type": "variable/set",
    "configuration": [
      {
        "id": "variable",
        "value": [0]
      }
    ],
    "values": [
      {
        "id": "value",
        "value": [1],
        "type": 1
      }
    ]
  },
  {
    "type": "event/onStart",
    "flows": [
      {
        "id": "out",
        "node": 0,
        "socket": "in"
      }
    ]
  }
]

The type property is required; it defines semantics and validation of the configuration, values, and flows arrays.

The same id value MUST NOT be used more than once within each of the configuration, values, and flows arrays.

If the node type has configuration, the configuration array MUST provide all configuration parameters as inline values.

If the node type has input value sockets, the values array MUST connect all input value sockets to other nodes or fill them with inline values. Additionally, for each element of the values array:

  • value and node properties MUST NOT be defined at the same time;

  • if value is defined, type MUST also be defined.