Tr3 (as in "tree") is a data flow graph with the following features:
- Node with names, values, edges, and closures
- Edges with inputs, outputs, and switches
- Values that transform, as it flows through a graph
- Realtime coordination between devices
- Human readable declarative data flow
- Play nice with syntax highlighting and code folding
- Minimal use of parsing cruft, like semicommas, and commas
- Explore live patching without crashing or infinite loops
- Concise expression of full body input as controller
- Synchronize state while managing circular references
- Synchronize amorphous devices and ledgers
Each node has two kinds of edges: tree and graph. The tree allows each node to be addressed by name and its relationship within a group. The graph allows each node to activate each other based on inputs and outputs.
Each node has a single parent with any number of children.
a { b c } // a has 2 children: b & c
// b & c have 1 parent and no children
Declaring a path will auto create a tree of names
a.b.c // produces the structure `a { b { c } }`
A tree can be decorated with sub trees
a {b c}.{d e} // produces `a { b { d e } c { d e } }`
A tree can copy the contents of another tree with a : name
a {b c}.{d e} // produces `a { b { d e } c { d e } }`
z: a // produces `z { b { d e } c { d e } }`
Each node may have any number of input and output edges, which attach to other nodes. A node can activate
- other nodes when its value changes as outputs(
>>
) or - itself when another node's value changs as inputs (
<<
), or - synchronize both nodes as both input and ouput
<>
).
b >> c // b flows to c, akin to a function call
d << e // d flows from e, akin to a callback
f <> g // f & g flow between each other, akin to sync
Tr3 allows cyclic graphs which auto break activation loops. When a node is activated, it sends an event to its output edges. The event contains a shared set of places that the event has visited. When it encounters a node that it has visited before, it stops.
a >> b // if a activates, it will activate b
b >> c // which, in turn, activates c
c >> a // and finally, c stops here
So, no infinite loops.
So, in the above a
, b
, c
example, the activation could start anywhere:
a! activates b! activates c! // starts at a, stops at c
b! activates c! activates a! // starts at b, stops at a
c! activates a! activates b! // starts at c, stops at b
This is a simple way to synchronize a model. Akin to how a co-pilot's wheel synchronizes in a cockpit.
Swift source code may attach a closure to a Tr3 node, which gets executed whenever a that node is activated.
Given the tr3 script snippet:
sky { draw { brush { size << midi.modulationWheel } } }
write a closure in Swift to capture a changed value
brush.findPath("sky.draw.brush.size")?.addClosure { tr3, _ in
self.brushRadius = tr3.CGFloatVal() ?? 1 }
In the above example, attach a closure to sky.draw.brush.size
, which then updates its internal value brushRadius
.
Each node may have a value of: scalar, expression, string, or embedded script
a (1) // scalar with an initial value of 1
b (0…1) // scalar that ranges between 0 and 1
c (0…127 = 1) // scalar betwwn 0 and 127, defaulting to 1
d "yo" // a string value "yo"
e (x 0…1, y 0…1) // an expression (see below)
Tr3 automatically remaps scalar ranges, given the nodes b
& c
b (0…1) // range 0 to 1, defaults to 0
c (0…127 = 1) // range 0 to 127, with initial value of 1
b <> c // synchronize b and c and auto-remap values
When the value of b
is changed to 0.5
it activates c
and remaps its value to 63
;
When the value of c
is changed to 31
, it activates b
and remapts its value to 0.25
A common case are sensors, which have a fixed range of values. For example, a 3G (gravity) accelerometer may have a range from -3.0
to 3.0
accelerometer (x -3.0…3.0, y -3.0…3.0, z -3.0…3.0) >> model
model (x -1…1, y -1…1, z -1…1) // auto rescale
a (0…1) >> b // may pass along value to b
b >> c // has no value, will forward a to c
c (0…10) // gets a's value via b, remaps ranges
Activations values can be passed as either inputs, outputs, or syncs
a >> b(1) // an activated a (or a!) sends 1 to b
b << c(2) // an activated c (or c!) sends 2 to a
d <> e(3) // d! sends a 3, while c! does nothing
f >> g(0…1 = 0) // f! sends a ranged 0 to g
h << i(0…1 = 1) // i! sends a ranged 1 to h
Sending a ranged value to receiver will remap values, which can become a convenient way to set min
, mid
, or max
values
j(10…20) << k(0…1 = 0) // k! maps j to 10 (min)
m(10…20) << n(0…1 = 0.5) // n! maps m to 15 (mid)
p(10…20) << q(0…1 = 1) // q! maps p to 20 (max)
In addition to copying a tree, a new tree can connect edges by name
a {b c}.{d e}
x@a <@ a // input from a˚˚
y@a @> a // output to a˚˚
z@a <@> a // synchronize with a˚˚
which expands to
a { b { d e } c { d e } }
x << a { b << a.b { d << a.b.d, e << a.b.e }
c << a.c { d << a.c.e, e << a.c.e } }
y >> a { b >> a.b { d >> a.b.d, e >> a.b.e }
c >> a.c { d >> a.c.e, e >> a.c.e } }
z <> a { b <> a.b { d <> a.b.d, e <> a.b.e }
c <> a.c { d <> a.c.e, e <> a.c.e } }
Thus, it is possible to mirror a model in realtime. Use cases include: co-pilot in cockpit, "digital twin" for building information modeling, overriding input contollers, dancing with robots
An epression is a series of named values and conditionals; they are expessed together as a group
a (x 1, y 2) // x and y are sent together as a tuple
b (x 0…1, y 0…1) // can contain ranges
c (x 0…1 = 1, y 0…1 = 1) // and default values
A receiver may capture a subset of a send event
z (x 1, y 2) // when z! (is activated)
d (x 0) << z // z! => d(x 1) -- ignore y
e (y 0) << z // z! => e(y 2) -- ignore x
f (x 0, y 0, t 0) << z // z! is ignored, no z.t
But, the sending event must have all of the values captured by the receiver, or it will be ignored
g (x==0, y 0) << z // z! is ignored as z.x != 0
h (x==1, y 0) << z // z! activates h(x 1, y 2)
i (x<10, y<10) << z // z! activates i(x 1, y 2)
j (x in -1…3, y 0) << z // z! activates j(x 1, y 2)
k (x 0, y 0, z 0, t 0) // z! ignored due to missing t
Override nodes with values
a {b c}.{d(1) e} // produces `a { b { d(1) e } c { d (1) e } }`
a.b.d (2) // changes to `a { b { d(2) e } c { d (1) e } }`
Include subtrees with wildcards. The new ˚
(option-k) wildcard behaves like an Xpath /*/
where it will perform a search on children, grandchildren, and so on. Using ˚.
includes all leaves, and ˚˚
will include the whole subtree
a {b c}.{d e} // produces `a { b { d e } c { d e } }`
p << a.*.d // produces `p << (a.b.d, a.c.d)`
q << a˚d // produces `q << (a.b.d, a.c.d)`
r << a˚. // produces `r << (a.b.d, a.b.e, a.c.d, a.c.e)`
s << a˚˚ // produces `s << (a a.b, a.b.d, a.b.e, a.c, a.c.d, a.c.e)`
Wildcard searches can occur on both left and rights sides to support fully connected trees and graphs
˚˚<<.. // flow from each node to its parent, bottom up
˚˚>>.* // flow from each node to its children, top down
˚˚<>.. // flow in both directions, middle out?
Because the visitor pattern breaks loops, the ˚˚<>..
maps well to devices that combine sensors and actuators, such as:
- a flying fader on a mix board,
- a co-pilot's steering wheel
- the joints on an Human body capture skeleton
- future hash trees (like Merkle trees) and graphs
Edges may contain ternaries that switches dataflow. Somewhat akin to railroad switch, traffic may flow in either direction and do need to reevealate the switch as passes through.
conditionals may switch the flow of data
a >> (b ? c : d) // a flows to either c or d, when b activates
e << (f ? g : h) // f directs flow from either g or h, when f acts
i <> (j ? k : l) // i synchronizes with either k or l, when j acts
m <> (n ? n1 | p ? p1 | q ? q1) // radio button style, akin to solo switch
conditionals may also compare its state
a >> (b > 0 ? c : d) // a flows to either c or d, when b acts (default behavior)
e << (f == 1 ? g : h) // g or h flows to e, based on last f activation
i <> (j1 < j2 ? k : l) // i syncs with either k or l, based on last j1 or j2 acts
m <> (n > p ? n1 | p > q ? p1 | q > 0 ? q1) // radio button style
when a comparison changes is state, it reevaluates its chain of conditions
- when
b
activates, it reevaluatesb > 0
- when
f
activates, it reevaluatesf == 1
- when either
j1
orj2
activates, it reevalsj1 < j2
- when
n
,p
, orq
acts, it reevalsn>p
,p>q
, andq>0
Ternaries act like railroad switches, where the condition merely switches the gate. So, each event passing through a gate does not need to re-invoke the condition
- when
b
acts, it connectsc
and disconnectsd
- when
n
,p
, orq
acts, it is switching betweenn1
,p1
,q1
Ternaries may aggregate multiple ihputs or broadcast to multiple outputs
a {b c}.{d e}.{f g} // produces `a{b {d {f g} e {f g}} c {d {f g} e {f g}}}`
p >> (a.b ? b˚. | a.c ? c˚.) // broadcast p to all leaf nodes of either b or c
q << (a.b ? b˚. | a.c ? c˚.) // aggregate to q from all leaves of either b or c
Tr3 may include external script inside of double curly brackets {{ whatever }}
. Whatever is inside the double bracks is ignored by the script, but is available calling swift code. This is intended for the app DeepMuse
to embed shader code, which can be safely recompiled at runtime.
example {
metal {{
// Metal code goes here
…
}}
gl {{
// OpenGL code goes here
…
}}
js {{
// javascript
…
}}
whatever {{
// you want
…
}}
}
When activating example.*!
the Tr3 nodes named metal
, gl
, js
, and whatever
are activated.
Any closure, attached to those nodes, can get the contents between the brakets {{ … }}
The contents are whatever you want, they are interpreted by the closure at run time.
Basic example of syntax may be found in the test cases here:
Tests/Tr3Tests/Tr3Tests.swift
contain basic syntax tests
The Deep Muse
app script should provide some insight as to how Tr3 is used in a production app, which is also in the test suite
Sources/Tr3/Resources/*.tr3.h
contains scripts fromDeep Muse
appSources/Tr3/Resources/test.output.tr3.h
contains scripts fromDeep Muse
app
- Add timeline class to record, playback, state of tr3 (2021)
- Extend expressions with string parsing, using Par syntax (2022)
- Add node based visitor coordination to synchronize beyond a single device (2022)
- Support cryptographically synchronized timelines between devices (2022)
- Map Pythonic
INDENT
/REDENT
/DEDENT
to{
/,
/}
respectively - Better error trapping for parsing errors
- Merge with Par, where modified BNF becomes a value type
- Fully connected node layers
- Sha256
- L-systems to generate Merkle trees
- Tanh, RELU support
- Refractory periods for edges (ADSR style)
- String parser as expression
- Runtime edge creation, akin to neuroplasticity
- Command line tool to activate and inspect ontology
- Command line interpreter to create and manipulate graphs
- embed Metal shaders (for Muse Sky app) (worked for OpenGL)
- optimize for MLIR?
- support AutoGraph
- Visualize Graph, port of Prefuse version
- Visualize dataflow as cross between Sankey and Node diagram (see Vizster demos)
- auto layout vertical / horizontal
- auto map
INDENT
/REDENT
/DEDENT
to{
/,
/}
respectively - Pinch to Fold/Unfold
Par - parser for DSLs and flexible NLP in Swift
- tree + graph base parser
- contains a definition of the Tr3 Syntax
- vertically integrated with Tr3
- Source here
Tr3D3 (pending)
- simple visualization of the Tr3 graph, using D3JS
- continuation of prototype of previous version of Tr3
- Proof of concept here (using Prefuse toolkit)
Tr3Dock (pending)
- Tr3 based UI with dataflow broadcasting and ternaries
- Demo of UI here
Toy Visual Synth for iPad and iPhone called "Muse Sky"
- See test script in Sources/Tr3/Resources/*.tr3.h
- See test output in Sources/Tr3/Resources/test.output.tr3.h
- Code folding and syntax highlighting works in Xcode
- Demo here
Encourage users to tweak Tr3 scripts without recompiling
- pass along Tr3 scripts, somewhat akin to Midi files for music synthesis
- connect musical instruments to visual synth via OSC, Midi, or proprietary APIs
Inspired by:
- Analog music synthesizers, like Moog Modular, Arp 2600, with patchchords
- Media Dataflow scripting : Max, QuartzComposer, Plogue Bidule
Check out test.robot.input.tr3.h
, which defines a Humanoid robot in three lines of code:
body {left right}.{shoulder.elbow.wrist {thumb index middle ring pinky}.{meta prox dist} hip.knee.ankle.toes}
˚˚ <> ..
˚˚ { pos(x 0…1, y 0…1, z 0…1) angle(roll %360, pitch %360, yaw %360) mm(0…3000)})
Imagine wearing a motion capture suit that you have record every movement and then playback
- Record total state of
graph << body˚˚
- Playback total state of
graph >> body˚˚
- Create a functional mirror
twin: body ←@→ body
- Inspired by a Kinect/OpenNI experiment, shown here
In 2004, NASA put on a conference called Virtual Iron Bird to encourage modeling of Spacecraft. One question was how to manage the dataflow between sensors and actuators. How does one simulate a vehicle? Or synchonize co-pilot controls?
- A billion people with a billion private graphs ,
- each with millions of nodes,
- which synchronize