Skip to content

Learning Aardvark.Media #1

ThomasOrtner edited this page Feb 2, 2018 · 8 revisions

'Model-Update-View': Apps in Aardvark.Media

The following example shall lead us through all topics relevant to building powerful apps in Aardvark.Media combining 3D graphics and a web-based GUI in a declarative and functional way. We will render a couple of 3D objects and transform them with controls we write from scratch. We demonstrate how the Model-Update-View paradigma works for generating GUI and 3D views and how to build powerful apps by composition. In the later chapters we we will demonstrate how our mod system and adaptive data structures handle change and are essential to tame complexity in larger apps.

Building a Numeric Control

First, we will build a numeric control, which lets the users increment and decrement an float value. We will approach this problem in the sequence of Model-Update-View, starting with the model.

We simply build a record type holding our float value. For future use, we also attribute it with 'DomainType'.

[<DomainType>]
type Model = { value : float }

Specifying update is a two-fold effort of first defining actions of what does our app do to the model ...

type Action = 
    | Increment
    | Decrement

... and second of defining how do these actions affect our model.

let update (m : Model) (a : Action) =
    match a with 
        | Increment -> { m with value = m.value + 0.1 }
        | Decrement -> { m with value = m.value - 0.1 }                

The typical signature of update is Model->Action->Model, taking the current model and an action and returning an updated version of our model. Although, actions can in principle come from anywhere our actions for now will originate from our app's UI, which we specify in the view function. Consequently, the view function defines how our model is visualized and how the user can manipulate the model via UI elements.

First, we create two buttons , one for incrementing and one for decrementing our model's value.

button [clazz "ui button"; onClick (fun _ -> Increment)] [text "+"]
button [clazz "ui button"; onClick (fun _ -> Decrement)] [text "-"]

Besides the architecture of ELM we were also inspired by ELMs way of constructing DOM (Document Object Model) elements, such as divs, buttons, and unordered lists. This allows us to emit HTML code for our views directly from F# code. The code above roughly equivalents to:

<button class="ui button" onClick=Increment()>+</button>
<button class="ui button" onClick=Decrement()>-</button>

You may notice that we write the 'class' attribute as 'clazz'. This is not because we are the cool kids, but naturally 'class' is a reserved keyword in F#. Most of the html and Aardvark.UI tags are 'elems' consisting of an attribute list and a child list, while 'voidelems' such as 'br' only have attributes. A list of all available tags can be found here. The finished view function, also containing a visualization for our model, looks like the following:

let view (m : MModel) =
    require Html.semui ( 
        body [] (        
            [
                div [] [
                        button [clazz "ui button"; onClick (fun _ -> Increment)] [text "+"]
                        button [clazz "ui button"; onClick (fun _ -> Decrement)] [text "-"]
                        br []
                        text "my value:"
                        br []
                        Incremental.text (m.value |> Mod.map(fun x -> sprintf "%.1f" x))
                    ]
            ]
        )
    )

'require Html.semui' enables the emitted html code to refer to web ressources such as css style sheets or javascript libraries. In this case we emit a script reference for the UI styling library Semantic UI. For now we ignore MModel, Incremental.text, and Mod.map which will be discussed in later sections.

The final lines of every app is putting the individual parts together...

let app =
    {
        unpersist = Unpersist.instance        //ignore for now
        threads   = fun _ -> ThreadPool.empty //ignore for now
        initial   = { value = 0.0 }
        update    = update
        view      = view
    }

let start() = App.start app

...resulting in the following output:

numeric control output

Building a Vector Control

Our vector control shall allow us to manipulate XYZ of a 3D vector. Naturally, we want to reuse our previous example and just combine three numeric controls via composition. First, we create a composed VectorModel by using a record type consisting of individual NumericModels for X,Y and Z.

[<DomainType>]
type VectorModel = { 
    x : NumericModel
    y : NumericModel
    z : NumericModel
}

Our composed app now has a composed model and of course needs a composed action describing what our VectorControl can do. If we come accross an action in this control either X,Y or Z is incremented or decremented. However, increment and decrement are already captured in the NumericControl.Action, so we don't want to specify that again. Since an action is either an update on x or on y or on z our composed action is a union type.

type Action = 
    | UpdateX of NumericControl.Action
    | UpdateY of NumericControl.Action
    | UpdateZ of NumericControl.Action

Looking at Model and Action, our VectorControl is quite limited, it can't do anything by itself really. Therefore, our update funciton just simply calls update functions from the NumericControl.

let update (m : VectorModel) (a : Action) =
    match a with
        | UpdateX a -> { m with x = NumericControl.update m.x a }
        | UpdateY a -> { m with y = NumericControl.update m.y a }
        | UpdateZ a -> { m with z = NumericControl.update m.z a }

The results are handed to the relevant part of our composed model, e.g. the X coordinate. The match on UpdateX unpacks the composed action a and we can directly pass a NumericControl.Action to update. Update returns a new NumericModel which we then use to update the VectorModel.

Such pure composition apps are quite frequent, for instance when making a property control consisting of numeric, bool, and string values, or when making a sidebar app consisting of multiple property apps. But more often composing apps add their own actions. In this case we could add Normalize and Reset to our VectorControl.

type Action = 
    | UpdateX of NumericControl.Action
    | UpdateY of NumericControl.Action
    | UpdateZ of NumericControl.Action
    | Normalize
    | Reset

let update (m : VectorModel) (a : Action) =
    match a with
        | UpdateX a -> { m with x = NumericControl.update m.x a }
        | UpdateY a -> { m with y = NumericControl.update m.y a }
        | UpdateZ a -> { m with z = NumericControl.update m.z a }
        | Normalize -> 
                let v = V3d(m.x.value,m.y.value,m.z.value)
                v.Normalized |> toVectorModel                                
        | Reset -> VectorModel.initial

Before starting with the view, we will adapt the viewing code of the NumericControl a bit. In the way we programmed it above it is not very suitable to be stacked together into a VectorControl. That being said, a composing app is responsible for how the composed parts should be viewed together.

let view' (m : MNumericModel) =
    require Html.semui (         
        table [][ 
            tr[] [
                td[] [a [clazz "ui label circular Big"] [Incremental.text (m.value |> Mod.map(fun x -> " " + x.ToString()))]]
                td[] [
                    button [clazz "ui icon button"; onClick (fun _ -> Increment)] [text "+"]
                    button [clazz "ui icon button"; onClick (fun _ -> Decrement)] [text "-"]                                        
                ]
            ]
        ]        
    )

This is our updated view function view', which uses an html table to align things more neatly. The result looks like this:

improved numeric control

Our VectorControl's view function should now call the individual NumericControl view functions and put them in a nice table layout.

let view (m : MVectorModel) =
    require Html.semui (             
        div[][
            table [] [
                tr[][
                    td[][a [clazz "ui label circular Big"][text "X:"]]
                    td[][NumericControl.view' m.x |> UI.map UpdateX]
                ]
                tr[][
                    td[][a [clazz "ui label circular Big"][text "Y:"]]
                    td[][NumericControl.view' m.y |> UI.map UpdateY]
                ]
                tr[][
                    td[][a [clazz "ui label circular Big"][text "Z:"]]
                    td[][NumericControl.view' m.z |> UI.map UpdateZ]
                ]              
                tr[][
                    td[attribute "colspan" "2"][
                        div[clazz "ui buttons small"][
                            button [clazz "ui button"; onClick (fun _ -> Normalize)] [text "Norm"]
                            button [clazz "ui button"; onClick (fun _ -> Reset)] [text "Reset"]
                        ]
                    ]
                ]
            ]               
        ]
    )

Everything looks quite familiar, except for UI.map. The view function of the VectorControl is supposed to return DomNode, while NumericControl.view' returns a DomNode<NumericControl.Action> resulting in a type mismatch. UI.map resolves this problem by lifting the action type of the subapp to the action type of the composing app by simply mapping it to the respective composing action.

VectorControl

The deep nesting of domnodes results from the quite cumbersome way to specify tables in html. Further, the vector control is probably not the most elegant control you have ever seen. However, we will stick to the this level of verbosity for explanatory reasons. Using the UI primitives of Aardvark.Media results in something more like this:

Html.SemUi.accordion "Scale" "Expand" true [
    div [] [Vector3d.view model.scale |> UI.map V3dMessage]
]

nice vector control

In the described implementation we used composition to build a vector control by just reusing our numeric control code. In the same way we can compose a transformation app for manipulating 3 vectors for translation, rotation, and scale. Instead of implementing this step by step we use it to revisit the composition illustration from earlier filling it with actual code.

nice vector control

In this chapter we covered how the meaning of model-update-view and what apps look like in Aardvark.Media. We built a small app from scratch and composed a more powerful app from that. We further illustrated how to use domnodes which ultimately emit html and javascript code for our GUI.

Before expanding our example with a 3D view we will revisit the ELM architecture and cover incremental evaluation.

Incremental Evaluation

When naively implementing the ELM architecture illustrated above we execute the whole view function on every update of the model. This does not scale well with larger DOM trees, larger SceneGraphs, nor high frequency updates, such as camera movement. We actually want to only update those parts of the scene which have actually changed - so we want incremental updates.

incremental elm loop

Aardvark.Rendering is a powerful rendering engine which supports incremental updates of 3D scenes, while Aardvark.Media does the same thing for DomNodes. We will not go into details here (refer to publications) but at the heart of all this is the Mod System. Instead of specifying on how to react to change explicitly a dependency is created between a data field in the model and a visual representation, e.g. a color : C4b and the rendering color of an Sg.Cube.

Within our nice unidirectional dataflow we don't want to use the mod system directly. Therefore, a compile step generates an MModel from our Model, translating every data field to a mod field which is aware of being changed. Most Sg nodes take mod version and for GUI elements there are incremental tags, such as Incremental.Text taking an IMod.

incremental elm loop with code

Here the incremental loop is illustrated with actual code. Our model contains a 3D vector (V3d) specifying a scale. Our compiler, the diff generator, generates an MModel wrapping the V3d into a mod. Our 3D view shows a cube which is transformed and further triggers a reset action on double click. Sg.trafo takes Mod. To generate a Mod we use Mod.map and speficy a mapping Mod -> Mod. The Sg view and the domnode view can both trigger a reset action which changes our model. This change is reflected in an update of MModel.value which only triggers a reevaluation of Sg.trafo.

  • mods

  • adaptive

  • MModels for 3D view

  • MModels for GUI views (AttributeMaps)

  • adaptive pendants of datastructures (alist, aset, amap, ...)

Scaling 3D Objects

(work in progress)

continue with Learning Aardvark.Media #2

Clone this wiki locally
You can’t perform that action at this time.