Skip to content

Learning Aardvark.Media #2

ThomasOrtner edited this page Feb 2, 2018 · 3 revisions

Repos and The build system

(work in progress)

Adaptive Datastructures

In the previous sections only dealt with very limited scene modifications, namely the change of a single value or property e.g. the scale of a box. In this section we will focus on how to handle structural changes to our scene. Our leading example will be the BoxSelectionDemo, also availabe in Aardvark.Media. Our application can add and remove boxes and further offers interaction to select and hover boxes in 3D as well as in a GUI list view. Adaptive datastructures are crucial for dealing with structural change, but we will also cover other relevant topics along the way, such as integrating a camera controler,...

Compilation Scheme

When recalling the previous chapter, we got to know two types that 'react to change', IMod<'a> for wrapping a value into a Mod and the mutable type (MType) provided by the diffgenerator. Since functional programming is all about data structures we also need adaptive versions of collections such as lists, sequences, or maps.

compilationscheme

This tables shows the compilation scheme of the diffgenerator, telling us which type is mapped to which adaptive type, for instance, a plist in our domain type Abc will be an alist in our mutable type MAbc. More details on how the diff generator converts one type to another can be found here.

3D rendering control

Back to our BoxSelectionApp, we start again with the model

[<DomainType>]
type BoxSelectionDemoModel = {
    boxes : plist<Box3d>
}

containing a plist of Box3ds. In MBoxSelectionDemoModel this will be accessible for our view function as alist. We want our 3D view and our GUI, a list of boxes, to react to changes of this list, e.g. add a new 3D cube if a box is added. Alist not only reacts to structural changes, but also to changes to the properties of each element efficiently propagated via the dependency graph.

[<DomainType>]
type VisibleBox = {
    geometry : Box3d
    color    : C4b    

    [<TreatAsValue>]
    id       : string
}

[<DomainType>]
type BoxSelectionDemoModel = {
    camera : CameraControllerState
    
    boxes : plist<VisibleBox>    
    boxHovered : option<string>    
}

To make our naked box geometries a bit more interesting we wrap them into VisibleBox which also holds a color and an id. The attribute TreatAsValue tells the diff generator to not compile this field to a Mod. Of course, we want our ids constant, otherwise it will be difficult to identify our boxes.

As you can see, we also included a CameraControllerState into our model. To have an interactive 3D view, we simply compose a CameraControllerApp into our app starting with just adding its model, which should rather be called CameraControllerModel if we were a 100% strict with the naming. Next, we wrap the camera controller actions into our BoxSelection actions and handle the action in the update function, as with any other composition.

type Action =
    | CameraMessage of CameraControllerMessage
    ...
    
let update (model : BoxSelectionDemoModel) (act : Action) =
    match act with
        | CameraMessage m -> 
            { model with camera = CameraController.update model.camera m }

Finally, we embed the CameraController into our view function. The CameraController's view is exposed as an incremental control. On second thought, this is just another view function taking a model, an action, and some additonal attributes. Most notable here are the AttributeMap and the child nodes as last argument.

 let view (model : MBoxSelectionDemoModel) =                                   
        let frustum = Mod.constant (Frustum.perspective 60.0 0.1 100.0 1.0)
              
        div [clazz "ui"; style "background: #1B1C1E"] [
            CameraController.controlledControl model.camera CameraMessage frustum
                (AttributeMap.ofList [
                    attribute "style" "width:65%; height: 100%; float: left;"
                ])
                (
                    Sg.box (Mod.constant C4b.Gray) (Mod.constant Box3d.Unit)
                        |> Sg.shader {
                            do! DefaultSurfaces.trafo
                            do! DefaultSurfaces.vertexColor
                            do! DefaultSurfaces.simpleLighting
                            }                
                        |> Sg.requirePicking
                        |> Sg.noEvents                       
                )
        ]

AttributeMaps are a special way of defining attributes for a DomNode, which may change. In the above case we us static attributes for an incremental DomNode, so we can just use static syntax and convert it to an AttributeMap via AttributeMap.ofList. The child node of the renderControl is of the type ISg<'msg>, in our case a static grey box not capable of triggering any actions. Most interestingly the output of the RenderControl is a DomNode of Action, since the 3D rendering frames are written into an image. Therefore our RenderControl composes with any other DomNodes, just as a div or a text would. Our interactive intermediate result looks like this:

grey cube

ISg of Action

Before dealing with multiple boxes we think of the actions we want to support and implement them for a single box.

type Action =
    | CameraMessage    of CameraControllerMessage
    | Select           of string     
    | ClearSelection
    | HoverIn          of string 
    | HoverOut                     
    | AddBox
    | RemoveBox    

We want to click on a box and Select it resulting in a persistent highlighting (color red) or a peek highlighting on HoverIn (color blue) which stops on HoverOut. Similar to attaching actions to DomNodes (onclick button) we can specify which actions are triggered when clicking on a Sg.Box

Sg.box color box.geometry
    |> Sg.shader {
        do! DefaultSurfaces.trafo
        do! DefaultSurfaces.vertexColor
        do! DefaultSurfaces.simpleLighting
        }                
    |> Sg.requirePicking
    |> Sg.noEvents
    |> Sg.withEvents [
        Sg.onClick (fun _  -> Select (box.id |> Mod.force))
        Sg.onEnter (fun _  -> Enter  (box.id |> Mod.force))
        Sg.onLeave (fun () -> Exit)
    ]

Since we need to know which box has been selected/hovered later on we send the box.id with the action into the model update. With Mod.force we unpack box.id, which is a Mod, to a string. This is the only valid situation where we can just unpack a mod. Further, our id is not modifyable anyway in this case. In any other case we will most likely use adaptive behavior where we still need it. For simplicity we will we will only go through the hovering interaction here.

let update (model : BoxSelectionDemoModel) (act : Action) =
    match act with
        | CameraMessage m -> 
            { model with camera = CameraController.update model.camera m }
        | HoverIn id -> { model with boxHovered = Some id }            
        | HoverOut   -> { model with boxHovered = None }        
        | _          -> model

We simple map our hovering interaction to the option of boxid. Either, there is an id or no hovered has taken place. Now we want to react on this changing hovering state by changing the cubes color. The diffgenerator gives us an IMod<Option> in the viewfunction, while we need an IMod. Once again, we want to just map things from IMod<'a> to Imod<'b> using Mod.map.

let color = 
    model.boxHovered |> Mod.map (
        function x -> match x with
                        | Some k -> if k = "box" then C4b.Blue else C4b.Gray
                        | None -> C4b.Gray)

open topics

  • Incremental Domnodes and Attribute maps
  • performance implications
  • BoxSelection Demo

continue with Aardvark.Media #3

Clone this wiki locally
You can’t perform that action at this time.