Scenegraph

The scenegraph is an extensible core structure which contains the rendering and input state of Wayfire. It is built like a tree, where each nodes implements the scene::node_t interface. Each node may have children (usually nodes with children are subclasses of scene::floating_inner_node_t), or it may be a leaf node. Inner nodes serve two purposes. On one hand, they can be used to group nodes together (so, views can be organized in outputs, layers, etc.). On the other hand, nodes can apply transformations to their children. A core design principle is the following: each node, and it alone, knows what kind of transformation it applies to its children. Examples of nodes are layers, outputs, views, view transformers and plugin grabs. So how does all of this mesh together?

Stacking order

The first and simplest task accomplished with the scenegraph is that its structure encodes the stacking order of views and other rendering/input elements. This used to be a task of each output's workspace-manager. And as they say, a picture is worth a thousand words, so here is an example of Wayfire's scenegraph at runtime (many nodes were omitted for the sake of clarity):

There are multiple notable things here.

First, there is one node per layer. Each output has its own node in each layer, which contains the views on that output belonging to that layer. This is different than previous verions of Wayfire, where each output has its own list of layers. The new version enables core to handle views visible on all outputs - this can be achieved by for example putting the view directly in the children list of the layer node (however, most of the plugins in the main repo may have a hard time dealing with such views, because they assume each view is bound to a single output).

Each view contains multiple nodes. A view has a root node which contains the view and its child views (most commonly dialogs). As a child of the view root, each view has a so called "transformed" node. This is the node which contains the view with its subsurfaces and transformers. In the hierarchy, it is followed by the transformer nodes themselves (in the example blur) and then finally by a surface root node. The surface root marks the beginning of the surface hierarchy belonging to the view. The principle is this: each surface is represented by a surface root, and a surface node attached to it. The root nodes of subsurfaces are children of the root nodes of their parents, as can be seen on the picture.

With this structure, the scenegraph can represent the entire state required for rendering and input handling in a unified manner. Next, we will take a look at the interfaces that scenegraph nodes have to facilitate these operations.

Pointer and touch input

When a pointer or touch event arrives, core needs to figure out which node should receive this event. To this purpose, node_t subclasses can implement the node_t::find_node_at() function, which returns the node in the node's subtree which should receive input focus. To facilitate implicit grabs and other transformations, nodes can implement node_t::to_local() and node_t::to_global(). These functions convert between the node's coordinate system (local) and its parent's (global). Remember that each node may apply arbitrary transformations to nodes, but the details are hidden behind the node_t interface - so scenegraph users know that there are local and global coordinate systems, but cannot know the exact mapping without knowing the exact node type. Finally, to actually handle the events, nodes may override pointer_interaction() and touch_interaction() which should return pointers to structures which can handle the input events. Often times, the nodes themselves implement pointer_interaction_t and return this in these functions, so that their own callbacks are called on pointer enter/motion/etc. when they have pointer focus.

At this point it may be worth noting which nodes typically apply a transformation to their input: output nodes apply an offset to their children so that the current workspace of each output always appears to have its top-left corner at (0, 0). Surface root nodes apply offsets so that their main surfaces are at (0, 0). Last but not least, view transformers usually apply slightly more complex transformations (for example view_2d_transformer supports arbitrary affine 2D transformations).

Note that the find_node_at() method allows nodes to cut off parts of their children by returning a null node for a set of input points. Output nodes do that to ensure a view is visible on its primary output only, except if the experimental workarounds/remove_output_limits is set. Nodes may also use this function to transfer input to another node or do other similar transformations.

Interesting detail here are drag-and-drop actions. Currently, drag-and-drop is implemented by the originating node: if the client starts DnD, the originating node keeps its implicit grab on the input as long as the user keeps their pointer button pressed. The implicit grab means that it receives all input events. However, the node internally passes the input to the node currently under the pointer, by calling find_node_at().

Rendering

Rendering in scenegraph could have been similar to input handling (and to pre-scenegraph rendering), that is, each node could in theory have a simple node_t::render(target) method. However, one of the toughest problems in rendering is damage tracking - keeping track of which regions of an output or a buffer need to be repainted. As the scenegraph tries not to assume that nodes are rendered in a particular order, or that each node may be rendered only once (imagine a drag and drop icon straddling two outputs), we need a way to track each appearance of a node. Enter scene::render_instance_t(). Render instances are specific for each node (as each node may have different transformations), so to generate render instances for a node, we can use the factory method node_t::gen_render_instances(). Then, each time we want to render the node (and its children), we need to run the so-called render pass. The documentation in scene-render.hpp contains a full description of a render pass, however, the gist of it is the following:

1. We calculate the damaged region for each render instance by calling render_instance_t::schedule_instructions(). Nodes may add or subtract damage in the process as they see fit. For example, opaque surfaces subtract their region from the damage, so that nodes below them do not repaint the region unnecessarily (and of course transformers may block this, as they are above them in the hierarchy).
2. We execute the generated instructions - this is the actual rendering step.

One thing node implementations have to keep in mind is that each node's render instance needs to generate and manage render instances for its children. Sometimes, this is as easy as calling their gen_render_instances() method without applying any transformations. However, often times render instances keep a private list of children instances, and manually call their methods and track damage coming from them.

This mechanism is enough to completely replace the view transformers found in Wayfire up to the 0.7 version. View transformer nodes are used to create various effects which modify the appearance of a view. Transformers are supposed to work in the following way:

1. By overriding to_local() and to_global(), transformer nodes may transform the input to a given surface. This is done for example by the 2D and 3D transformers and some others. (Prior to Wayfire 0.8.0, view transformers had this via the transform/untransform_point methods).
2. Nodes should always report a correct bounding box for themselves. As this bounding box depends on the exact transformer node type, nodes may override the virtual get_bounding_box() function. To get the bounding box of the children of a node without their parent's transformation applied, plugins can use the node_t::get_children_bounding_box() method.
3. When frame_done or presentation_feedback events arrive at a transformer node, it should generally forward them to its children.
4. As previously explained, rendering happens via render instances. This means that each transformer node needs to have an associated render instance class, which is then generated in the factory method node_t::gen_render_instances(). In addition, transformers render instances should generate "nested" render instances for their children and transform the damage which comes from them.
5. When a render instance is instantiated, it gets a damage callback, which it can call to report that a region (in the node's coordinate system) needs to be repainted. Transformer render instances in turn also need to pass a damage callback to their children's instances when they generate them. Usually, transformer nodes have a function which takes the damage from the children and transforms it, then reports the transformed damage region via the callback the transformer's render instance has received during instantiation.
6. Finally, transformer render instances need to do the actual rendering. Most transformers render their children to an auxiliary buffer and then render this buffer with a custom shader.

As all these operations are rather complex, but repetitive, most transformer render instances can instead derive from scene::transformer_render_instance_t. The base class does most of the managing of children instances. This base class takes care of 3. and 4. For 5., view transformer instances using this helper class only need to implement the transform_damage_region(). As for 6., they can use get_texture() before rendering to get a texture containing the children nodes.

Lastly, view transformer nodes are usually not added directly as a scenegraph nodes, but with the add_transformer and remove_transformer methods of the transformed node of each view. These functions take care of adding the transformer at the correct place in the hierarchy and support specifying ordering between the different transformers.

Keyboard input

Keyboard input also gets special treatment with the scenegraph. Namely, some nodes need to block others below them from obtaining input (plugin grabs), or may need to request a higher priority for receiving keyboard input (layer-shell exclusive input). This is done via the node_t::keyboard_refocus() method. There, each node may specify whether it wants to receive input focus and with what importance it wants input focus. Ties are resolved by keeping track of the last time each node was focused (last_focused_timestamp).