New Renderer Proposal

[aarthificial]

The current renderer - based on Konva - has limitations and problems that are making it increasingly harder to maintain and extend.
A lot of them cannot be fixed without heavily modifying the Konva library itself.
What follows is a proposal for a brand new renderer that would be built from the ground up with Motion Canvas in mind.

General Idea

Here's a sample scene utilizing the new renderer:

view.add(
  <Rect yoga fill="#242424" radius={8}>
    <Rect
      layout={{ padding: 20, alignItems: "stretch", height: 80 }}
      fill="#c0b3a3"
    >
      <Icon
        fill="rgba(0, 0, 0, 0.87)"
        layout={{ marginRight: 20 }}
        type="object"
      />
      <Text fill="rgba(0, 0, 0, 0.87)">Object</Text>
    </Rect>
    <Layout paddingTop={10} paddingBottom={10} direction="column">
      <Layout padding={10} height={60}>
        <Icon fill="#242424" layout={{ marginRight: 20 }} type="property" />
        <Text fill="rgba(255, 255, 255, 0.54)">field</Text>
      </Layout>
      <Layout padding={10} height={60}>
        <Icon fill="#242424" layout={{ marginRight: 20 }} type="event" />
        <Text fill="rgba(255, 255, 255, 0.54)">event</Text>
      </Layout>
    </Layout>
  </Rect>
);

The result is the following object node:

The above example is made out of primitives, but if the element is meant to be used often, it can be simplified by creating two custom components:

view.add(
  <ObjectContainer title="Object">
    <ObjectProperty type="property">field</ObjectProperty>
    <ObjectProperty type="event">event</ObjectProperty>
  </ObjectContainer>
);

Hierarchy

Just like with Konva, the scene is represented by a document object model.
Created elements are organized in a tree hierarchy.
The main difference is that any node can have children.
This is why we can use primitives such as Rect as containers:

<Rect yoga fill="#242424" radius={8}>
  // children here
</Rect>

Creation

When using a JSX tag, the runtime will create an instance of the adequate class.
All properties defined on the tag are passed to the constructor as a single object.
This includes any possible children, which is a major difference compared to the old renderer.
This allows us to control how we want to treat the child nodes and what kind of nodes we can accept.

For instance, the Text primitive only accepts a single string node and treats it as its value.
The following are interchangeable:

<Text>My Custom Text</Text>
<Text value="My Custom Text" />

Placement

By default, nodes are placed using the following attributes:

• x - position along the x-axis
• y - position along the y-axis
• rotation - rotation along the z-axis
• scaleX - scale along the x-axis
• scaleY - scale along the y-axis

These attributes are relative to the parent node. And work the same way as in Konva.
The position and scale attributes can be used as shortcuts.

Additionally, it's possible to define the origin of the node using the originX and originY attributes.
Here, the main difference is that origin is relative to the size of the node.
For instance, (1, 1) would mean the upper right corner of the node.
(-1, 0) would be the middle left. etc.
origin attribute can be used together with the Origin type to easily configure it:

<Node origin={Origin.BottomRight}>
// same as:
<Node originX={1} originY={-1}>

Layout

The biggest advancement is the use of the Yoga layout engine.
Any node can be marked with a dedicated yoga attribute to delegate control over its size and position to Yoga:

<Rect yoga>{/* ... */}</Rect>
// same as:
<Rect yoga={true}>{/* ... */}</Rect>

These attributes cascade down, making all children of the node controlled by Yoga as well.
Children can opt-out of this by explicitly setting the yoga attribute to false.

When in control, Yoga will override the size and position attributes.
This means that disabling Yoga in the middle of the animation will keep the last calculated size and position.
The following values provide more gradual control over Yoga:

• yoga="content" - only the size of the node is controlled by Yoga, position can still be manually adjusted.
• yoga="pop" - the node is no longer controlled by Yoga, but the parent layout still respects it. There's a "hole" left in the parent layout as if the element was still there. Useful when we want to temporarily "pop" a node out of its parent.

All attributes related to the Yoga layout can be adjusted through the layout attribute:

<Rect layout={{ padding: 20, alignItems: "stretch", height: 80 }}>
  {/* ... */}
</Rect>

When all we need is a simple container that doesn't render anything, a Layout node can be used for convenience.
It acts as a basic node but allows us to set the layout attributes as individual JSX properties:

<Layout padding={20} alignItems="stretch" height={80} />
// same as:
<Node layout={{ padding: 20, alignItems: "stretch", height: 80 }} />

Properties

The approach to properties doesn't change much compared to the current version.
Each of them acts as a getter/setter/animator depending on the number of attributes passed:

const fill = rect.fill();             // get current fill
             rect.fill("#242424");    // set current fill
yield *      rect.fill("#242424", 2); // animate current fill over 2 seconds

As a new addition, a function can be passed as the first argument to make the value dynamic:

// make the fill of the `rect` dependent on the stroke of `anotherRect`:
rect.fill(() => anotherRect.stroke());

anotherRect.stroke("red");
console.log(rect.fill()); // logs: "red"

A more detailed discussion about this feature can be found in #58

Style Settings

The Style interface is the universal type used to configure how elements are rendered.
It contains all properties necessary to configure a canvas:

interface Style {
  fill?: string;
  stroke?: string | CanvasGradient;
  strokeWidth?: number;
  lineJoin?: LineJoin;
  lineCap?: LineCap;
  dash?: number[];
  dashOffset?: number;
  shadowColor?: string;
  // ...
}

In the case of most primitives, such as Rect, these properties can be configured through JSX:

<Rect fill="red" strokeWidth={2} stroke="white" />

But more complex components can use it to make their style customizable.
For example:

<ColorPicker
  // `contianer` is a property of type `Style` 
  // that defines how the container will be rendered.
  container={{ strokeWidth: 2, stroke: "white" }} 
  
  // Similarly, `text` is responsible for 
  // rendering the text inside.
  text={{ fill: 'rgba(0, 0, 0, 0.87)' }} 
/>

Under the hood, a helper function is used to apply a style to the context:

applyStyle(context, style);

This function can be used in custom components to make them easily customizable.

Type System

The new renderer continues to use duck typing for all of its basic types.
That is, types such as vectors, rects, and sizes are not instances of specific classes.
The shape of the object defines its type, so any object matching the following interface:

interface Vector2 {
  x: number;
  y: number
}

is considered a valid two-dimensional vector.

To make defining and modifying basic types easier, a set of helper functions will be provided.
For instance:

vec2(2, 1) // creates {x: 2, y: 1}
vec2.one   // creates {x: 1, y: 1}
vec2.zero  // creates {x: 0, y: 0}
vec2.left  // creates {x: -1, y: 0}

vec2.add(vec2(1, 2), vec2.up)        // adds two vectors
vec2.magnitude(vec2(-2, 7))          // calcualtes the magnitude
vec2.lerp(vec2.left, vec2.top, 0.2); // interpolates lineary between two vectors
                                     // `vec2.lerp` is also a valid `TweenFunction<Vector2>`

size(10, 20)     // creates {width: 10, height: 20}
rect(0, 0, 2, 2) // creates {x: 0, y: 0, width: 10, height: 20}
rect(vec2.zero, size.one);
rect.expand(rect(0, 0, 10, 20), 10)

// and more

Custom components

Just like in the old renderer, custom components can be created by extending an existing Node class:
(Example of a class component in the old renderer.)

interface SurfaceProps extends NodeProps {
  background: string;
}

class Surface extends Node {
  public constructor({ background, ...rest }) {
    super(rest);
    this.add(<Rect fill={background} />);
  }
  public render(context, canvas) {
    // do some custom rendering
    super.render(context, canvas)
  }
}

Or by creating a function component:

interface SurfaceProps extends NodeProps {
  background: string;
}

function Surface({ background, ...rest }: SurfaceProps) {
  return (
    <Node {...rest}>
      <Rect fill={background} />
    </Node>
  );
}

Unlike in React, function components are stateless.
(Example of a function component in the old renderer.)

Additional features

I omitted certain technical features, such as hit detection, events, and querying elements.
These are rather easy to implement and can be added later on, once the general shape of the renderer has been established.

---

Thanks for reading!
This is just a proposal and everything is subject to change, so feel free to discuss anything below.

[Magnogen]

This looks incredibly helpful, will users be able to define custom jsx elements, similar to how React defines elements? Or will you create node types with a different syntax?

I.e. With classes that have a .render() attribute, that the user can then define to return a bunch of primitives

[aarthificial]

I added a section about custom components

[Magnogen]

Oh yes of course you have, I think I might have glossed over last night.

Implementation makes sense, 👍

[aarthificial]

No no, it wasn't there before. I added it as a response to your comment

[Ross-Esmond]

The current plan for computed properties is to run the function on every read, right? I think that's where things wound up. Out of curiosity, what is the plan for conjoined properties? You mentioned them before.

circ.origin(() => compOrigin());
\\ or
circ.originX(() => compX());

Is the plan for origin, the conjoined property, to sets originX and originY internally, or the other way around?

// SUPER simplified code
origin(arg: {x: number, y: number}) {
  this.originX(x);
  this.originY(y);
}

Is your plan to keep refs the same?

My gut reflex is to use arrays for things like vectors, [5, 10] as opposed to {x: 5, y: 10}. I know that in the past you wanted to be able to detect the difference between a size and a vec2, but now that deepTween is the core interpolation, is there a particular reason to use objects? If it's just your preference that's totally fine. I wonder what others would think. Any vec2 function would be able to work on both sizes as well, since size is basically just a vector. Like a vec2.scale([x, y]).

EDIT: Will arrows and pins be done with computed properties?

EDITx2: Rotations are in degrees, right?

EDITx3: How many shapes do you intend to implement before release? Circle and Rectangle I'm assuming, but Konva had some weird ones.

[aarthificial]

The current plan for computed properties is to run the function on every read, right?

No, we still need a Solid-like dependency tracking. It's necessary to cache things like matrices and to update the Yoga layout only when something changes. The continuous time idea is the only thing I'm still not sure about. And I think we should come back to it once the first prototype of the renderer is ready.

what is the plan for conjoined properties?

// this:
rect.position(vec2(4, 5));
// would do:
rect.x(4).y(5);

// and this:
rect.position(() => circ.position());
// would do:
const fn = () => circ.position();
rect.x(() => fn().x).y(() => fn().y);

Is your plan to keep refs the same?

Yes

My gut reflex is to use arrays for things like vectors

The code would be really difficult to read, though.
Especially when it comes to rects, where width * height would be written as rect[2] * rect[3]
You also lose the ability to do destructuring:

const rect = rect.expand(rect(somePosition, someSize), 10);
view.add(
  <Rect {...rect} />
  <Circle {...somePosition} />
);

EDIT: Will arrows and pins be done with computed properties?

It's difficult to say, we'll find out.

EDITx2: Rotations are in degrees, right?

Yes

EDITx3: How many shapes do you intend to implement before release?

The planned primitives are:
Rect, Circle (Ellipse, Wedge), Line (Arrow), Text, Image

[Ross-Esmond]

That all sounds great.

I hadn't thought about prop destructuring.

[Ross-Esmond]

Do you want any help? I still have work to do on my code highlighter, but it doesn't look like I should bother building a Konva component out of it.

[aarthificial]

Thanks, but it's a bit too early for splitting the work. I'd like to at least get the initial version out first

[Ross-Esmond]

That's what I assumed. 👍