Using renderlet with rive‐renderer
Easily add 2D vector capabilities to any 3D canvas by integrating renderlet's portable GPU infrastructure and compiler with rive-renderer's vector engine.
renderlet is a framework that makes it easy to build interactive applications. Developers can use renderlet to embed high-performance 3D graphics inside any application and deploy them on desktop, mobile, and web.
renderlet works by compiling graphics code to embeddable WebAssembly modules, enabling it to run in any cross-platform application by using our wander SDK (this repo).
rive-renderer is a GPU-based renderer for 2D vector graphics. This is similar to tools like Skia (the rendering engine behind Google Chrome), but takes advantage of more modern GPU architectures. Using tools including compute shaders and pixel local storage, rive-renderer can provide high-performance hardware-accelerated 2D graphics with an SVG-like interface.
rive-renderer is now the default rendering backend for Rive, a popular 2D graphics design tool. During GDC last week, the Rive team open-sourced rive-renderer, which made quite the splash on Hacker News: https://news.ycombinator.com/item?id=39766893
As-is, renderlet does not provide any 2D vector capabilities, however, technology enabling rasterization of 2D vectors is built on top of 3D graphics hardware APIs.
Can we use something like rive-renderer with renderlet to add 2D vector capabilities out of the box? Integrating this kind of native code by-hand with an existing rendering system is challenging, but we can let wander take care of the integration and expose the APIs through a higher level of abstraction.
This opens up new use-cases, like vector-based texture generation, 2D overlays for a UI, or even colors and gradients for 2D surfaces.
Can this be done? Absolutely! It works now on wander V1 even without support for custom shaders, and with wander V2 the integration will be even cleaner.
Let’s start from the example path_fiddle provided in rive-renderer’s repo:
V1 of wander can render any data produced by renderlet using a shader attached by the host application. This is because of an architectural challenge: although wander exposes GPU functions through a cross-platform API for vertex/index buffers, textures, and some pipeline state, building cross platform shaders programmatically is very complex.
In wander, we expose the GPU to WebAssembly by communicating over a lightweight binary wire format between the wasm guest and the host. This enables wander to efficiently upload data to the GPU irrespective of the graphics API.
We evaluated adding a new backend to rive-renderer to write to this wire format, however, it still requires generating compute shaders at runtime. Therefore, we opted to run rive-runtime as a part of wander itself outside of Wasm, giving rive-renderer direct access to its existing GPU backends. The Wasm guest then writes the actual rive-renderer commands to the wire format, instead of the raw GPU commands produced by rive-renderer’s backends:
BinaryOutStream ms;
for (const auto& command : m_command_list)
{
ms << std::get<0>(command);
ms << std::get<1>(command).size();
for (const auto& path : std::get<1>(command))
{
ms << std::get<0>(path);
ms << std::get<1>(path).size();
for (const auto& point : std::get<1>(path))
{
ms.Write(&point.x, sizeof(float));
ms.Write(&point.y, sizeof(float));
}
}
}In V2, the entire rive-runtime can become a WebAssembly module, making it even easier to distribute.
V2 of wander exposes WebGPU directly to WebAssembly, solving the shader problem. This makes two approaches possible, each leading to a cleaner architecture:
- Adding a renderlet platform backend as described above, so the entire rive-runtime library and all dependencies are compiled to WebAssembly, and the platform backend lightly wraps the WebGPU apis.
- Using the existing browser-based Wasm/WebGPU build of rive-renderer directly, and link renderlet modules to it via Emscripten. This may require polyfills through WASI for browser functions, and possibly replacing some interface code with the browser to let rive-renderer run as more of a plugin.
As V2 becomes a reality, we’ll experiment with which approach gives the best performance and requires the least upstream changes.
What does this integration look like? After adding support to our compiler to take 2D vector expressions that generate code to our wire format, and adding code to wander to use the rive-renderer backend, we’re able to easily embed this in our example app.
We’ll implement this quickly using the Windows DX11 example, as wander doesn’t fully support Metal yet, and rive-renderer requires OpenGL 4.6 which MacOS doesn’t support.
Here’s a quick demo of the same vectors in rive-renderer's path-fiddle example being used to generate a texture on the GPU that we then apply to the roof of the 3D building in our example:
Now let’s make this dynamic. With renderlet, we provide a high-level graphics specification out of the box, including procedural geometry functions. Let’s start by parameterizing the texture to take in width, height, and time from the app (with some sane defaults):
attr:
-
name: "height"
value: "1024.0"
-
name: "width"
value: "768.0"
-
name: "time"
value: "0.0"Next let’s split the texture by the x and y dimension, so the vector only renders in the top left quadrant:
rule:
-
name: "Start"
op:
-
size:
x:
value: "width"
type: "absolute"
y:
value: "height"
type: "absolute"
-
split:
axis: "x"
sizes:
-
value: "0.5"
name: "Width"
type: "absolute"
-
value: "0.5"
name: "null"
type: "absolute"
rule:
-
name: "Width"
op:
-
split:
axis: "y"
sizes:
-
value: "0.5"
name: "Geometry"
type: "absolute"
-
value: "0.5"
name: "null"
type: "absolute"Finally, let’s give it some animated vectors to render. We'll draw an ellipse where we modulate the width at a rate of cos(time / 2), and height at (time / 4). We’ll draw it twice, once as a line, and once as a fill:
rule:
-
name: "Geometry"
op:
-
ellipse:
width: "scope.x * cos(time / 2.0)"
height: "scope.y * cos(time / 4.0)"
-
paint:
style: "fill"
color: "0xFFFFFFFF"
-
paint:
style: "stroke"
color: "0x8000FFFF"
thickness: "70.0"
join: "miter"
cap: "butt"Here's the final result in our example app:
We’ve built an entire procedural 2D texturing system that can interface with wander’s 3D system by utilizing rive-renderer’s 2D vector support out of the box. Instead of having to write lots of platform specific code, we can express design intent with high-level expressions, and in integrate with our app in only a few lines of code:
auto now = std::chrono::system_clock::now();
auto f_secs = std::chrono::duration_cast<std::chrono::duration<float>>(now - then);
runtime->PushParam(renderlet_id_vector, static_cast<float>(bitmap.width));
runtime->PushParam(renderlet_id_vector, static_cast<float>(bitmap.height));
runtime->PushParam(renderlet_id_vector, f_secs.count());
tree_id_vector = runtime->Render(renderlet_id_vector, tree_id_vector);
auto tree_vector = runtime->GetRenderTree(tree_id_vector);
for (auto i = 0; i < tree->Length(); ++i)
{
auto node = tree->NodeAt(i);
// Set procedural texture
tree_vector->NodeAt(0)->RenderVector(runtime, 0, bitmap.width, bitmap.height);
node->RenderFixedStride(runtime, stride);
}We’re just getting started making the tools to build the next wave of interactive applications. We’re very excited to see how open-source technologies like the rive-renderer can bring cutting-edge graphics technologies to more apps.
Come join us! Try out the rive-renderer integration example here in wander (contributions welcome!), and reach out if you’re interested in learning more about renderlet’s tools.
