Rendering Layers

A rendering layer is one of the biggest abstractions in Opengine. It is also still under development, so many of the fine details in this document will be suspect. The high level concept is pretty solid, though.

The core data element of rendering layers is the ImageSet. An ImageSet is a logical image at some stage of rendering. Rendering layers work by moving an ImageSet from one state to another, sometimes making an ImageStateState or combining mutliple ImageSet into a single.

First: VulkanWindow

A VulkanWindow represents a window as displayed to a user. Whenever want to eventually render to a VulkanWindow, you need to start with an ImageSet that is designed for that window, so we need to start by asking the window for an ImageSet that is custom tailored.

ImageSet startingImageSet = vulkanWindow.getFramebuffer();

Next: Layers

Next we want to use rendering layers to alter that image to get closer to what we ultimately want.

ImageSet rendered = modelRenderer.renderModels(startingImageSet, models);

Sidenote: Logical vs Actual Images

In this example above, it would be a waste to create more textures in GPU memory just to have GPU textures for each ImageSet. An ImageSet in Java is not necessarily backed by a texture in GPU memory. But they could be, if Opengine determines it's needed. Opengine uses two heuristics to determine when a texture needs to be realized as actual GPU memory: first, a rendering layer can explicitly note that it needs to actualize texture memory; secondly, if an image is used for two different inputs it will probably create two copies of it.

Example: Split Layers

For stylistic reasons, we could render a model in a blurry manner, and then render it non-blurry on top. That should look pretty cool.

ImageSet blurry = blurrer.blur(rendered);
ImageSet overlay = overlay.overlay(blurry, rendered);

To create overlay, it might be more efficient to spawn more GPU memory and then actually overlay. [In this case, it likely wouldn't be, but we're going for simple examples.] Opengine can decide (using whatever heuristics it has, you don't need to know about those) whether to actualize the extra memory or not. You are free to just think about the rendering layers and how you would logically composite the effects and leave Opengine to figure out how to optimize it.

Debugger: Rendering Nodes

The plan is to create a debugger that draws the node graph as well as what choices were made (and why) to a secondary debug window.

ImageSet secondWindow = secondVulkanWindow.getFramebuffer();
secondWindow = debugger.renderNodeGraph(secondWindow, overlay);

Last: Rendering

Finally, we send an ImageSet back to the VulkanWindow. The VulkanWindow handles all the annoyances of double/triple buffering.

vulkanWindow.setRender(overlay);
secondVulkanWindow.setRender(secondWindow);

With all of our above examples, we would have a cool stylistic half-blurry render of a model on one screen and simultaneously a debug view in a second window.

Bonus: Compute

I also want to natively support compute shader invocation and interaction. The specifics of that interaction are still in planning. In general though, there will be specific types of data buffers (including a generic catch-all) that can be passed into layers. Those layers can output the same data buffer, different type(s) of data buffer, or images. Those images can then be rendered as normal.

One type of compute that I definitely want to support are layers for skeletal animation and inverse kinematics, but as with everything else, I am still figuring out how that API looks.