Renderer Frame Flow

Renderer Frame Flow

RendererSystem has three main runtime phases:

• validateResources
• prepareResources
• render

It also performs small ECS update work each tick, currently including material instance cache pruning.

This page describes frame sequencing. See Platform and GPU Support for hard Vulkan requirements, Windows ARM64 behavior, texture portability, tile-GPU policy, and temporal convergence constraints.

Resource Validation

validateResources(width, height, sampleCount) is called before rendering. It ensures frame-size dependent resources exist and match the active dimensions.

Validation work includes:

1. log ray tracing capability once
2. ignore zero-size frames
3. create or reuse graph-facing resources and prepared task recording state
4. create or reuse deferred frame targets
5. create the deferred compute composition pipeline and the Graphics presentation pipeline matched to the current presentation framebuffer
6. create AVBOIT pipelines
7. create mesh view buffer
8. create CSG interval peel resources

If any required resource cannot be created, validation fails and the frame should not proceed through the normal renderer path.

Invalidation

invalidateResources clears prepared state and invalidates all renderer state groups:

• mesh state
• material state
• draw state
• CSG state
• deferred state
• AVBOIT state
• ray tracing state

It also clears the current graph compilation, recorded packets, accepted-submission transaction, imported state seeds, prepared CSG frame state, and shadow-visibility readiness flag. Per-frame task-state reset itself lives in renderer_frame_pipeline_resources.cpp (resetFrameTaskState), called from the execute path.

Prepare Resources

prepareResources(framebuffer) creates per-frame renderer state before GPU task-graph declaration and native packet recording.

Main steps:

1. clear prepared shadow visibility readiness
2. reject null framebuffer
3. prune unused runtime mesh resources
4. reset prepared CSG frame state
5. skip most work if deferred targets are not valid
6. build CSG frame state when CSG candidates exist
7. prepare opaque material pass resources
8. prepare AVBOIT resources if transparent renderers exist
9. refresh opaque material pass resources after transparent preparation when transparent renderers exist
10. verify graph-facing frame resources and packet-callback prerequisites
11. prepare shadow visibility and caustic resources
12. freeze the frame-local inputs that graph tasks will retain through recording

Shadow and caustic trace resources are prepared before graph declaration so BLAS/TLAS or software-BVH tasks can consume frozen inputs and declared states during the render phase. The graph retains accepted state seeds for cross-frame imports instead of treating prepared outputs as Unknown/UNDEFINED; it only publishes a new seed after the producing packet accepts.

Material preparation also creates or reuses material pipelines, push-constant layouts, typed material storage, and mesh heap handles before draw recording starts. The render phase expects those resources to be prepared already, then uploads per-frame data and records draws against the prepared heap-slot contexts.

Prepared binding lifetime

AVBOIT preparation can grow the shared instance and typed-material buffers used by both transparency classes. A replacement buffer requires a fresh heap handle so an in-flight graph packet never observes a rewritten descriptor. Therefore, a frame with transparent renderers prepares the opaque pass once, prepares AVBOIT, then refreshes opaque heap-slot resources before any PreparedOnly opaque draw gathering occurs. New preparation work that can grow either shared buffer must preserve this ordering or refresh every affected heap-slot context before recording draws.

GPU Task Graph Execution

render(framebuffer) builds one bounded GpuTaskGraph rather than resolving a renderer-owned packet plan. Each renderer subsystem declares semantic work, resource uses, and external completions. GpuTaskScheduler::scheduleGraph (in core/task/gpu/scheduler_compilation.cpp, called once per frame from renderer_frame_pipeline_graph_schedule.cpp) then freezes queue assignment against a current physical-queue pressure snapshot and initializes the caller-owned runtime artifacts for the immutable plan: it infers hazards, chooses compatible physical queues, forms packets, and places the state/ownership barriers and timeline waits needed to execute those declarations. The renderer passes FrontierSafe packetization with a packet-timing envelope from the shadow-prepare task through the frame-timing-end task. GpuTaskScheduler records and submits the compiled packets; CPU recording work uses the shared CpuTaskScheduler. Buffer uses may declare byte ranges so disjoint intervals do not create resource hazards.

The shared deferred graph includes shadow preparation, per-frame uploads, G-buffer and CSG work, post-G-buffer normalization, AVBOIT, shadow visibility, caustics, surfel GI, deferred lighting, composite, presentation, optional history/readback tails, and recovery. Post-G-buffer geometry normalization records state preparation and declares read-only geometry inputs; it does not write geometry contents. Earlier upload/UAV producers remain dependencies, and acceptance publishes normalization readiness. The exact packet count is intentionally a compiler result: a task may merge with a compatible neighbor or split at a declared dependency, resource boundary, or explicit submission boundary.

flowchart LR
    Validate["Validate resources"]
    Prepare["Prepare ECS and<br/>renderer frame state"]
    Declare["Declare graph tasks,<br/>uses, and completions"]
    Compile["Analyze hazards; assign queues;<br/>form packets and barriers"]
    Record["Native-record ready packets"]
    Submit["Submit packets with<br/>accepted timeline waits"]
    Finish["Commit accepted state<br/>or discard/recover"]

    Validate --> Prepare --> Declare --> Compile --> Record --> Submit --> Finish

Loading

Queue routing and state

Tasks request required capabilities and a preference/fallback policy; they do not choose a renderer-wide lane. The compiler selects actual Graphics, Compute, or Transfer queues from the device registry. With no dedicated Compute or Transfer queue, compatible tasks can resolve to Graphics. With multiple physical queues, independent work can overlap only when its declared dependencies and resource contracts allow it.

The compiler owns normal packet-boundary state transitions. For an exclusive resource crossing queue families, it emits the paired ownership transfer and the submitter waits on the accepted producer timeline. Concurrently shared resources remain concurrent. A same-family movement between physical queues still needs a timeline dependency, but not a Vulkan queue-family transfer. CommandListResourceStateHandoff provides tracked resource states and queue ownership at recording boundaries; it does not define renderer topology. Compatible concurrent readers may share a proven packet-final state source without waiting for one another; source range, native-close state, and sharing requirements still apply. See GPU Task Graph.

Live deferred Lighting prepares return-state candidates for the current shadow, caustic, and surfel targets from its recorded packet-final snapshot and commits them only after native acceptance, on either Graphics or Compute. The next Compute producer imports those accepted states rather than an earlier writable snapshot. Lagged Lighting reads separate history targets and therefore leaves the current-output return caches to their actual producers or history copy.

When Lighting reads current-frame results, compatible same-queue Surfel Snapshot and GI stages may join the preceding effect packet through their existing explicit successor dependencies. Their shadow and GI timestamp boundaries remain separate. Lighting with lagged history retains the independent effect publication boundaries. Composite may share its compatible serial predecessor. Independent hardware Caustics retains its sibling dependencies; software Caustics retains its existing role as the effects predecessor. Queue and frontier checks remain compiler-owned.

Recording, submission, and recovery

Native command-list recording remains the normal runtime path. Compatible independent packets may opt into parallel ready-frontier recording. Recording freezes the graph declaration for the exact compiled plan and publishes each retained command list only after native close. Ready packet selection follows the deterministic compiled plan, while ordinary packets for independent native queues may cross the native submit boundary concurrently. The transaction serializes only the irreversible CPU publication tail—timing resolution, lifecycle callbacks, completion tokens, and accepted-frontier updates—so a packet with a pre-submit hook or an accepted-frontier join takes an exclusive boundary. Packet acceptance is transactional: accepted callbacks publish state and completion tokens; a rejected packet cannot become a dependency. If an observer throws after native acceptance, the exact token is committed before rethrow and every unresolved packet/timing ticket is terminalized callback-free. A graph-owned recovery tail can wait on the latest accepted work on other physical queues before resources are reused. Its conditional work contributes zero estimated normal overlap and queue load. Failure to establish that recovery requests device recreation instead of guessing state or completion.

Frame-lagged async lighting

RendererSystem::setFrameLaggedAsyncLightingEnabled(true) remains opt-in. When the graph has accepted lagged-history targets, lighting can consume the prior shadow/caustic/surfel snapshot while the current frame declares work that produces the next snapshot. The history copy is a graph tail whose completion becomes the next frame's external dependency. This route also works on a graphics-only topology: the small history-slot upload and deferred lighting stay in the selected Graphics packet when no Compute transport exists; a dedicated Compute route can still expose overlap when the topology permits it.

After enable, target recreation, an invalid history, or a rejected history packet, the renderer stays on the current-frame path until a new history snapshot is accepted. The graph never substitutes a missing completion token, so lagged lighting cannot consume an unaccepted or stale snapshot.

Draw Item Partitions

Opaque draw items are partitioned into:

• regular draws
• CSG draws
• CSG receiver surface draws

This lets the renderer run interval work only for receivers and cutters that participate in the CSG frame state.

Scene Buffers

The renderer updates:

• mesh view buffer
• scene shading buffer
• instance GPU data
• material typed bytes
• CSG frame data
• shadow visibility resources
• caustic accumulator, scratch, and irradiance resources
• surfel pool, live-only guide pool (four float4 lanes per surfel, never snapshotted), world-hash, counters, snapshots (core pool + cell heads only), indirect trace arguments, half-resolution irradiance, and resolved full-resolution irradiance

Camera data comes from ActiveCameraComponent, CameraComponent, and TransformComponent. Light data comes from LightComponent plus scene transform data.

Selected BLAS/TLAS and software-BVH graph preparation consume frozen mesh and scene inputs with exact imported resources. Missing required identities fail preparation instead of regathering native geometry during recording. Caustic preparation requires its mesh-view input. Hardware/software feature selection and supported resource aliasing remain part of the current rendering contract.

Shadow Visibility

The renderer has a full-resolution shadow visibility target consumed by deferred lighting.

The graph declares shadow work after its required G-buffer and trace-preparation dependencies. Shadow callbacks require graph-declared outer inputs; stage-internal transitions still sequence the selected dispatches. Its task requests compatible compute capability, so the compiler can place it on Graphics when necessary or on a dedicated Compute queue when available. The graph preserves the sharing contract of common inputs; shadowVisibility and soft-shadow scratch/history retain their declared ownership/state through their later lighting consumer and next-frame reuse.

During render:

• hardware RayQuery writes opaque visibility through the shared half-resolution soft resolve when soft resources are ready; if not, it falls back to one direct full-resolution opaque visibility sample per shadow-slot light
• when RT hardware is active and transparent occluders exist, the staged hardware gather/evaluate/overflow-continuation path traces a colored half-resolution signal, the RGB resolve filters it into HalfA while the opaque wavelet output stays in HalfB, and one combined upsample resolves both with a shared float4 guided fit; if that fold is unavailable, software GPU BVH traversal multiplies colored transparent transmittance onto the hardware opaque mask
• when the hardware RayQuery shadow path is unavailable, software GPU BVH traversal writes a full-resolution hard opaque pre-pass when software resources are ready, or the light-space depth path resolves visibility from the D32 view array when its snapshot is ready; the depth path supports Reference/FittedVolume coverage, ReferenceGrid9/CompactCross5/Center1 blocker search, and EveryFrame/ReuseOneFrame/ReuseTwoFrames capture cadence (see Renderer Feature Paths)
• when software soft-shadow resources are ready, the software path overwrites active shadow slots through the same half-resolution cone-jittered trace, optional temporal reproject-merge, one-pass scalar a-trous denoise, and bilateral upsample chain (resolve pipelines are specialized per filtering stage: scalar/RGB wavelet plus upsample, with a combined-upsample variant)
• when the soft transparent path is ready, a separate colored half-resolution transparent signal is RGB-denoised into HalfA and combined with the retained opaque HalfB output in one upsample; the combined pipelines are prepared on both hardware and software frames, so the software fold is the same fusion. Otherwise transparent shadows use the adaptive transparent fallback fold
• otherwise the visibility image is cleared to all-lit white

Deferred lighting always samples the visibility image, so it must never contain stale data.

Caustic Irradiance

The renderer has one caustic irradiance target consumed by deferred lighting as additive, scene-referred light.

Caustic tasks retain the prepared mesh-view snapshot. The graph declares accumulator reset/bootstrap/decay, the selected software-BVH or hardware dispatchRays photon producer, geometry downsample, resolve preparation, five wavelet stages, and upsample with their exact resource states. Accepted producer callbacks publish the accumulator lifecycle; a frame without photon work retains the typed black irradiance clear. On suitable hardware, independent hardware-caustic and Compute shadow/surfel work can occupy different physical queues; where a dedicated queue is unavailable, compatible work routes to Graphics. The compiler supplies the required resource/timeline handoff to lighting, while caustic state remains graph-owned across accumulator, resolve/history, geometry cache, and full-resolution causticIrradiance.

During render:

• hardware ray tracing writes caustic photons when acceleration structure support is active and caustic resources are ready
• software GPU BVH traversal writes caustic photons when hardware ray tracing is unavailable but software resources are ready
• both caustic producers emit only one checkerboard half of the full photon grid per frame and alternate frameIndex phases (resolve uses separate prepare/wavelet/upsample pipelines with constant stage variants, plus a direct-texture wavelet for dilations 8 and 16); CausticQualitySettings::photonGridDivisor (1, 2, or 4) scales the grid side first, and emission-target preparation drops opaque instances before resource resolution
• otherwise the irradiance image remains black after the per-frame clear

Both producers write a shared RGB integer accumulator. It uses the renderer's internal 0.85 temporal decay and persists as splat-space history: the first active frame or a resized target clears it, then later accepted frames decay it in place before each new photon splat. The producers halve the per-frame photon work by selecting alternating checkerboard parities of the full grid; per-photon flux is normalized so the two-frame union preserves steady-state brightness. The common resolve path first downsamples receiver world/depth state into a half-resolution geometry cache, then converts accumulator flux into half-resolution irradiance, runs five spatial a-trous wavelet passes through half-resolution ping-pong targets, and edge-aware upsamples into the full-resolution irradiance target. Deferred lighting samples the result before tone mapping and multiplies it by receiver albedo.

Surfel GI

The graph declares the complete surfel sequence—clear, age/free, hash build, spawn, trace-argument build, trace, resolve, and upsample—with the selected backend. Active GI requires complete prepared resources for all seven typed stages after initialization; unavailable preparation fails graph construction. Inactive GI retains its initialization lifecycle and no-work terminal endpoint. There is no monolithic recording fallback. Its hardware path is an inline-RayQuery compute shader, not dispatchRays. The compiler selects a compatible physical queue and preserves the resolved indirect irradiance state through deferred lighting.

During render:

• live surfels unseen for the configured age are recycled into a persistent free list
• the world-hash cell-head table is cleared and rebuilt from live surfels
• screen tiles sample the G-buffer and spawn surfels into empty hash buckets
• a one-thread build-args pass writes indirect trace dispatch dimensions from the live bump counter and update divisor
• the trace pass uses a 64-thread workgroup and traces up to 64 rays for a seeding surfel, then 12 fresh rays after its history is established (4 uniform + up to 8 guided), through the software scene BVH or hardware inline RayQuery backend
• the resolve pass gathers the surfel hash into a half-resolution irradiance target
• the upsample pass reconstructs full-resolution surfelIrradiance with G-buffer normal and world-position guidance

The persistent surfel field, snapshots, indirect arguments, first-use clear, count-readback staging, and full-resolution surfelIrradiance retain their declared graph states across frames. Deferred lighting samples the resolved texture through the compiler-planned consumer handoff; alpha zero falls back to hemi-ambient, so disabled GI does not inject stale light.

Surfel GI is progressive. The bootstrap frame updates every selected surfel; steady state updates one quarter per frame and retains a bounded running mean. Stable scenes improve over time, but motion and surfel recycling continue to change the result, and no fixed frame count guarantees convergence. See Platform and GPU Support.

Reflection Radiance

DeclareReflectionTasks owns one graph domain that classifies eligible opaque and glass pixels, builds and reduces the min/max visible-depth pyramid, dispatches screen tracing and the bounded hardware continuation through indirect arguments, applies temporal accumulation and separate spatial filtering, and publishes opaque and glass radiance targets. The temporal task returns before render-pass termination, binding, push constants, timing, or dispatch when history cannot update (non-reused history or temporalMaxSamples of 1). The spatial kernel skips halo/position/normal loads for uniformly ineligible tiles via a workgroup-wide eligibility flag. Unchanged accepted scenes reuse the optical metadata upload with zero upload tasks and zero graph upload blobs; ReflectionFrameSnapshot::hasHardwareWork() keeps zero-budget frames from declaring a hardware consumer. Hardware reads import the frozen scene, material, and lighting declarations from the shared trace sets; when hardware preparation is not ready the domain dispatches its prepared fallback pipeline, logs the substitution once, and keeps the declared resource set conservative.

Resource handles and descriptor slots are copied into a frame snapshot before declaration, so only parameter values vary per frame and no recording callback consults live resource owners. The domain imports feedback state, frozen selectors, and statistics readback; accepted packets publish radiance, feedback, and completed-frame counters, while discarded packets leave prior history untouched. See Reflections.

Refraction Resolve

When transparent renderers and refraction or reflection work exist, the AVBOIT domain declares the primary-refractor capture after opaque visibility and transparent CSG intervals are ready. The resolve task declares reads on captured identity/depth/normal/IOR, opaque color, reflection radiance, world position, camera depth, AVBOIT accumulations, and the foreground attachments, then writes the resolved transmission target. On ray-query hardware it shares the scene TLAS and material trace sets; otherwise it uses the prepared screen-space fallback. Its dependencies join reflection completion, the AVBOIT final task, and surfel GI, so glass pixels receive transmitted reflection and GI without unordered reads. See CSG and Transparency.

Deferred Lighting and Composite

After material, CSG, shadow, caustic, and surfel GI work, deferred lighting reads scene buffers, G-buffer/deferred targets, shadow visibility, caustic irradiance, and surfel indirect irradiance.

The composite pass combines opaque output, reflection radiance, AVBOIT accumulation, and the refraction resolve into linear compositeColor. The graph places deferred lighting, composite, and the terminal presentation task on compatible physical queues while retaining their declared resource and completion dependencies. Independent UI raster work may run separately from scene work; the terminal Graphics pass joins the optional premultiplied display-linear UI layer after scene mapping. SDR blends in linear Rec.709 before the presentation attachment's sRGB conversion; HDR10 blends scene and UI in linear Rec.2020 nits before one PQ encoding into the 10-bit surface. The optional lagged-lighting path instead consumes prior accepted ray-effect history; see Frame-lagged async lighting and HDR10 Presentation.

Once the frame's GPU work is recorded and submitted, device garbage collection resolves the global bindless descriptor heap's retired slots with device.getDescriptorHeap().collectRetired() on the live Device&. A slot freed during scene mutation returns to the free list only after every earlier command-buffer heap use has completed on its queue timeline or been discarded. See Bindless Descriptor Heap.

GPU Timing and Telemetry

Renderer command recording includes GPU timing scopes such as frame, opaque regular, opaque CSG receiver surface, opaque CSG, shadow visibility, caustic photons, caustic resolve, surfel spawn, surfel age-free, surfel hash-build, surfel trace, surfel resolve, surfel upsample, deferred lighting, and AVBOIT passes. RendererSystem reserves query storage for the common scope set during resource validation so first-use timing does not depend on render-pass allocation. Shadow visibility timing covers the selected hardware/software shadow work, including the staged hardware transparent gather/evaluate/continuation passes or the software transparent multiply fallback when they run.

Every timed compiled packet receives a timing ticket. Packet identity is the timing key, so queue assignment and compatible packet merging cannot leave a separate renderer timing-routing table behind; optional history/readback tails can remain execution-only. The render.frame transaction begins with the graph's frame work and is published only when the terminal presentation packet accepts, so it measures the critical path rather than summing concurrent queue durations. Where queue timestamp domains support comparison, render.deferred_graph.queue_overlap measures overlap among compiled timed packets. Packet merging and queue placement determine which intervals can overlap.

If a later frame packet is rejected after the prefix was accepted, a non-publishing frame-recovery submission retires the begun frame query instead of leaking it or publishing a partial frame. If recovery cannot be accepted, the renderer requests device recreation. RendererSystem also implements IFrameGraphContributor, so frame graph capture can include renderer pass information when telemetry capture is enabled.