A walkthrough of bevy's rendering

[nicopap]

This is a guide exploring step-by-step how a Mesh goes from a Handle<Mesh> to pixels on screen. It might be useful to people who want to implement their own rendering on top of bevy's render graph. It started as an answer to #9036, but it answers more general questions.

1. Extract mesh into the render world: Meshes are an Asset, they are passed to the render world through their RenderAsset dark world render world counterpart. What is accessed by the render world is not a Mesh, but a GpuMesh, the transformation is done at the extraction stage.
2. Tell wgpu to render the mesh: pass.draw_indexed(…) is where the rendering takes place

bevy/crates/bevy_pbr/src/render/mesh.rs

Lines 1251 to 1269 in 9478432

	if let Some(gpu_mesh) = meshes.into_inner().get(mesh_handle) {
	pass.set_vertex_buffer(0, gpu_mesh.vertex_buffer.slice(..));
	match &gpu_mesh.buffer_info {
	GpuBufferInfo::Indexed {
	buffer,
	index_format,
	count,
	} => {
	pass.set_index_buffer(buffer.slice(..), 0, *index_format);
	pass.draw_indexed(0..*count, 0, 0..1);
	}
	GpuBufferInfo::NonIndexed => {
	pass.draw(0..gpu_mesh.vertex_count, 0..1);
	}
	}
	RenderCommandResult::Success
	} else {
	RenderCommandResult::Failure
	}

3. Create a render command with DrawMesh: We have the code, but it needs to be ran, where is it ran? Well, first we make DrawMesh part of a larger render command:

bevy/crates/bevy_pbr/src/material.rs

Lines 342 to 348 in 9478432

	type DrawMaterial<M> = (
	SetItemPipeline,
	SetMeshViewBindGroup<0>,
	SetMaterialBindGroup<M, 1>,
	SetMeshBindGroup<2>,
	DrawMesh,
	);

4. Add the command to a render phase: Bevy's rendering uses RenderGraph. A graph is made of Nodes, nodes setup "render pass" for wgpu and render a RenderPhase<I> for a specific I that implements PhaseItem. What's the relationship with DrawMaterial? Well, DrawMaterial is a RenderCommand, which is associated with a specific PhaseItem. Here we associate DrawMaterial with three different PhaseItems: Transparent3d, Opaque3d and AlphaMask3d. More on this later.

bevy/crates/bevy_pbr/src/material.rs

Lines 194 to 199 in 9478432

	render_app
	.init_resource::<DrawFunctions<Shadow>>()
	.add_render_command::<Shadow, DrawPrepass<M>>()
	.add_render_command::<Transparent3d, DrawMaterial<M>>()
	.add_render_command::<Opaque3d, DrawMaterial<M>>()
	.add_render_command::<AlphaMask3d, DrawMaterial<M>>()

add_render_command wraps the RenderCommand in a RenderCommandState and add it as a Box<dyn Draw<I>> to the DrawFunctions<I> resource:

bevy/crates/bevy_render/src/render_phase/draw.rs

Lines 293 to 305 in 0181d40

	let draw_function = RenderCommandState::<P, C>::new(&mut self.world);
	let draw_functions = self
	.world
	.get_resource::<DrawFunctions<P>>()
	.unwrap_or_else(|| {
	panic!(
	"DrawFunctions<{}> must be added to the world as a resource \
	before adding render commands to it",
	std::any::type_name::<P>(),
	);
	});
	draw_functions.write().add_with::<C, _>(draw_function);
	self

5. Create a node to access the render phase: To run any rendering code, we need to add it as node to the RenderGraph. bevy defines a set of default nodes in bevy_core_pipeline, one of which is MainOpaquePass3dNode.

bevy/crates/bevy_core_pipeline/src/core_3d/main_opaque_pass_3d_node.rs

Line 25 in 9478432

pub struct MainOpaquePass3dNode;

6. In ViewNode: create a render pass: There is a lot of shared boilerplate between different Nodes. So bevy defines a wrapper type ViewNodeRunner that does the shared boilerplate. The unique work is implemented in the ViewNode trait implementation. Generally, nodes create render passes and execute them. Render passes are a webGPU concept, I'm not quite familiar with it.

bevy/crates/bevy_core_pipeline/src/core_3d/main_opaque_pass_3d_node.rs

Lines 68 to 102 in 9478432

	let mut render_pass = render_context.begin_tracked_render_pass(RenderPassDescriptor {
	label: Some("main_opaque_pass_3d"),
	// NOTE: The opaque pass loads the color
	// buffer as well as writing to it.
	color_attachments: &[Some(target.get_color_attachment(Operations {
	load: match camera_3d.clear_color {
	ClearColorConfig::Default => {
	LoadOp::Clear(world.resource::<ClearColor>().0.into())
	}
	ClearColorConfig::Custom(color) => LoadOp::Clear(color.into()),
	ClearColorConfig::None => LoadOp::Load,
	},
	store: true,
	}))],
	depth_stencil_attachment: Some(RenderPassDepthStencilAttachment {
	view: &depth.view,
	// NOTE: The opaque main pass loads the depth buffer and possibly overwrites it
	depth_ops: Some(Operations {
	load: if depth_prepass.is_some()
	|| normal_prepass.is_some()
	|| motion_vector_prepass.is_some()
	{
	// if any prepass runs, it will generate a depth buffer so we should use it,
	// even if only the normal_prepass is used.
	Camera3dDepthLoadOp::Load
	} else {
	// NOTE: 0.0 is the far plane due to bevy's use of reverse-z projections.
	camera_3d.depth_load_op.clone()
	}
	.into(),
	store: true,
	}),
	stencil_ops: None,
	}),
	});

7. In ViewNode: call PhaseItem::render on RenderPhase: RenderPhase is a component. They are added to the render-world equivalent of cameras. Each view may have several RenderPhases of different types. RenderPhase is a collection of PhaseItems, according to the docs, each PhaseItem is an "renderable" entity that is visible from the camera, one needs to add them manually in queue systems.

bevy/crates/bevy_core_pipeline/src/core_3d/main_opaque_pass_3d_node.rs

Lines 110 to 111 in 9478432

	// Opaque draws
	opaque_phase.render(&mut render_pass, world, view_entity);

8. In PhaseItem::render, call the draw method on all the render phase items: This is complicated but we are almost at the end. Remember add_render_command in point (4)? This is where DrawFunctions<I> becomes important! We will execute the Draw<I>::draw for all items in DrawFunctions<I>:

bevy/crates/bevy_render/src/render_phase/mod.rs

Lines 77 to 91 in 9478432

	pub fn render<'w>(
	&self,
	render_pass: &mut TrackedRenderPass<'w>,
	world: &'w World,
	view: Entity,
	) {
	let draw_functions = world.resource::<DrawFunctions<I>>();
	let mut draw_functions = draw_functions.write();
	draw_functions.prepare(world);
	for item in &self.items {
	let draw_function = draw_functions.get_mut(item.draw_function()).unwrap();
	draw_function.draw(world, render_pass, view, item);
	}
	}

Now we can close the loop. Remember the code in point (2)? We asked in point (3) where is it ran. It's here! The implementation of Draw<I>::draw for RenderCommandState is nothing more than a single call to RenderCommand::render, the code in point (2)!

bevy/crates/bevy_render/src/render_phase/draw.rs

Lines 260 to 272 in 9478432

	fn draw<'w>(
	&mut self,
	world: &'w World,
	pass: &mut TrackedRenderPass<'w>,
	view: Entity,
	item: &P,
	) {
	let param = self.state.get_manual(world);
	let view = self.view.get_manual(world, view).unwrap();
	let entity = self.entity.get_manual(world, item.entity()).unwrap();
	// TODO: handle/log `RenderCommand` failure
	C::render(item, view, entity, param, pass);
	}

[IceSentry]

Thank you for writing this! I've wanted to make a blog post covering all of this ever since I started contributing to bevy's renderer.

I'd like to add a bit more details for anyone finding this.

Nodes can technically run any arbitrary code or even multiple render passes or just compute pass without any rendering. The only restriction they have is that they only have read only access to the ECS. We've talked about making Nodes just bevy systems to reduce the amount of different apis required, but there are some limitations with this approach that make this not ideal. Here's what cart had to say on this:

1. We need to be able to run the graph command encoding in parallel. The same instance of a system can only run once at a time, but we might want to run multiple instances of a graph in parallel (ex: encode two unrelated cameras at the same time).
2. We need to be able to run multiple nodes in parallel generally. The whole graph system was designed to use &World in a read-only way to enable node parallelism.

In other words, part of the design is intentional but not finished so we have to deal with a lot of complexity while not currently benefitting from the potential gains.

[superdump]

This is off the top of my head so there may be mistakes:

In my mind there are separate flows for assets and entities.

Assets like Mesh and materials are extracted from the main world when they are added/modified/removed. In the PrepareAssets set, the are transformed into their prepared form, eg GpuMesh for a Mesh asset, or a PreparedMaterial for a Material. Material preparation involves bind group creation, which is where as_bind_group() is called.

Mesh entities are extracted in extract_meshes. This involves extracting mesh transforms, flags, and the mesh handle. Separately the material handle is extracted.

Then, in the Queue set, if the assets have been prepared, they are looked up and various properties of them are used to create a specialised render pipeline. Specialisation is done based on things like the required mesh vertex attributes and their layout, flags in the material key that might impact the layout of the bind group (eg layout of material bindings), view level things like MSAA, and so on. The entity is queued to the appropriate render phase (if it’s a 2D mesh, Transparent2d; if it’s a 3D mesh, then it could be Opaque3d, AlphaMask3d, or Transparent3d, depending on the AlphaMode of the material.)

In the Sort set, the render phases are sorted into draw order by the phase item sort keys, for example back to front for transparent phases so that alpha blending works correctly.

In Prepare we now have two subsets: PrepareResources and PrepareBindGroups. In PrepareResources the new batch_and_prepare_render_phase system is run. Its job is to gather information about the phase items (entities) in draw order, prepare the per-entity data (the mesh transforms and flags), and if possible, merge the phase items to result in instanced draws of multiple mesh entities with one draw command when the draw commands are encoded later. At the end of the system, the command to write the buffer of mesh transforms to VRAM is queued. After the data has been prepared into buffers, bind groups are (re)created.

In the Render set, the render graphs are run per corresponding view. So the 3D graph is run for 3D views such as shadow mapping views or 3D cameras. The graphs are directed cyclic graphs of render graph nodes. They define ordered execution of render passes. For 3D this is broadly speaking prepasses (to render depth, normals, motion vectors, used in later passes), the main lighting passes (opaque, alpha mask, transparent), and then post processing passes (bloom, TAA, …).

If we focused on meshes, and take the simple case of not having any prepasses, the main 3D pass nodes are run. They first process the Opaque3d render phase, creating a TrackedRenderPass, and iterating through the items in the phase. For each render phase item, its DrawFunction (an ordered tuple of RenderCommands) is executed. Each RenderCommand can query for resources or components from the view and/or mesh entities that it can use to then encode the draw command by calling functions on the TrackedRenderPass to set the pipeline, bind groups, index/vertex buffers, and finally call draw, saying which vertices from the bound index/vertex buffers to draw, and what range of instances.

[superdump]

Post-processing passes like bloom and TAA can use fullscreen draws because they aren’t tied to rendering mesh entities. They don’t usually have render phases because they’re either on or off and don’t need multiple phase items with item-specific draw commands. So they’re simpler to just hard-code to do exactly what they need to do.

[superdump]

And to be clear, I think the dissection of DrawFunction and RenderCommand is very useful as a lot of people struggle to understand that abstraction.

[Atlas16A]

Ive been looking into Ray Tracing and need to implement my own, while wgpu completely lacks support for it ive found you can do it within a compute shader. What im having trouble understanding is the purpose of all this render graph stuff in regards to the pipeline. When looking at the pipeline for rasterization and how they are explained else where, i don't really understand whats going on, is the graph just a massive optimization method or something? Idk it just feels so disconnected from the pipeline structure that's described by lower level rendering apis that I'm left confused on what each thing actually does so I'm struggling to know where i can even start to replace the current rendering system with the compute shader.

[MiniaczQ]

From what I understand, graph is how you reuse parts of render pipeline, you can treat every graph node as a 'function', it takes inputs and outputs (graph edges).

[MiniaczQ]

If you want to use raytracing look into bevy-hikari, it has software raytracing

[doonv]

Wait the vertex buffer and index buffers are set for each mesh independently? I thought bevy did some advanced stuff to combine all the vertices into one buffer. But I guess that's not necessary? How efficient is this?

[ODtian]

Yes, there is a number limit of vertex to submit to gpu at once, that's why drawcall num is important, you can actually merge different mesh but also gain more cost.

Another technic is indirect draw, that is draw same mesh but with different args(transform, color etc.), you don't need to drawcall on cpu but tell gpu how many you want and let gpu to loop around.