Implement lightmaps. #10231

pcwalton · 2023-10-23T00:24:54Z

Objective

Lightmaps, textures that store baked global illumination, have been a mainstay of real-time graphics for decades. Bevy currently has no support for them, so this pull request implements them.

Solution

The new Lightmap component can be attached to any entity that contains a Handle<Mesh> and a StandardMaterial. When present, it will be applied in the PBR shader. Because multiple lightmaps are frequently packed into atlases, each lightmap may have its own UV boundaries within its texture. An exposure field is also provided, to control the brightness of the lightmap.

Note that this PR doesn't provide any way to bake the lightmaps. That can be done with The Lightmapper or another solution, such as Unity's Bakery.

Changelog

Added

A new component, Lightmap, is available, for baked global illumination. If your mesh has a second UV channel (UV1), and you attach this component to the entity with that mesh, Bevy will apply the texture referenced in the lightmap.

TODO: Explain this better. This pull request introduces a new render set, `PrepareBatches`, makes it run after `PrepareResources`, and moves all the batching logic into that set. It's needed because the lightmap indices must be assigned before the batches are built, because the `Mesh` data structure contains the lightmap index. Additionally, this PR makes `MeshBindGroups` generic over *mesh layout keys*. A mesh layout key is a set of flags that determines the type of bind group that the mesh needs. Prior to this PR, all the combinations of bind groups were hard-coded: `model_only`, `skinned`, `morphed`, and `morphed_skinned`. Because this PR introduces `lightmapped` as a new type of key, and that doubles the number of bind groups, to aid maintainability all of the mesh layout keys `SKINNED`, `MORPHED`, and `LIGHTMAPPED` are now flags. These flags are dealt with generically, to avoid having to write out combinations explicitly. Bind groups are further subdivided into *generic* bind groups, which can be shared among all meshes, and *mesh-specific* bind groups, which are specific to a mesh. All combinations of bind groups are generated up front.

alice-i-cecile · 2023-10-23T00:29:14Z

This pull request introduces a new render set, PrepareBatches, makes it run after PrepareResources, and moves all the batching logic into that set. It's needed because the lightmap indices must be assigned before the batches are built, because the Mesh data structure contains the lightmap index.

Can we split this into a second PR? It'll make this one easier to review, but more importantly, we should be able to merge it quickly and avoid merge conflicts :)

Additionally, this PR makes MeshBindGroups generic over mesh layout keys. A mesh layout key is a set of flags that determines the type of bind group that the mesh needs.

Same comment about this actually :)

github-actions · 2023-10-23T00:45:24Z

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

PR bevyengine#10231 needs to add some logic that runs as part of `PrepareResources` but before batch building, because it creates data that batch building needs. This commit adds a new render set for this. It was split out of bevyengine#10231 at reviewer request.

Currently, Bevy hardcodes the bind group layouts and bind groups for every combination of mesh features: `model_only`, `skinned`, `morphed`, and `morphed_skinned`. This causes a combinatorial explosion when new mesh features are added, as PR bevyengine#10231 does for lightmaps. To solve this, PR introduces `MeshLayoutKey`, which is a set of flags that define which mesh features are enabled. All possible combinations of bind group layouts and bind groups are generated generically, so new mesh features can be easily added in the future. To avoid a runtime combinatorial explosion, the following optimizations have been added: * Some flags are marked *generic*, which means that bind groups with them only have to be generated once and then can be reused across all meshes. * `SKINNED` and `MORPHED` bind groups are never generated if the scene doesn't contain any skinned or morphed meshes, respectively. * `SKINNED` and `MORPHED` bind groups are only generated for meshes that actually have skins or morph targets, respectively.

We now support two sets of UVs, so `uv` and `VERTEX_UVS` in shaders are now ambiguous. PR bevyengine#10231 needs to use the second UV set. To solve this, we explicitly mark the UV set we're using in the shaders, so as to distinguish it from the second UV set used for lightmaps in PR The obvious question is "why use `UV_A` and `UV_B` instead of UV0 and UV1?" The answer is that Naga doesn't currently allow preprocessor definitions that end in numbers. To work around this restriction, I renamed UV0 to `UV_A` and UV1 to `UV_B`.

pcwalton · 2023-10-23T04:55:05Z

I've split parts of this PR out into three separate PRs: #10234, #10235, and #10237. Those should land before this does.

pcwalton · 2023-10-23T09:39:03Z

I pushed a new commit that gets rid of MeshLayoutKey and builds on top of Bevy's current system instead. So #10235 is no longer a dependency.

github-actions · 2023-10-23T09:59:19Z

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

Adriaaaaan · 2023-10-23T16:07:07Z

How easy would this be to extend to other UV maps and to potentially use different shaders? I ask this because I use lightmaps generated by unreal and they currently go up to at least UV4 and a custom shader is required to decode them

pcwalton · 2023-10-23T19:23:41Z

@Adriaaaaan You probably want to look at https://github.com/pcwalton/bevy-baked-gi which shows how to implement lightmaps without modifying Bevy itself. Note that it's targeted toward 0.12 and doesn't have ExtendedMaterial; with ExtendedMaterial it'd be a lot simpler.

pcwalton · 2023-10-24T00:22:05Z

Updated to remove the renaming of uv to uv_a per consensus on Discord. This means that #10237 is no longer a dependency.

github-actions · 2023-10-24T00:40:18Z

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

crates/bevy_pbr/src/render/lightmaps.rs

github-actions · 2023-11-24T01:28:11Z

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

pcwalton · 2023-12-29T06:58:34Z

The lightmaps in the example appear to be generated without sufficient padding, resulting in aliasing artifacts. It's probably worthwhile to try to have properly generated lightmaps since they are adding around 2.3MB to the repo.

This should be fixed now.

pcwalton · 2023-12-29T06:58:53Z

This currently crashes when used with any scene that also uses a depth prepass (or any prepass).

Caused by:
    In a RenderPass
      note: encoder = `<CommandBuffer-(0, 4, Vulkan)>`
    In a draw command, indexed:true indirect:false
      note: render pipeline = `pbr_prepass_pipeline`
    The pipeline layout, associated with the current render pipeline, contains a bind group layout at index 1 which is incompatible with the bind group layout associated with the bind group at 1
    ```

Fixed.

This reverts commit ac22e82.

pcwalton · 2023-12-29T07:07:31Z

Looks like the Clippy failure in CI is truly unrelated, as fixing it led to another clearly-unrelated Clippy failure. I'm going to just revert that attempt to fix it; another PR will need to fix the Clippy issues, as bundling them into this one will just be confusing.

DGriffin91 · 2023-12-29T07:10:14Z

I think we should include vertex uv b in the prepass as well.

At some point we should probably consolidate the code a bit between the prepass and forward specialization so they don't diverge.

pcwalton · 2023-12-29T07:21:15Z

I think we should include vertex uv b in the prepass as well.

At some point we should probably consolidate the code a bit between the prepass and forward specialization so they don't diverge.

Done.

pcwalton · 2023-12-29T07:26:00Z

This should be ready to go, if nothing else comes up--the CI failure is just unrelated Clippy stuff.

DGriffin91 · 2023-12-29T07:32:53Z

~~On native vulkan/dx12 the lightmaps are now working when the depth prepass is present. But on webgl2 I get this error in both firefox and chrome: Tex storage 2D multisample is not supported.~~

~~I'm also getting this with the 3d_scene example if I add the depth prepass. Now going to check bevy main...~~

Update: I'm also getting this error on bevy main. I'll create an issue.

I forgot that MSAA isn't supported when using the prepass on webgl2. No issue here with bevy main or this PR.

alice-i-cecile · 2024-01-01T16:39:00Z

@pcwalton if this doesn't go through, can you please merge main in? We're struggling with CI difficulties.

This was missed in bevyengine#10231. Closes bevyengine#11190.

This was missed in #10231. Fixes #11190.

# Objective Provide a public replacement for `Into<MeshUniform>` trait impl which was removed by #10231. I made use of this in the `bevy_mod_outline` crate and will have to duplicate this function if it's not accessible. ## Solution Change the MeshUniform::new() method to be public.

pcwalton marked this pull request as draft October 23, 2023 00:24

alice-i-cecile added this to the 0.13 milestone Oct 23, 2023

alice-i-cecile added C-Feature A new feature, making something new possible A-Rendering Drawing game state to the screen D-Complex Quite challenging from either a design or technical perspective. Ask for help! labels Oct 23, 2023

pcwalton mentioned this pull request Oct 23, 2023

Add a PrepareBatches render set that runs after PrepareResources. #10234

Closed

pcwalton mentioned this pull request Oct 23, 2023

Refactor mesh bind groups to be based on a set of flags. #10235

Closed

pcwalton mentioned this pull request Oct 23, 2023

Rename uv to uv_a and VERTEX_UVS to VERTEX_UVS_A in shaders. #10237

Closed

pcwalton added 2 commits October 23, 2023 02:11

Merge remote-tracking branch 'origin/main' into lightmaps

2dd3d2c

Remove MeshLayoutKey as it conflicts with other work

ff21b6c

alice-i-cecile added the S-Blocked This cannot move forward until something else changes label Oct 23, 2023

pcwalton self-assigned this Oct 23, 2023

Rename uv_a back to uv per consensus

44a64a9

coreh reviewed Oct 25, 2023

View reviewed changes

crates/bevy_pbr/src/render/lightmaps.rs Outdated Show resolved Hide resolved

pcwalton added 2 commits November 21, 2023 21:40

Merge remote-tracking branch 'origin/main' into lightmaps

2b6344d

Merge fallout

c21aa58

Fix prepass

d561631

pcwalton added 3 commits December 28, 2023 23:02

Remove unneeded #else branch in pbr_fragment.wgsl

a47e589

Fix Clippy failure

ac22e82

Revert "Fix Clippy failure"

a75098e

This reverts commit ac22e82.

Expose lightmap UVs to the prepass

faec5c3

alice-i-cecile enabled auto-merge December 30, 2023 17:23

alice-i-cecile disabled auto-merge January 1, 2024 16:38

alice-i-cecile enabled auto-merge January 1, 2024 16:39

Merge remote-tracking branch 'origin/main' into lightmaps

df3c1d6

auto-merge was automatically disabled January 2, 2024 20:03
Head branch was pushed to by a user without write access

cart added this pull request to the merge queue Jan 2, 2024

Merged via the queue into bevyengine:main with commit dd14f3a Jan 2, 2024
26 checks passed

rparrett mentioned this pull request Jan 3, 2024

many_foxes, animated_fox, split_screen examples panic with shader error #11190

Closed

pcwalton added a commit to pcwalton/bevy that referenced this pull request Jan 3, 2024

Bump the vertex attribute index for prepass joints.

84112b1

This was missed in bevyengine#10231. Closes bevyengine#11190.

pcwalton mentioned this pull request Jan 3, 2024

Bump the vertex attribute index for prepass joints. #11191

Merged

github-merge-queue bot pushed a commit that referenced this pull request Jan 3, 2024

Bump the vertex attribute index for prepass joints. (#11191)

5697fee

This was missed in #10231. Fixes #11190.

nicopap mentioned this pull request Jan 29, 2024

0.13 announcement post bevyengine/bevy-website#891

Merged

komadori mentioned this pull request Feb 15, 2024

Change MeshUniform::new() to be public. #11880

Merged

doonv mentioned this pull request Feb 16, 2024

Lightmaps don't work with shadow-enabled directional lights. #11898

Closed

mgi388 mentioned this pull request Jul 15, 2024

Use bicubic filtering in lightmaps #14322

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement lightmaps. #10231

Implement lightmaps. #10231

pcwalton commented Oct 23, 2023 •

edited

Loading

alice-i-cecile commented Oct 23, 2023 •

edited

Loading

github-actions bot commented Oct 23, 2023

pcwalton commented Oct 23, 2023

pcwalton commented Oct 23, 2023

github-actions bot commented Oct 23, 2023

Adriaaaaan commented Oct 23, 2023

pcwalton commented Oct 23, 2023

pcwalton commented Oct 24, 2023

github-actions bot commented Oct 24, 2023

github-actions bot commented Nov 24, 2023

pcwalton commented Dec 29, 2023

pcwalton commented Dec 29, 2023

pcwalton commented Dec 29, 2023

DGriffin91 commented Dec 29, 2023

pcwalton commented Dec 29, 2023

pcwalton commented Dec 29, 2023

DGriffin91 commented Dec 29, 2023 •

edited

Loading

alice-i-cecile commented Jan 1, 2024

Implement lightmaps. #10231

Implement lightmaps. #10231

Conversation

pcwalton commented Oct 23, 2023 • edited Loading

Objective

Solution

Changelog

Added

alice-i-cecile commented Oct 23, 2023 • edited Loading

github-actions bot commented Oct 23, 2023

pcwalton commented Oct 23, 2023

pcwalton commented Oct 23, 2023

github-actions bot commented Oct 23, 2023

Adriaaaaan commented Oct 23, 2023

pcwalton commented Oct 23, 2023

pcwalton commented Oct 24, 2023

github-actions bot commented Oct 24, 2023

github-actions bot commented Nov 24, 2023

pcwalton commented Dec 29, 2023

pcwalton commented Dec 29, 2023

pcwalton commented Dec 29, 2023

DGriffin91 commented Dec 29, 2023

pcwalton commented Dec 29, 2023

pcwalton commented Dec 29, 2023

DGriffin91 commented Dec 29, 2023 • edited Loading

alice-i-cecile commented Jan 1, 2024

pcwalton commented Oct 23, 2023 •

edited

Loading

alice-i-cecile commented Oct 23, 2023 •

edited

Loading

DGriffin91 commented Dec 29, 2023 •

edited

Loading