Implement omnidirectional cameras for real-time reflection probes. #13840
+1,744
−456
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit introduces a new type of camera, the omnidirectional camera. These cameras render to a cubemap texture, and as such extract into six different cameras in the render world, one for each side. The cubemap texture can be attached to a reflection probe as usual, which allows reflections to contain moving objects. To use an omnidirectional camera, create an
OmnidirectionalCameraBundle
.Because omnidirectional cameras extract to six different sub-cameras in the render world, render world extraction code that targets components present on cameras now needs to be aware of this fact and extract components to the individual sub-cameras, not the root camera component. They also need to run after omnidirectional camera extraction, as only then will the sub-cameras be present in the render world. New plugins,
ExtractCameraComponentPlugin
andExtractCameraInstancesPlugin
, are available to assist with this.Each side of an omnidirectional camera can be individually marked as active via the
ActiveCubemapSides
bitfield. This allows for the common technique of rendering only one (or two, or three) sides of the cubemap per frame, to reduce rendering overhead. It also allows for on-demand rendering, so that an application that wishes to optimize further can choose sides to refresh. For example, an application might wish to only rerender sides whose frusta contain moving entities.In addition to real-time reflection probes, this patch introduces much of the infrastructure necessary to support baking reflection probes from within Bevy as opposed to in an external program such as Blender, which has been the status quo up to this point. Even with this patch, there are still missing pieces needed to make this truly convenient, however:
Baking a reflection probe requires more than just saving a cubemap: it requires pre-filtering the cubemap into diffuse and specular parts in the same way that the glTF IBL Sampler does. This is not yet implemented in Bevy; see GenerateEnvironmentMapLight #9414 for a previous attempt.
The cubemap needs to be saved in
.ktx2
format, as that's the only format that Bevy presently knows how to load. There's no comprehensive Rust crate for this, though note that my glTF IBL Sampler UI has code to do it for the specific case of cubemaps.An editor UI is necessary for convenience, as otherwise every application will have to create some sort of bespoke tool that arranges scenes and saves the reflection cubemaps.
The
reflection_probes
example has been updated in order to add an option to enable dynamic reflection probes, as well as an option to spin the cubes so that the impact of the dynamic reflection probes is visible. Additionally, the static reflection probe, which was previously rendered in Blender, has been changed to one rendered in Bevy. This results in a change in appearance, as Blender and Bevy render somewhat differently.Partially addresses #12233.
Changelog
Added
OmnidirectionalCameraBundle
has been added in order to render to a cubemap. This allows reflection probes to reflect the dynamic scene.