Screen Space Ambient Occlusion (SSAO) MVP #7402

JMS55 · 2023-01-28T19:13:58Z

Objective

Add Screen space ambient occlusion (SSAO). SSAO approximates small-scale, local occlusion of indirect diffuse light between objects. SSAO does not apply to direct lighting, such as point or directional lights.
This darkens creases, e.g. on staircases, and gives nice contact shadows where objects meet, giving entities a more "grounded" feel.
Closes Screen Space Ambient Occlusion #3632.

Solution

Implement the GTAO algorithm.
- https://www.activision.com/cdn/research/Practical_Real_Time_Strategies_for_Accurate_Indirect_Occlusion_NEW%20VERSION_COLOR.pdf
- https://blog.selfshadow.com/publications/s2016-shading-course/activision/s2016_pbs_activision_occlusion.pdf
Source code heavily based on Intel's XeGTAO.
Add an SSAO bevy example.

Algorithm Overview

Run a depth and normal prepass
Create downscaled mips of the depth texture (preprocess_depths pass)
GTAO pass - for each pixel, take several random samples from the depth+normal buffers, reconstruct world position, raytrace in screen space to estimate occlusion. Rather then doing completely random samples on a hemisphere, you choose random slices of the hemisphere, and then can analytically compute the full occlusion of that slice. Also compute edges based on depth differences here.
Spatial denoise pass - bilateral blur, using edge detection to not blur over edges. This is the final SSAO result.
Main pass - if SSAO exists, sample the SSAO texture, and set occlusion to be the minimum of ssao/material occlusion. This then feeds into the rest of the PBR shader as normal.

Future Improvements

Maybe remove the low quality preset for now (too noisy)
WebGPU fallback (see below)
Faster depth->world position (see reverted code)
Bent normals
Try interleaved gradient noise or spatiotemporal blue noise
Replace the spatial denoiser with a combined spatial+temporal denoiser
Render at half resolution and use a bilateral upsample
Better multibounce approximation (https://drive.google.com/file/d/1SyagcEVplIm2KkRD3WQYSO9O0Iyi1hfy/view)

Far-Future Performance Improvements

F16 math (missing naga-wgsl support [meta] f16 support gfx-rs/wgpu#4384)
Faster coordinate space conversion for normals
Faster depth mipchain creation (https://github.com/GPUOpen-Effects/FidelityFX-SPD) (wgpu/naga does not currently support subgroup ops)
Deinterleaved SSAO for better cache efficiency (https://developer.nvidia.com/sites/default/files/akamai/gameworks/samples/DeinterleavedTexturing.pdf)

Other Interesting Papers

Visibility bitmask (https://link.springer.com/article/10.1007/s00371-022-02703-y, https://cdrinmatane.github.io/posts/cgspotlight-slides/)
Screen space diffuse lighting (https://github.com/Patapom/GodComplex/blob/master/Tests/TestHBIL/2018%20Mayaux%20-%20Horizon-Based%20Indirect%20Lighting%20(HBIL).pdf)

Platform Support

SSAO currently does not work on DirectX12 due to issues with wgpu and naga:
- DX12 increase max_storage_buffers_per_shader_stage and max_storage_textures_per_shader_stage gfx-rs/wgpu#3798
- Implement Pack/Unpack for HLSL gfx-rs/naga#2353
SSAO currently does not work on WebGPU because r16float is not a valid storage texture format https://gpuweb.github.io/gpuweb/wgsl/#storage-texel-formats. We can fix this with a fallback to r32float.

Changelog

Added ScreenSpaceAmbientOcclusionSettings, ScreenSpaceAmbientOcclusionQualityLevel, and ScreenSpaceAmbientOcclusionBundle

Co-authored-by: JMS55 <47158642+JMS55@users.noreply.github.com>

nicopap

I only took a cursory glance at the shader code, but it looks good to me. Furthermore, the the render code is very good. I found it pleasing to read.

Just a few minor nits.

nicopap · 2023-06-11T16:08:00Z

crates/bevy_pbr/src/ssao/mod.rs

+    Low,
+    Medium,
+    #[default]
+    High,
+    Ultra,
+    Custom {
+        /// Higher slice count means less noise, but worse performance.
+        slice_count: u32,
+        /// Samples per slice side is also tweakable, but recommended to be left at 2 or 3.
+        samples_per_slice_side: u32,
+    },


There is no explanation on those settings. "slice_count", "samples_per_slice_side"? What does this mean?

They're how many samples you take per pixel from the screen's depth buffer to determine occlusion. The details don't really matter, other than increasing them gives you less noise at the cost of more performance.

crates/bevy_pbr/src/ssao/mod.rs

examples/3d/ssao.rs

crates/bevy_pbr/src/render/mesh_view_bindings.wgsl

crates/bevy_pbr/src/ssao/mod.rs

crates/bevy_pbr/src/render/mesh_view_bindings.wgsl

examples/3d/ssao.rs

Co-authored-by: Nicola Papale <nicopap@users.noreply.github.com>

cart

Fantastic work! I did a rough "port pass" over the shader code and things seem to line up well.

crates/bevy_pbr/src/ssao/gtao.wgsl

crates/bevy_pbr/src/ssao/mod.rs

handwritten bit packing/unpacking for dx12

cart

Great work!

cart · 2023-06-18T19:20:37Z

I cancelled the merge because I just noticed that the ScreenSpaceAmbientOcclusionPlugin is added by default, which means we're spamming the console with this error by default:

ERROR bevy_pbr::ssao: SSAO is being used which requires Msaa::Off, but Msaa is currently set to Res(Sample4)

We'll need to resolve that first.

![image](https://github.com/bevyengine/bevy/assets/47158642/dbb62645-f639-4f2b-b84b-26fd915c186d) # Objective - Add Screen space ambient occlusion (SSAO). SSAO approximates small-scale, local occlusion of _indirect_ diffuse light between objects. SSAO does not apply to direct lighting, such as point or directional lights. - This darkens creases, e.g. on staircases, and gives nice contact shadows where objects meet, giving entities a more "grounded" feel. - Closes bevyengine#3632. ## Solution - Implement the GTAO algorithm. - https://www.activision.com/cdn/research/Practical_Real_Time_Strategies_for_Accurate_Indirect_Occlusion_NEW%20VERSION_COLOR.pdf - https://blog.selfshadow.com/publications/s2016-shading-course/activision/s2016_pbs_activision_occlusion.pdf - Source code heavily based on [Intel's XeGTAO](https://github.com/GameTechDev/XeGTAO/blob/0d177ce06bfa642f64d8af4de1197ad1bcb862d4/Source/Rendering/Shaders/XeGTAO.hlsli). - Add an SSAO bevy example. ## Algorithm Overview * Run a depth and normal prepass * Create downscaled mips of the depth texture (preprocess_depths pass) * GTAO pass - for each pixel, take several random samples from the depth+normal buffers, reconstruct world position, raytrace in screen space to estimate occlusion. Rather then doing completely random samples on a hemisphere, you choose random _slices_ of the hemisphere, and then can analytically compute the full occlusion of that slice. Also compute edges based on depth differences here. * Spatial denoise pass - bilateral blur, using edge detection to not blur over edges. This is the final SSAO result. * Main pass - if SSAO exists, sample the SSAO texture, and set occlusion to be the minimum of ssao/material occlusion. This then feeds into the rest of the PBR shader as normal. --- ## Future Improvements - Maybe remove the low quality preset for now (too noisy) - WebGPU fallback (see below) - Faster depth->world position (see reverted code) - Bent normals - Try interleaved gradient noise or spatiotemporal blue noise - Replace the spatial denoiser with a combined spatial+temporal denoiser - Render at half resolution and use a bilateral upsample - Better multibounce approximation (https://drive.google.com/file/d/1SyagcEVplIm2KkRD3WQYSO9O0Iyi1hfy/view) ## Far-Future Performance Improvements - F16 math (missing naga-wgsl support https://github.com/gfx-rs/naga/issues/1884) - Faster coordinate space conversion for normals - Faster depth mipchain creation (https://github.com/GPUOpen-Effects/FidelityFX-SPD) (wgpu/naga does not currently support subgroup ops) - Deinterleaved SSAO for better cache efficiency (https://developer.nvidia.com/sites/default/files/akamai/gameworks/samples/DeinterleavedTexturing.pdf) ## Other Interesting Papers - Visibility bitmask (https://link.springer.com/article/10.1007/s00371-022-02703-y, https://cdrinmatane.github.io/posts/cgspotlight-slides/) - Screen space diffuse lighting (https://github.com/Patapom/GodComplex/blob/master/Tests/TestHBIL/2018%20Mayaux%20-%20Horizon-Based%20Indirect%20Lighting%20(HBIL).pdf) ## Platform Support * SSAO currently does not work on DirectX12 due to issues with wgpu and naga: * gfx-rs/wgpu#3798 * gfx-rs/naga#2353 * SSAO currently does not work on WebGPU because r16float is not a valid storage texture format https://gpuweb.github.io/gpuweb/wgsl/#storage-texel-formats. We can fix this with a fallback to r32float. --- ## Changelog - Added ScreenSpaceAmbientOcclusionSettings, ScreenSpaceAmbientOcclusionQualityLevel, and ScreenSpaceAmbientOcclusionBundle --------- Co-authored-by: IceSentry <c.giguere42@gmail.com> Co-authored-by: IceSentry <IceSentry@users.noreply.github.com> Co-authored-by: Daniel Chia <danstryder@gmail.com> Co-authored-by: Elabajaba <Elabajaba@users.noreply.github.com> Co-authored-by: Robert Swain <robert.swain@gmail.com> Co-authored-by: robtfm <50659922+robtfm@users.noreply.github.com> Co-authored-by: Brandon Dyer <brandondyer64@gmail.com> Co-authored-by: Edgar Geier <geieredgar@gmail.com> Co-authored-by: Nicola Papale <nicopap@users.noreply.github.com> Co-authored-by: Carter Anderson <mcanders1@gmail.com>

IceSentry and others added 30 commits December 29, 2022 22:56

clean up pbr_prepass

134b27c

Apply suggestions from code review

2b0f4f9

Co-authored-by: JMS55 <47158642+JMS55@users.noreply.github.com>

Extract constants

c157cae

Remove newline

b6acf22

fix unwrap on empty frag shader in prepass

715ca65

docs

35093ed

Merge commit '35093ed2e81b737fadfa5b50c32a6e49245acdf0' into gtao

5c9bb20

Fix merge

826b19a

Merge branch 'main' into depth-prepass

d95cc01

Merge branch 'main' into depth-prepass

8e1ffbc

update to main

d64286f

add alpha_mask_phase items is_empty check

c9159a5

shorten comment

c99f582

fix ci

27db3b8

WIP FSR2 plugin

060c52d

Further plugin setup

14efad2

Merge branch 'main' into depth-prepass

fdbb4d2

Merge branch 'main' into depth-prepass

a5ec2f5

Merge remote-tracking branch 'icesentry/depth-prepass' into gtao

0af0d67

Fix merge

a3e5c1b

WIP: FSR2Plugin

77ca0c8

FSR2 jitter (breaks TAA for now)

8f5fda4

Fix shader ifdefs

1bb796f

Extract projection

63003f7

Misc

0b757e0

Fix dep path

f48bb83

Add neccesary texture usages

1d48c29

Misc

3d5c81c

Add device field

ff68974

Cleanup jitter

e9942d7

nicopap approved these changes Jun 11, 2023

View reviewed changes

nicopap reviewed Jun 11, 2023

View reviewed changes

crates/bevy_pbr/src/ssao/mod.rs Show resolved Hide resolved

nicopap reviewed Jun 11, 2023

View reviewed changes

crates/bevy_pbr/src/ssao/mod.rs Outdated Show resolved Hide resolved

IceSentry reviewed Jun 11, 2023

View reviewed changes

crates/bevy_pbr/src/render/mesh_view_bindings.wgsl Show resolved Hide resolved

examples/3d/ssao.rs Outdated Show resolved Hide resolved

JMS55 and others added 4 commits June 11, 2023 17:27

Update crates/bevy_pbr/src/ssao/mod.rs

b39b8c0

Co-authored-by: Nicola Papale <nicopap@users.noreply.github.com>

Apply PR feedback

4ba6850

Apply more PR feedback

36cd601

handwritten bit pack/unpacking

6039c18

cart reviewed Jun 17, 2023

View reviewed changes

crates/bevy_pbr/src/ssao/gtao.wgsl Show resolved Hide resolved

crates/bevy_pbr/src/ssao/mod.rs Outdated Show resolved Hide resolved

JMS55 added 4 commits June 18, 2023 11:23

Merge pull request #12 from Elabajaba/ssao-bitpacking

460baef

handwritten bit packing/unpacking for dx12

Refactor ssao settings

bb025a2

Fix ssao example

e308d3b

Add ssao platform docs

2e73f48

JMS55 added the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Jun 18, 2023

Appease clippy

b5256b9

cart approved these changes Jun 18, 2023

View reviewed changes

cart enabled auto-merge June 18, 2023 19:09

cart disabled auto-merge June 18, 2023 19:17

JMS55 added 2 commits June 18, 2023 16:50

Fix ssao error message always triggering

04e800c

Tweak error message

5e9b6bb

cart enabled auto-merge June 18, 2023 20:53

cart added this pull request to the merge queue Jun 18, 2023

Merged via the queue into bevyengine:main with commit af9c945 Jun 18, 2023
24 checks passed

JMS55 mentioned this pull request Jun 18, 2023

SSAO followup tracking issue #8888

Open

13 tasks

This was referenced Jun 28, 2023

News: Release 0.11 (wrong branch ... see #657) bevyengine/bevy-website#656

Closed

News: Release 0.11 bevyengine/bevy-website#657

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Screen Space Ambient Occlusion (SSAO) MVP #7402

Screen Space Ambient Occlusion (SSAO) MVP #7402

JMS55 commented Jan 28, 2023 •

edited

Loading

nicopap left a comment

nicopap Jun 11, 2023

JMS55 Jun 11, 2023

cart left a comment

cart left a comment

cart commented Jun 18, 2023

Screen Space Ambient Occlusion (SSAO) MVP #7402

Screen Space Ambient Occlusion (SSAO) MVP #7402

Conversation

JMS55 commented Jan 28, 2023 • edited Loading

Objective

Solution

Algorithm Overview

Future Improvements

Far-Future Performance Improvements

Other Interesting Papers

Platform Support

Changelog

nicopap left a comment

Choose a reason for hiding this comment

nicopap Jun 11, 2023

Choose a reason for hiding this comment

JMS55 Jun 11, 2023

Choose a reason for hiding this comment

cart left a comment

Choose a reason for hiding this comment

cart left a comment

Choose a reason for hiding this comment

cart commented Jun 18, 2023

JMS55 commented Jan 28, 2023 •

edited

Loading