-
-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Request: Support for dynamically scalable effect resolution #192
Comments
Hi!
Some effects do have a Blur effects like bloom usually need to be configured manually to look right for a given scene. The final parameter configuration is always tied to a specific resolution at which the bloom overlay will be rendered. Changing the resolution without adjusting the blur kernel size or blur scale would produce different results. So instead of scaling render resolutions based on the screen resolution, consider defining parameter configurations for specific fixed resolution settings and switch between those sets depending on the available performance budget. Note that only the render height or width needs to be specified, either via the constructor or the resolution property. The counterpart will be calculated based on the aspect ratio.
The effects included in this library do render things at lower resolutions whenever possible and are configurable in that regard as explained above. Although it makes sense to render SSAO at a lower resolution, I intentionally ditched down-scaling and blurring to avoid issues like haloing and flickering. However, I agree that it would be nice to have an option to trade quality for performance after all.
That's a nice idea! I'll try that out when I get a chance. I'm still worried about ugly gaps between objects and AO, though.. |
Thanks for the example!
Right -- I think this would typically mean that effects need to be scale-aware and ensure that they are resolution independent in some way or that the parameters are specified in normalized screen space. Some effects may depend on exact pixel samples, though, which I'm not necessarily sure how to handle so generally.
Great I see now this is dynamically settable, now. I'll take a deeper look.
It might not get rid of artifacts entirely but it could help improve performance quite a bit and be "good enough" in most cases. My impression is that this is a common technique in modern games but I'm just diving into this stuff. And of course setting the scale to Thanks again! |
I contemplated going down this path before arriving at the fixed resolution solution. AFAIK, there's no trivial solution for scale-aware parameters. Say a developer specifies On a side note, games usually offer steplike settings and often hide the fine-grained parameters. Some MMOs like WoW and GW perform realtime optimizations in crowded areas, but I think they also adjust the settings more roughly while focusing on the critical stuff such as the number of animated characters.
Not quite, actually. Right now, SSAO is performed entirely in the main effect shader and uses no additional render operations. If we want to render AO into a down-scaled render target, the shader needs to be converted into an independent material first. The SSAO effect shader would then just blend the AO texture with the scene colors while taking the depth texture into account to filter artifacts. Assuming we'd add a I just realized that SSAO also produces inconsistent results across different screen resolutions (the sampling radius is fixed, but the resolution isn't). The result still looks surprisingly good at very low resolutions, though, because the effect relies on the scene depth and normals. |
I tried to add a resolution setting to the For example, everything is fine if the screen resolution is 1920x480 and the SSAO resolution is 960x240. But if the screen resolution is 1920x490 and SSAO renders at 940x240, blocky patterns appear:
None of the SSAO parameters affect the pattern. I tried basing the sampling step size on the main resolution but that didn't help. I'm trying to find a good solution, but all I can think of is requiring the SSAO resolution to be a multiple of the main render resolution. Any ideas? Sandbox for direct testing: https://codesandbox.io/s/ssao-resolution-9xpel (check line |
that sounds like something that wouldn't cause too much trouble in user land, no? i would most certainly use it if it came out like that. |
I consider this solution a last resort because the dependence on the main resolution defeats the purpose of the fixed resolution setting. It would be too fragile. I'm currently looking into depth/normal downsampling to fix the patterns. And while I'm at it I'll also implement depth-aware SSAO upscaling based on this article: https://eleni.mutantstargoat.com/hikiko/depth-aware-upsampling-6 Hopefully the downscaling will solve the pattern issue. |
I was taking a look at that article a little bit ago, as well. I haven't implemented depth aware upscaling but it seems like fixing the AO resolution to a 1/2 or 1/4 size like you suggested might be easiest. I'm not sure what I would do in the case where the main resolution is not a factor of 2, though...
I'm not sure how the AO effect works in this library but it seems like an effect where you could divide out the resolution to make the effect scale independent right? Though I know you mentioned it might be more complicated than that. |
Sadly, this didn't remove the pattern, but added more artifacts. In summary, using a smaller render target for SSAO results in regular blocky patterns when the depth texture size is not a multiple of the SSAO render target size (had that backwards last time). Using a custom downsampling algorithm that picks the most representative depth in a 2x2 neighborhood from the high-res depth texture results in additional regular artifacts that look like broken lines. This issue could be a bug in the downsampling shader, but the resulting depth texture looks fine. My guess is that there's a bug somewhere in the SSAO sampling algorithm. I'll try replacing it with another one to check if that affects anything.
Yes, and if the |
SSAO effect has large performance impact, thus having independent res is much needed. |
I revised the SSAO shader based on https://research.nvidia.com/publication/scalable-ambient-obscurance and managed to improve the overall quality of the effect. One important change is that the sampling radius will now be properly scaled based on how far the fragments are away from the camera. The new shader also handles the artifacts from the downsampled depth texture much better. Raising the While I was debugging the effect, I also found another issue that isn't too obvious: the sampling algorithm produces incorrect shadows at the screen edge when the edge touches a flat surface in a certain angle. Apparently this is a well-known limitation of SSAO, but I couldn't find an official fix for it. My guess is that most people don't bother with this because it's a bit of an edge case (no pun intended). Anyway, I fixed this by skipping samples that lie outside the screen. The range check adds a few GPU cycles, but the performance impact should be minimal. Here are some screenshots:
The effect is almost complete; I still need to add the depth-aware upscaling logic to the final blend shader and I have to figure out how to make all these changes backward-compatible. One particularly important change is that depth and normals are packed into a single floating-point texture which adds a new hardware requirement to the effect. |
@vanruesc those sound like some great improvements! World-space scale for AO size is a nice addition. For fun I've been working on a GTAO implementation to learn a bit which is a bit more expensive but produces some really nice results. To help with low resolution and low sample count I've tried out the depth and normal-aware upscaling / blur and it looks pretty nice:
The code's a bit messy with me just hacking around but you can check out the demo here -- the images were taken with 2 rotation and 6 steps which works out to 24 texture samples. Still trying to find a good jitter method to smooth out the initial sampling noise better. I'll be interested in seeing your AO looks up close with world space scaling. Feel free to check out the code for the upscale blur if you want -- that's located here. I haven't decided how far I'm going to take this yet but there's a bit of optimization that could be done, yet, but I'm pretty happy with the way it looks. The approach is probably a bit heavier than you'd want for lower end platforms, though, so I'm not sure how much real use it could get. As an added bonus after seeing someones screen space illumination experiment bounce around on Twitter I added a color gather too so you can get a little bit of bounce light illumination:
It only works okay without the base color pass. The bounce light in the sponza scene makes it look more washed out than anything. Maybe MRT support will come some day 😬 |
Cool stuff, thanks for sharing! Haven't heard of GTAO before. |
Added a |
Hello! Great work on the effects.
It would be great to support dynamically scalable effect resolution for some of the more intensive effects or at least a fixed effect scale on instantiation. SSAO, for example, can be a bit expensive depending on how many samples you want to use but this can be improved by rendering it at half or quarter resolution. A depth / normal aware upscale might be needed afterward to remove some artifacts depending on the effect.
To explain a bit more in my application I iteratively increase / decrease the resolution of or enable / disable the effects based on the measured framerate of the web app. So if the application is running slowly we might lower the resolution scale of the SSAO or boom effects or even disable them until the framerate reaches an acceptable threshold. And inversely if we have some extra frame time we'll enable or improve effect quality until we see frame dips.
Here's something like I'm imagining:
Thanks again for the awesome work!
The text was updated successfully, but these errors were encountered: