Feature: more fine grain MSAA control

[TrickMe]

Godot v3.1.1.stable.official

Linux Mint 19.2
Mesa 19.1.2-0bpadoka0
AMD RX 570

I know devs don't want to hear about it, because rumor has it, that supersampling is too resource demanding.
I benchmarked performance decrease of about 20% to 25% with FSAA/SSAA ( 4 full samples per pixel) enabled with every graphics card I had since 2005!
SSAA delivers the best picture quality of all antialiasing modes and algorithms so far (2019).
It doesn't just rely on antialiasing polygon edges, but also seamlessly takes texture transparency, shadows, etc. into account.
For some people, like me, it is worth wasting 20% to 25% performance.

If you would be kind enough to enable users to control the amount of sample-types (cover-, z-, color-samples i guess) to use in MSAA, I would really appreciate it.
Especially because the sample-types must have been integrated already to enable basic MSAA, I just want to be able set all samples to full samples within the project settings.
Since the Vulkan-renderer is being worked on right now, more fine grain MSAA control could eventually be enabled along the way, at least for Vulkan.

Please also consider, that "rotated-grid 4xMSAA" (often called 6xMSAA) delivers better picture quality, at no measurable performance cost, than regular 4xMSAA.
2xMSAA is, in my opinion, mostly useless, because it multisamples only one dimension of a 2-dimensional pixel area.

edit:
replaced HQMSAA with FSAA, I got it mixed up.

[clayjohn]

Note that in the current 3.x versions of Godot we do not implement our own MSAA, we use the built-in OpenGL MSAA. So it is not a matter of exposing more options to users, we would have to implement custom MSAA from scratch. At this point implementing anything more advance than we have is out of the question.

You can do SSAA yourself by rendering to a large Viewport.

[TrickMe]

You can do SSAA yourself by rendering to a large Viewport.

Rendering to a larger Image resolution and downsampling to display resolution would have a serious performance impact.
SSAA through Full-sample-MSAA has the benefits of being processed entirely on the GPU, with low performance impact.

I'm pretty sure, although not entirely sure, that graphic-apis offer a way to multisample, using only full samples.

On windows you can literally switch from MSAA to full-sample MSAA (therefor SSAA), with a switch in the driver configuration (AMD driver).
Even if applications do not support SSAA, MSAA is then done using only full samples.

[clayjohn]

SSAA through Full-sample-MSAA has the benefits of being processed entirely on the GPU, with low performance impact.

FSAA and SSAA are two different techniques. FSAA is a hardware-specific form of MSAA that results in quality similar to SSAA.

I'm pretty sure, although not entirely sure, that graphic-apis offer a way to multisample, using only full samples.

OpenGL certainly doesn't... Maybe Vulkan does.

On windows you can literally switch from MSAA to full-sample MSAA (therefor SSAA), with a switch in the driver configuration (AMD driver).

That makes sense as this would be a hardware-specific feature. It is certainly not available on all hardware/driver configurations.

[TrickMe]

we would have to implement custom MSAA from scratch

Don't ever do that!
There MUST be a way to do full-sample MSAA with openGL.
There are applications using it, Tomb Raider(2013) for example.
Tumb Raider even runs on linux (ported by Feral) and has a benchmark option, so you could even check the performance impact.

[clayjohn]

You are confused about how the MSAA settings work, they are not dependant on OpenGL nor on operating system. The settings you are asking about are connected to your GPU hardware.

For example, the form of FSAA you are taking about (Sparse Grid Supersampling Anti-Aliasing) is an AMD-only feature. It is enabled in your AMD driver settings, not by OpenGL and certainly not by Godot. Similarly, users of AMD or NVidia cards can use Coverage Sampled Anti-Aliasing (called "enhanced quality AA, by AMD). But again, these features are hardware/driver dependent and Godot cannot access them, let alone control them.

OpenGL literally only supports 2x, 4x, 8x, and 16x MSAA, You can alter the coverage of samples by manually implementing the Multisample resolve instead of blitting.

[TrickMe]

Is there a specific reason why openGL doesn't support it?
Is it patented, licensed, proprietary or such?
Do you know?

[clayjohn]

@TrickMe They are two completely different things. OpenGL provides a basic graphics API that all hardware vendors implement. Within that API you can have certain options for rendering things.

Outside of the graphics API some vendors provide settings that override the graphics API. These are hardware dependent and they are selected inside of a settings menu connected to the graphics driver. What these settings do is instruct the GPU to ignore OpenGL and instead use the options selected by the user. OpenGL has no control over this as it is just a software API.

What extra options these hardware manufacturers decide to create and implement is up to them. CSAA and EQAA are both proprietary. However, not only that, they are not algorithms that can be run on other hardware, they are features built into the hardware itself. Asking how to run CSAA on an intel chip is like asking how to switch gears on your horse. CSAA has no meaning outside of NVidia chips that contain that feature. By nature OpenGL is hardware agnostic, so it generally can't contain hardware specific features.

I say "generally" because hardware vendors implement their own versions of OpenGL which may enable access to those features, but only on their hardware, if you try to access them on other GPUs, they will crash. If for example, you were writing a program that would only run on NVidia cards, you could enable a NVidia-only OpenGL extension that gives users access to CSAA. However, Godot is made to run on all graphics cards, so it sticks to regular OpenGL that doesn't have access to hardware specific features.

[TrickMe]

So, some applications (e.g. Tomb Raider) check, if certain proprietary features are exposed by the vendor specific openGL driver, and make them selectable by the user, if available?

If FSAA (and Sparse Grid Supersampling Anti-Aliasing) are vendor specific proprietary features, the patent must run out someday.
In 3 years from now, at most, I assume.
I hope openGL and Vulkan can use them then.

[clayjohn]

So, some applications (e.g. Tomb Raider) check, if certain proprietary features are exposed by the vendor specific openGL driver, and make them selectable by the user, if available?

Exactly. :)

If FSAA (and Sparse Grid Supersampling Anti-Aliasing) are vendor specific proprietary features, the patent must run out someday.

Its not a matter of being patented. These are hardware features. Not something that can be implemented with a software API like OpenGL or Vulkan. The physical GPU needs to be designed and built to support these features.

[TrickMe]

The physical GPU needs to be designed and built to support these features.

The GPU must always be designed to support specific (API) features.
https://mesamatrix.net/
https://www.x.org/wiki/RadeonFeature/

Basically any GPU that support MSAA should also support FSAA.
Those are very old features, I played around with them in 2002 on my GeForce 256!
Sparsed grid about 2005 with my AMD-GPU.

I don't get why MSAA can be used within openGL, and FSAA just can't.
Those specific features emerged simultaneously.

Although I have to say: When the patent on "anisotropic filtering" expired a few years ago, I also was surprised about it. Maybe FSAA is still blocked by a patent.

[clayjohn]

Those are very old features, I played around with them in 2002 on my GeForce 256!
Sparsed grid about 2005 with my AMD-GPU.

Age doesn't matter here. They are still hardware specific features. Back to the car example, just because cars have had gearboxes for 100 years doesn't mean that you should be able to switch gears on a horse. Just because NVidia has had CSAA for a long time and AMD has had sparsed grid doesn't mean anything for other hardware vendors.

Although I have to say: When the patent on "anisotropic filtering" expired a few years ago, I also was surprised about it. Maybe FSAA is still blocked by a patent.

I doubt its a patent issue. My guess is that AMD's EQAA is the exact same as NVidias CSAA. Its more likely Khronos felt it was unnecessary to force all vendors to implement those features.

Like I said before, I don't know what Vulkan includes in the spec, it may be that for Godot 4.0 it is as easy as toggling these features on. But for that to happen, Vulkan would have to implement these different features in a platform agnostic way.

[TrickMe]

But Godot supports MSAA, and FSAA uses exactly the same types of samples as MSAA, if I got that correctly.
So all the features to support FSAA are already implemented in GPUs that support MSAA.
Am I wrong somehow?

[clayjohn]

But Godot supports MSAA, and FSAA uses exactly the same types of samples as MSAA, if I got that correctly.

I couldn't tell you exactly, but from AMD's description it sounds like it is very different

SSAA improves image quality by taking more samples than MSAA and AAA, reducing aliasing from all textures. SSAA has the highest impact on FPS of all AA settings within Radeon Settings.
Taken from: https://www.amd.com/en/support/kb/faq/dh-012

OpenGL supports a maximum of 16 samples. My guess is that Sparse Grid Supersampling Anti-Aliasing takes a higher number (maybe 32 or 64) and then has a very specific hardware feature that makes this not bring the GPU to a crawl. Note that it is a different setting from MSAA and not the same as turning MSAA samples up to max which likely implies something very different is happening under the hood.

[TrickMe]

On this image:
https://img.purch.com/amd-eqaa-radeon-hd-6970/w/755/aHR0cDovL21lZGlhLmJlc3RvZm1pY3JvLmNvbS9WLzkvMjczMjg1L29yaWdpbmFsL2VxYWEuanBn
you see what I would like to be implemented.

The 4x MSAA and 8x MSAA examples show it pretty clearly: 4x full samples with color, not horizontal nor vertically alligned, but "rotated" (2005 AMD called that 6xMSAA), and 8x full samples with color. (the picture is from 2011)

Although these pictures indicate full samples with MSAA, in practice it just doesn't correspond to the actual image quality. With 4xMSAA you can still see Aliasing and flickering of details, which was not the case back in 2005 with 4xSSAA or "6x"SSAA. I checked that a lot back in the day and I still do now, and I don't think that MSAA actually does full samples.

Extra coverage samples like EQAA or CSAA have never been interesting to me, since they only take polygons into account (I think).
Only full samples make sense to me, coverage samples are a waste of time.

Back in 2005, even with resolutions like 320x240 4xSSAA, I got a nice picture, now, with 1920x1080 8xMSAA I can still see edges and flickering of details.
If MSAA REALLY uses only full samples, then I don't get why the picture-quality doesn't represent that at all!
Is the document/picture indicating the sample usage incorrectly?

[TrickMe]

Is MSAA maybe implemented differently in openGL?
If so, where can I find out the way it works in openGL?

[TrickMe]

This Phoronix article is also very interesting:
https://www.phoronix.com/scan.php?page=news_item&px=RadeonSI-EQAA-Mesa-18.2

If the radeonsi mesa openGL driver for AMD-GPUs can enforce the MSAA-mode, then they must be present within openGL, as stated in the article, a presume.
Although I don't get the usage of this environment variable, it never worked for me.
I must be using it wrong.

Again: I'm only interested in the modes stated "MSAA" in the article, which I presume are implemented in openGL and should use full samples, although I somehow assume they don't.
I'm aware that EQAA might be vendor specific, but I think MSAA is available even by mobile vendors or openGL ES.

If MSAA (with full samples), is not available by openGL (ES) and mobile vendors, etc., then how exactly does the openGL MSAA implementation work?

[clayjohn]

@TrickMe This is getting to be a bit much. I have already explained this a few times now.

You seem to not understand the difference between GPU, drivers, Graphics API, and game engine.

Just because a feature is present on an old GPU does not mean that a newer graphics API can bring it to other hardware. CSAA, EQAA, AMDs SSAA are all hardware features and are only present on the hardware that implements them. OpenGL does not come into the question. The drivers for the hardware that supports those features will provide an extension to OpenGL to use them. However, core OpenGL does not touch hardware specific features.

The form of MSAA in the image you linked above is an image of exactly how OpenGL specifies MSAA works. It is the form of MSAA that Godot uses. The second row with the extra coverage is how CSAA and EQAA are able to get better looking AA with only slightly more cost.

[TrickMe]

This is getting to be a bit much. I have already explained this a few times now.

You seem to not understand the difference between GPU, drivers, Graphics API, and game engine.

I understand that I'm repeating myself.
It's not that I don't understand what you're saying, I'm just trying to find a way to make other people, and you, understand my issue with the general/openGL specific implementation of MSAA.
CSAA, EQAA or vendor specific features are not what I'm referring to.

The form of MSAA in the image you linked above is an image of exactly how OpenGL specifies MSAA works.

It does not work like that, that is exactly what I'm trying to make clear.
If MSAA would actually use multiple full samples (with color) for every pixel under any condition, all the time, every frame, then there wouldn't be an issue.

You can open the godot TPS-demo, select 4xMSAA or even 8xMSAA, and if you look closely, you will see aliasing artifacts everywhere, especially at the light sources.
Turn down the resolution to make it more visible, if you don't believe me.

And it is not just a godot-, openGL- or AMD- specific issue.

[TrickMe]

I think it could might be useful, if I provide you devs with some basic overview of how MSAA and MSAA-derived Anti-aliasing works, and why I think that SSAA/FSAA doesn't rely on vendor specific extensions/features.

SSAA/FSAA relies on the same sample-types that are used in MSAA, the only difference between variants are the amount, which are always a power of 2, and the positioning/alignment of the samples, within the pixel area, as this picture shows:
https://img.purch.com/amd-eqaa-radeon-hd-6970/w/755/aHR0cDovL21lZGlhLmJlc3RvZm1pY3JvLmNvbS9WLzkvMjczMjg1L29yaWdpbmFsL2VxYWEuanBn

The picture shows the same amount of color- and coverage-samples within the pixel area with MSAA.
But actually it seems that coverage samples are always used with every pixel, while the amount of color-samples used is determined for every pixel individually, based on the result of the coverage samples.
So the coverage samples are used to determine the necessary amount of color-samples to provide anti-aliasing of the geometry/mesh.

The difference of MSAA and EQAA/CSAA is, that with MSAA the amount of color-samples is equal to the amount of coverage-samples, while with EQSS/CSAA the amount of color-samples is half of the amount of coverage samples.
That could possibly be the reason, why modern MSAA does not support 16xMSAA, although the GPUs officially provide 16xMSAA. Because the amount of coverage-samples is twice the amount of max. color-samples. As the picture shows, 16xcoverage samples are used in 8xEQAA/8xCSAA.
Also, it indicates that the openGL-implementation of MSAA used by Godot does provide EQSS/CSAA, therefor those variants are not vendor-specific.

So much to basic MSAA.

With SSAA/FSAA things are a bit different.
Color-samples are ALWAYS used in the given amount, 4x, 8x or even possibly 16x (very nice), within EVERY pixel area.
Since there is no need for coverage-samples to determine the amount of color-samples, coverage samples could possibly be dropped completely. Although I'm not entirely certain, if coverage-samples might or might not be used for other rendering purposes, and therefor be necessary to render a sample, I haven't seen any docs on that.

by @clayjohn :

SSAA improves image quality by taking more samples than MSAA and AAA, reducing aliasing from all textures. SSAA has the highest impact on FPS of all AA settings within Radeon Settings.
Taken from: https://www.amd.com/en/support/kb/faq/dh-012

This doc most certainly is referring to color-samples:
SSAA improves image quality by taking more "COLOR-"samples than MSAA and AAA, reducing aliasing from all textures.

MY POINT IS:
All the sample-types used by SSAA/FSAA are already being used by basic MSAA, but with a higher amount of color-samples than basic MSAA.
SSAA/FSAA are not vendor-specific features, if a GPU provides MSAA-functionality, it also provides SSAA/FSAA and EQAA/CSAA functionality, because the sample-types are the same.
That is also true for API-implementations of those features provided by GPUs, like in openGL.
OpenGL provides basic MSAA, therefor it also provides SSAA/FSAA and even EQAA/CSAA without the need of vendor-specific features.
In fact, I'm most certain, that Godot is using EQAA/CSAA through openGL, since 16xMSAA is not provided.
There is also no need for SSAA used Tomb Raider to use vendor-specific features.
I used Tomb Raider as a reference in an earlyer post and I think it was a bad/false example.

There apparently is also a sample-type called z-sample as mentioned in this article:
https://www.phoronix.com/scan.php?page=news_item&px=RadeonSI-EQAA-Mesa-18.2

[akien-mga]

Please keep the discussion constructive and respectful, otherwise I'll lock the thread.

[Zireael07]

Uh, I can't see anything disrespectful or non-constructive in his latest posts?

[TrickMe]

Please keep the discussion constructive and respectful, otherwise I'll lock the thread.

Fine, but I hereby declare being annoyed!

Uh, I can't see anything disrespectful or non-constructive in his latest posts?

I updated it just now

[clayjohn]

@TrickMe I understand that you are frustrated. This has been a very frustrating exchange for both of us. I am sorry I have made you feel like your ideas are not being heard.

Please let me assure you that that I am not dismissing your ideas out of hand. I am only trying to point out the limitations of what we can implement in Godot due to the limitations of supporting a broad range of hardware.

Maybe it will be helpful to understand what MSAA looks like from the graphics API. Keep in mind, this is what we are limited to implemented, whether or not hardware supports more advanced features.

In OpenGL you enable MSAA with glEnable(GL_MULTISAMPLE). Thiss turns on MSAA, there are no other options for CSAA, EQAA, or anything else. OpenGL's concept of multisampling just "color" samples. It has no concept of of "coverage" samples. In the image you linked earlier, they called this "full samples" but in fact, it is just color samples. The article was calling them full samples in order to help distinguish between "coverage-only" samples on the one hand and "color+coverage" samples on the other. So, in OpenGL you can only turn basic MSAA on or off, there are no other options.

Next, if you want to actually use MSAA you have to enable it in each framebuffer. You do that by allocating a framebuffer with a multisample texture glTexImage2DMultisample( ..., num_samples, ..); one of the properties of this allocation is num_samples it accepts 0, 2, 4, 8, or 16 as parameters.

That is the extent of the control OpenGL provides over MSAA. Anything beyond that relies on specific hardware that implements proprietary features. Because Godot targets so many platforms (high-end desktop, low-end desktop, high-end mobile, low-end mobile), we cannot assume that everyone has a GPU that supports some proprietary form of coverage sampling so we stick to plain MSAA as provided by OpenGL.

I see in your above posts a few particular points that I would like to address as well to help clear things up.

All hardware supports coverage samples
This is not true. Coverage samples are only supported by dedicated GPUs provided by AMD or NVidia. Each company provides the feature in its own proprietary way. Users who do not have either an AMD or NVidia device cannot use coverage samples.

OpenGL uses coverage samples under the hood
OpenGL does not use coverage samples. It only performs one sample type, which is a color sample. OpenGL has no concept of coverage samples as distinct from color samples. However, the color sample OpenGL uses fills the role of both color and coverage sample, which is why the article refers to it as "full sample".

Coverage samples are just an extension of MSAA
Not necessarily. CSAA, EQAA, FSAA, Sparse grid SSAA, are all proprietary hardware features that rely on a different techniques than MSAA, they are not an extension of MSAA.

The article linked above explains how MSAA works on all hardware
The article you continue to reference only explains coverage sampling differs from MSAA on devices that utilize coverage sampling. What it says has no bearing on the rest of the devices that don't support coverage sampling.

As a final note, I know that it feels like I am trying to shut down your ideas when I disagree with you. But that is not my intention. I am just trying to explain what is actually going on between the hardware->drivers->software for you as it is different than what you currently think. If Godot could natively support a better form of MSAA than it already does, then I would be in full agreement with you. However, the issue is that what you suggest cannot be implemented in Godot until those features are support by all hardware.

[TrickMe]

OpenGL's concept of multisampling just "color" samples.

If that is true, then I'm still wondering why MSAA is not matching SSAA quality.
If only color-samples are used, it surely should!

I will let this sink in for a few days and maybe I'll find some docs, that indicate why MSAA-quality is so bad.

edit:
I'm using:
Linux Mint 19.2
Mesa 19.1.2-0bpadoka0
AMD RX 570
by the way.

[TrickMe]

Although there is very little documentation to be found, it seems that opengl provides functionality to enable MSAA, but does not expose any control over sample-types to an application using MSAA, besides the amount of samples to be used.
So far so limited.
I'm still convinced, that GPUs able to do MSAA are theoretically also able to do SSAA, because the sample types used by MSAA are the same as by SSAA.
But as long as applications are not provided with a more direct access to MSAA usage by opengl (or vulkan), the way those samples are used is only configurable by the driver, which implements opengl.

I've also found an interesting overview on MSAA, which could be worth reading:
https://mynameismjp.wordpress.com/2012/10/24/msaa-overview/
Under the section "Working with HDR and Tone Mapping", there could be a solution to the aliasing-problem of the light sources within the Godot-TPS-demo.

I will post updates if I think it could be helpful, but I think, for now, this is pretty much it.

edit:
With SSAA I'm referring to using multiple color-samples (or fragments) per pixels, which is more performant than rendering in a higher resolution with downsampling.

[TrickMe]

Please also note if SSAA would be implemented:

The "Working with HDR and Tone Mapping" in the link of my last post is refering to MSAA, not SSAA!
Since SSAA uses multiple color samples / fragments for every pixel, the issue is not present with SSAA.
This possible fix for MSAA with HDR would only generate overhead and must not be applied, if the implementation should be fast.

As I noticed, Tomb Raider's implements of SSAA is very slow (possibly even halfes framerate).
For a better implementation, try run Project Cars 2 (free demo available).
Even when using Proton (Valves fork of Wine) the framerate just drops about 25% for 4x SSAA, like it should be.
Also there are 4 quality settings of SSAA in Project Cars 2, which probably correspond to 2x, 4x, 8x and 16x supersampling, while 16x MSAA is not available for my GPU with Godot.

SSAA can be done efficiently, even if emulated on Linux using OpenGL.
If anyone knows how to do it with OpenGL / Vulkan, please shout out!

[Calinou]

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!