Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multisampled multiview support #2912

Open
jdashg opened this issue Jul 18, 2019 · 2 comments

Comments

@jdashg
Copy link
Contributor

commented Jul 18, 2019

We need to support multisampled multview rendering, but how to do so is tricky.

Since we are using texture arrays for multiview, there are two options:

EXT_multiview_texture_multisample has a good brief on the two:

This is one of two extensions that allow multisampling when using
OVR_multiview. Each supports one of the two different approaches to
multisampling in OpenGL and OpenGL ES:

Core OpenGL and OpenGL ES 3.1+ have explicit support for multisample
texture types, such as TEXTURE_2D_MULTISAMPLE. Applications can access
the values of individual samples and can explicitly "resolve" the
samples of each pixel down to a single color.

The extension EXT_multisampled_render_to_texture provides support for
multisampled rendering to non-multisample texture types, such as
TEXTURE_2D. The individual samples for each pixel are maintained
internally by the implementation and can not be accessed directly
by applications. These samples are eventually resolved implicitly to
a single color for each pixel.

This extension supports the first multisampling style with multiview
rendering; the OVR_multiview_multisampled_render_to_texture extension
supports the second style. Note that support for one of these multiview
extensions does not imply support for the other.

OVR_multiview_multisampled_render_to_texture was first-to-market, and is generally better supported on ES, particularly ES3.0. OVR_multiview_multisampled_render_to_texture is possible to implement on top of EXT_multisampled_render_to_texture, but not, strictly speaking, vice-versa.

The tricky part is the behavior of implicit resolve in OVR_multiview_multisampled_render_to_texture, as described in EXT_multisampled_render_to_texture:

When the texture level is used as a source or destination for any
operation, the attachment is flushed, or when the attachment is broken,
the GL implementation may discard the contents of the implicit multisample
buffer. If the contents are discarded, the subsequent operations on the
multisample buffer will behave as if all samples within a pixel have the
value most recently written to the color buffer for that pixel.

The operations which may cause the contents of the implicit multisample
buffer to be discarded include:

  • Drawing with the texture bound to an active texture unit
  • ReadPixels or CopyTex[Sub]Image* while the texture is
    attached to the framebuffer
  • CopyTex[Sub]Image*, Tex[Sub]Image*,
    CompressedTex[Sub]Image* with the specified level as
    destination
  • GenerateMipmap
  • Flush or Finish while the texture is attached to the
    framebuffer
  • BindFramebuffer while the texture is attached to the currently
    bound framebuffer.

Can we as the browser tell when this happens or prevent it? No and no:

  1. Should there be a way for applications to query if the multisample
    buffer has been resolved - and therefore is undefined?

    This may be useful if the operations that cause the multisample
    buffer to be resolved is allowed to vary between implementations.

RESOLVED: No, for two reasons: 1) This extension aims to be backwards
compatible with the IMG_multisampled_render_to_texture extension, which
did not include such a query, and 2) Given the resolution of issue 3 this
is not very useful as the application cannot control whether multisample
information is preserved or not.

  1. Should there be a way for applications to preserve the multisample
    buffer after a resolve?

This would be similar in spirit to the EGL_BUFFER_PRESERVED options in
EGL 1.4. Applications could - at a performance and memory cost - choose
to make the multisample buffer not undefined after a resolve.

RESOLVED: No. The purpose of this extension is to support multisampled
rendering in a lightway manner. Preserving the multisample buffer goes
against this intent.

Mid-render resolves cause shimmering artifacts between frames that are interrupted in different places. These artifacts can be acceptable for occasional frames, but it does violate portability of results. (It's these same artifacts that make it inviable to implement EXT_multiview_texture_multisample in terms of OVR_multiview_multisampled_render_to_texture)

It'll take some work in Firefox's WebGL implementation to ensure that we aren't incurring more implicit resolves than we have to, and even then, there will be times we can't protect from implicit resolves. (there are other OpenGL contexts rendering, after all!)

Can we use OVR_multiview_multisampled_render_to_texture to support multisampled multiview? Yes, but imperfectly, with important caveats.
Do we have much of a choice? Probably not, unless we're willing to forgo support on a lot of (most GLES) devices.

@RafaelCintron @kenrussell @grorg

@Artyom17

This comment has been minimized.

Copy link

commented Jul 20, 2019

Honestly, I think the fact that OVR_multiview_multisampled_render_to_texture drops unfinished buffers is actually a good thing: it clearly will indicate abuse of mid-frame flushes and will force devs to design more optimal experiences. The way how it is implemented in https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_multisampled_render_to_texture.txt is harmful, since it hides such design flaws as mid-frame flushes and produces sub-par quality of rendering.

@kenrussell

This comment has been minimized.

Copy link
Contributor

commented Jul 23, 2019

I think we should go down the route of building on top of OVR_multiview_multisampled_render_to_texture because it has the best likelihood of leaving the multisamples in chip memory on tiled architectures. Agree with @Artyom17 's assessment that eliminating shimmering may be something that developers will optimize for.

@jdarpinian spent a long time investigating and eliminating mid-frame resolves in Chrome on Android and, aside from poor application behavior, I'm optimistic that the auto-resolve solution can work well for VR/XR content in all web browsers.

@fernandojsg fernandojsg referenced this issue Jul 25, 2019
8 of 8 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.