Post processing doesn't work correctly when using multiple viewports

[IceSentry]

Bevy version

0.9 and main

What you did

In the split_screen example:

• Enable hdr on both cameras (enabling hdr only on one also breaks bevy, but this is a separate issue).
• Add BloomSettings::default() to the left camera
• Use an emissive color for the plane

I would have expected the left viewport to have bloom applied to it.

What went wrong

The bloom effect does not appear in the left viewport.

As you can see here, the bloom effect is indeed computed and generated correctly, but once it gets to the final upscaling pass, the texture modified by the post process is not used as an input.

I haven't investigated more than that, but it seems like the upscaling node is using the wrong texture when using multiple viewports combined with post processing.

[rparrett]

related to #7361

[IceSentry]

It might be related, but this happens with any post processing effect, not just bloom. Bloom is simply the easiest to test. For example, if I enable bloom on the right camera instead, it works as expected where only one camera has bloom applied.

[Elabajaba]

Multiple cameras with multiple windows work fine (where each window has its own camera), but multiple cameras that composite to the same window don't.

[Elabajaba]

It might be related, but this happens with any post processing effect, not just bloom. Bloom is simply the easiest to test. For example, if I enable bloom on the right camera instead, it works as expected where only one camera has bloom applied.

From what I can tell, it only works for bloom because bloom works on a separate set of textures that are limited to the view.

Fxaa (and my CAS pr) use the main_texture and end up applying to everything on screen instead of only the right camera's viewport.

[Elabajaba]

Debugging it on discord with @robtfm, this is a bug with Msaa. In the 2nd pass it's reading the main_texture_sampled from the previous pass, which doesn't have any post processing applied.

Disabling Msaa also revealed another bug, where because it now relies on the previous main_texture_a the output is reliant on what the last pass that wrote to main_texture_a was. With only bloom, it works fine (by accident), but add Fxaa and you end up with a double tonemapped final output because main_texture_a's was tonemapped before it started rendering the 2nd viewport.

[IceSentry]

From what I can tell, it only works for bloom because bloom works on a separate set of textures that are limited to the view.

Right, the other issue I had was for a post process effect that uses multiple textures like bloom does so I mistakenly assumed it was an issue with all post processing.

[robtfm]

there's a few post-processing issues compounding one another here. i think if we do the below, then almost everything works as intended:

• fxaa and tonemapping need to set viewport
  • currently they read/write full-screen and ignore viewports
• cameras that are not LAST need to blit their final result back to the base target (either the multisampled texture or main_texture_a depending on MSAA). this can be skipped if the final output is already to that target (i.e. if last write was to main_texture_a and MSAA is disabled)
  • currently we do not copy back, so the last postprocessing step may be ignored (if msaa is off and the last write was to main_texture_b), or all postprocessing may be ignored (if msaa is on)
• cameras that are not FIRST need to reverse-tonemap within their viewports
  • currently this is not done at all, and leads to darkening from multiple tonemap applications for overlapping viewports

the above still doesn't solve problems with overlapping non-destructive post-processing cameras (i.e. cameras that write to previously written pixels and don't overwrite every pixel in their viewport, then apply postprocessing). bloom (e.g.) applied to these cameras will affect previous camera's pixels as well because they are unavoidably there in the bloom input texture. you can avoid that in this particular case by running the bloom cameras first, but there's no general solution while rendering to the same texture.

the full solution for that is rendering to multiple texture targets and composing, which is supported already but requires a bit of extra work (see the render-to-texture example).

[robtfm]

one more, though it doesn't matter in the core pipeline and i believe is already addressed by the bloom rework:

• bloom needs to source from the current post_processing source texture
  • currently it builds the bindgroup in prepare and assumes that the source will be main_texture_a. if other post-processing steps are added before bloom it would potentially discard the last step prior to it

[Keelar]

Could this also be related to #5721? Seems like there are some similarities, as they are both influenced by MSAA and camera rendering order. Apologies if they're unrelated. I'm not too familiar how the engine works at a low level yet but some of the discussion in here reminded me of that issue so I thought it was worth bringing up.