Bad subpixel AA inside stacking context for mask

[mstange]

https://bugzilla.mozilla.org/show_bug.cgi?id=1454097

Now that we're using stacking contexts for masks (bug 1412375), if you go to a site with a long URL, like this one, the text in the URL bar will look funny. Firefox applies a fade-out mask effect to long URLs.

I think this is happening because we're incorrectly allowing the use of subpixel AA inside the intermediate surface for the stacking context, even though we're drawing onto transparency.

We should do the following:

1. Force grayscale AA inside stacking contexts unless we know that the text is being drawn onto opaque pixels within that stacking context.
2. As a future optimization and quality improvement, detect the case where the grouping provided by the stacking context is unnecessary, and discard the intermediate surface and apply the mask to the contents directly. This would give us the previous appearance for the URL bar.

[mrobinson]

@mstange Do you mind expanding on why the grouping is unnecessary here?

[glennw]

I think the problem is:

webrender/webrender/src/display_list_flattener.rs

Line 1274 in 1875203

let allow_subpixel_aa = composite_ops.count() == 0 &&

If this becomes:

let allow_subpixel_aa = composite_mode.is_none();

Then I think this will fix the problem.

The test itself is still super-conservative - in addition to the optimization mentioned in (2) above, we'll also want to detect cases where we have an intermediate surface but we know the background is opaque.

We don't need to do this now, but we should do it in the future to allow subpixel AA in more cases than we currently do. Detecting (what I expect to be) the common case for this should be really simple - we just determine when checking the visibility of a picture if the local rect of any solid rects in this picture cover the entire local rect of the picture. This would mean that it's safe to use subpixel AA (and could also be used as a cheap occlusion cull for any primitives that come before such a rect in this picture).

Anyway, hopefully the one line fix mentioned above should be enough to fix the reported bug here. I can try it next week if no-one else gets to it first.

[Gankra]

@mrobinson I believe grouping is unnecessary in this case because there's only one primitive that's getting masked (a textrun). Potentially an easy case to notice in gecko.

[mstange]

That's right. Another case where grouping is unnecessary is when you have multiple primitives but they don't overlap; but that case is probably a lot harder to detect.

[Gankra]

Confirmed glenn's fix seems to work (although I'm on a high-dpi display so it's really hard to tell if it's causing too much greyscale AA). Just need to figure out a good regression test...

[glennw]

Fixed by #2662.