Overview.bs

<h1>Filter Effects Module Level 2</h1>
<pre class='metadata'>
Status: ED
Work Status: Exploring
ED: https://drafts.fxtf.org/filter-effects-2/
TR: https://www.w3.org/TR/filter-effects-2/
Shortname: filter-effects
Level: 2
Link Defaults: css-transforms-1 (property) transform, svg (property) color-interpolation/fill/fill-opacity/fill-rule/stroke/glyph-orientation-horizontal/glyph-orientation-vertical/marker-start/marker-end/marker-mid/stop-color/stop-opacity/stroke-dasharray/stroke-dashoffset/stroke-linecap/stroke-linejoin/stroke-miterlimit/stroke-opacity/stroke-width/text-anchor/alignment-baseline/baseline-shift/dominant-baseline, css-masking-1 (property) clip-path/clip/clip-rule/mask, css-flexbox-1 (property) display, selectors-4 (type) <compound-selector>, css21 (type) <margin-width>, css-transforms-1 (type) <transform-function>, css-images-4 (property) image-rendering, css-color-3 (property) color, css-fonts-3 (property) font-family/font-stretch/font-style/font-variant/font-weight, selectors-4 (selector) :visited, css21 (dfn) containing block
Group: fxtf
Editor: Dean Jackson, Apple Inc. http://www.apple.com/, dino@apple.com, w3cid 42080
Editor: Chris Harrelson, Google Inc. http://www.google.com/, chrishtr@chromium.org, w3cid 90243
Former Editor: Dirk Schulze, Adobe Inc., dschulze@adobe.com, w3cid 51803
Test Suite: http://test.csswg.org/suites/filter-effects/nightly-unstable/
Abstract: Filter effects are a way of processing an element's rendering before it is displayed in the document. Typically, rendering an element via CSS or SVG can conceptually described as if the element, including its children, are drawn into a buffer (such as a raster image) and then that buffer is composited into the elements parent. Filters apply an effect before the compositing stage. Examples of such effects are blurring, changing color intensity and warping the image.
Abstract: This is Level 2 of the Filter Effects Module.
</pre>

<h2 id="intro">Introduction</h2>

<em>This section is non-normative</em>

A filter effect is a graphical operation that is applied to an element as it is
drawn into the document. It is an image-based effect, in that it takes zero or
more images as input, a number of parameters specific to the effect, and then
produces an image as output. The output image is either rendered into the
document instead of the original element, used as an input image to another
filter effect, or provided as a CSS image value.

This is Level 2 of the Filter Effects Module. It is currently written as
a "delta", describing any differences from Level 1.

Issue: This specification was drafted for discussion, and does not yet have Working Group consensus.
See <a href="https://github.com/w3c/fxtf-drafts/issues/53">discussion in issue 53</a>.


# Backdrop filters: the 'backdrop-filter' property # {#BackdropFilterProperty}

The description of the 'backdrop-filter' property is as follows:

<pre class='propdef'>
Name: backdrop-filter
Value: none | <<filter-value-list>>
Initial: none
Applies to: All elements. In SVG, it applies to <a>container elements</a> without the defs element and all graphics elements
Inherited: no
Percentages: n/a
Computed value: as specified
Media: visual
Animatable: yes
</pre>

If the value of the 'backdrop-filter' property is ''backdrop-filter/none'' then there is no
filter effect applied. Otherwise, the list of functions are applied in the
order provided.

## Filtering and Clipping ## {#backdrop-filter-operation}

An element (call it B) with a backdrop-filter property other than ''backdrop-filter/none'' is rendered as if
the following steps are performed:
1. Copy the <a>Backdrop Root Image</a> into a temporary buffer, such as a raster image. Call this buffer T’.
2. Apply the backdrop-filter’s filter operations to the entire contents of T'.
3. If element B has any transforms (between B and the Backdrop Root), apply the <b>inverse</b> of those transforms to the contents of T’.
4. Apply a clip to the contents of T’, using the border box of element B, including <a href="https://drafts.csswg.org/css-backgrounds-3/#border-radius">border-radius</a> if specified. Note that the children of B are not considered for the sizing or location of this clip.
5. Draw all of element B, including its background, border, and any children elements, into T’.
6. If element B has any transforms, effects, or clips, apply those to T’.
7. Composite the contents of T’ into element B's parent, using source-over compositing.

<pre class=link-defaults>
  spec:filter-effects-1; type:element; text:filter
</pre>

The first <a element>filter</a> function or filter reference in the <<filter-value-list>>
takes the element's <a>Backdrop Root Image</a> as the input image. Subsequent
operations take the output from the previous filter function or <a
element>filter</a> reference as the input image. The <a element>filter</a> element
reference functions can specify an alternate input, but still uses the previous
output as its SourceGraphic.

Filter functions must operate in the sRGB color space.

If the filter functions list includes a <a href="https://drafts.fxtf.org/filter-effects-1/#blurEquivalent">blur()</a>
filter, the filter will be applied with edgeMode="duplicate", with the edge defined by the clipped, transformed border
box of the element. See [[#BackdropRoot]].

Note: The effect of the backdrop-filter will not be visible unless some
portion of element B is semi-transparent. Also note that any opacity applied to
element B will be applied to the filtered backdrop image as well. Therefore, to
create a "transparent" element that allows the full filtered backdrop image to
be seen, you can use "background-color: transparent;".

A computed value of other than ''backdrop-filter/none'' results in the
creation of both a <a href="https://www.w3.org/TR/CSS21/zindex.html">stacking
context</a> [[!CSS21]] and a <a
href="https://developer.mozilla.org/en-US/docs/Web/CSS/Containing_block">Containing
Block</a> for absolute and fixed position descendants, unless the element it applies to
is a document root element in the current <a>browsing context</a>.

Note: This rule works in the same way as for the 'filter' property.

# Backdrop Root # {#BackdropRoot}

The <dfn>Backdrop Root Image</dfn> for an element E is the final image that would be produced by the following steps:
1. Start at the <a>Backdrop Root</a> element that is the nearest ancestor of E.
2. Paint all content, in <a href="https://www.w3.org/TR/CSS2/zindex.html#painting-order">painting order</a>, between (and including) the ancestor Backdrop Root element and element E.
3. Flatten the painted content into a 2D screen-space buffer.
4. Transform the border box of element E to 2D screen-space, and clip the final painted output to this border quad.

Issue: This specification does not yet have Working Group consensus, specifically on the definition of Backdrop Root.
See <a href="https://github.com/w3c/fxtf-drafts/issues/53">discussion in issue 53</a>.

Note: No content that is a DOM ancestor of the Backdrop Root element should
contribute to or affect the Backdrop Root Image.

A <dfn>Backdrop Root</dfn> is formed, anywhere in the document, by an element in any of the following scenarios. See [[#BackdropRootTriggers]] for more details on each:
 * The root element of the document (HTML).
 * An element with a filter property other than "none".
 * An element with an opacity value less than 1.
 * An element with mask, mask-image, mask-border, or clip-path properties with values other than “none”.
 * An element with a backdrop-filter value other than "none".
 * An element with a mix-blend-mode value other than "normal".
 * An element with a will-change value specifying any property that would create a Backdrop Root on non-initial value.

Note: this definition encompases fewer element types than the definition for a
Stacking Context. In particular, a Backdrop Root is not formed by elements with
z-index applied, fixed or sticky-positioned elements, and elements with
transforms applied. This allows elements with backdrop-filter or mix-blend-mode
to apply to elements higher up the DOM tree than would otherwise apply if they
stopped at the parent Stacking Context. For example, a container can be used to
contain elements with backdrop-filter applied, and that container can use
transforms for animation or positioning, while still allowing the
backdrop-filter to apply to the background behind the container.


## Motivation ## {#BackdropRootMotivation}

<em>This section is non-normative</em>

There are currently three related, but distinct, concepts in the web
platform:
 * The <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Positioning/Understanding_z_index/The_stacking_context">Stacking Context</a>. A Stacking Context is induced by many different types of element properties, and is primarily related to defining the painting (Z) order of elements.
 * The <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/Containing_block">Containing Block</a>. A Containing Block is also induced by several types of element properties, and is primarily related to defining how layout (X/Y) is performed for elements.
 * The <a href="https://drafts.csswg.org/css-transforms-2/#3d-rendering-contexts">3D Rendering Context</a>. A 3D Rendering Context is induced by several properties, mainly transform-style, but also any of the grouping properties, and is primarily related to defining the relevant coordinate space for 3D transforms.

There is an important motivation for the creation of a separate web platform
concept for the Backdrop Root, rather than just re-using an existing concept.
There are essentially two other choices that might be used as a definition of
the "backdrop": 1) “everything that painted before, up to the root node”, or 2)
“everything up to the parent stacking context”. However, there are ambiguities
with definition #1, and excessive limitations with definition #2. Therefore, the
Backdrop Root concept strikes a balance between the two.

In particular, it is important to note that including “everything that paints
before” an element is ambiguous in the case where an ancestor of the element
contains filtering and/or opacity. Because those effects are inherited by
descendant elements, including the element containing backdrop-filter or
mix-blend-mode, it is not clear “when” to apply the effect. Filters and opacity
create a stacking context, and the <a
href="https://drafts.fxtf.org/filter-effects/#FilterProperty">css filter-effect
specification</a> states that “All the [Stacking Context] descendants are
rendered together as a group with the filter effect applied to the group as a
whole.” If, somewhere inside that stacking context, an element contains a
backdrop-filter property other than “none”, then it is impossible to
honor the preceding sentence. At the point of the backdrop-filtered element, all
of the (partially-painted) contents of the stacking context need to be fully
rendered, including applied filters, opacity, and blending with the backdrop,
and the resulting image needs to be used as the input to the backdrop-filtered
element. That fundamentally breaks the atomicity of the stacking context. And the ambiguity
arises from the need, at the end of rendering the fully-completed stacking
context, to apply those filters and opacity <b>again</b> to the completed rendering.
The filters and opacity will, at this point, be applied twice to the portion of
the image that has been backdrop-filtered. This situation grows exponentially
worse if backdrop-filters are nested inside each other. For this reason, it is
necessary to prevent the backdrop-filter from “seeing" above nodes that have
filters, opacity, and the other conditions listed above in the definition of the
Backdrop Root.

Performance would also be an issue, if backdrop-filter and mix-blend-mode were
defined to filter "everything" that comes before them on the page. Each
application of backdrop-filter or mix-blend-mode would require a separate
rendering pass, to temporarily finish any partially-complete stacking contexts,
and get the "final" output that would appear behind each element. That would
double the required rendering time, and would potentially require twice the
memory usage and GPU bandwidth to store the intermediate graphics texture
holding the contents to be filtering. And assuming nested backdrop-filters or
mix-blend-mode elements were allowed, this doubling would become an exponential
performance breakdown. Clearly, this is not a scalable approach.

<div class="example">
<p>Given the following html code:</p>
<pre><code highlight=html>
&lt;html style="background:lightgrey;"&gt;
  &lt;div class="box"&gt;
    &lt;b&gt;"Box"&lt;/b&gt;&lt;br&gt;&lt;br&gt;Lorem ipsum dolor sit amet, consectetur adipiscing elit,
    sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim
    ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip
    ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate
    velit esse cillum
    &lt;div class="dialog"&gt;&lt;/div&gt;
  &lt;/div&gt;
&lt;/html&gt;
</code></pre>
<p>And the following style rules:</p>
<pre><code highlight=css>
&lt;style&gt;
  div {
    position: absolute;
    width: 200px;
    height: 200px;
    top: 70px;
    left: 70px;
  }
  .box {
    background: white;
    filter: invert(1);
  }
  .dialog {
    border: 10px solid blue;
    backdrop-filter: blur(2px);
  }
&lt;/style&gt;
</code></pre>
<p>Consider the sequence of painting operations to produce the final result,
   assuming that the "dialog" element accesses <b>everything behind it</b>, all
   the way to the root element. <b>Important note:</b> this is <b>not</b> how
   the Backdrop Root is defined - this example is intended to illustrate why a
   Backdrop Root is formed by elements containing filters.</p>
<div class="figure">
  <img alt="ALT TEXT HERE" src="examples/step1.png" style="width:30%"/>
  <p class="caption"><b>Step 1.</b> "Box" is rendered, but filters are not yet applied.</p>
  <img alt="ALT TEXT HERE" src="examples/step2.png" style="width:30%"/>
  <p class="caption"><b>Step 2.</b> "Dialog" is about to be rendered, but it has
    backdrop-filter applied. Since, for this example, the backdrop-filter "sees"
    all the way to the root, the "Box" element needs to be temporarily
    "completed" so that the final product can be used as input to "Dialog". This
    means applying its "filter: invert(1)" style. Note, this is where the
    atomicity of the Stacking Context created by "Box" is broken. This is also
    where the performance penalty applies - the GPU draw work that is
    performed here will end up being done twice.</p>
  <img alt="ALT TEXT HERE" src="examples/step3.png" style="width:30%"/>
  <p class="caption"><b>Step 3.</b> Start painting the "Dialog" element by
    reading back its backdrop image, applying the "backdrop-filter: blur(2px)"
    filter, and cropping those results to the border box of "Dialog". (The
    dotted black border has been added here for clarity: this is the border box
    for the "Dialog" element.)</p>
  <img alt="ALT TEXT HERE" src="examples/step4.png" style="width:30%"/>
  <p class="caption"><b>Step 4.</b> Now that the backdrop-filtered contents have
    been read back and filtered, discard the previously "completed" Box element
    and go back to the unfiltered version. Note that you can now see the black
    text at the bottom bleeding through the white, blurred version inside
    "Dialog".</p>
  <img alt="ALT TEXT HERE" src="examples/step5.png" style="width:30%"/>
  <p class="caption"><b>Step 5.</b> Draw the contents of the "Dialog" element -
    the <b>blue</b> border.</p>
  <img alt="ALT TEXT HERE" src="examples/step6.png" style="width:30%"/>
  <p class="caption"><b>Step 6.</b> Complete the Stacking Context formed by the
    "Box" element - apply the "filter: invert(1)" to "Box" and all of its
    contents, including the now-completed "Dialog" element. Note that the area
    inside the yellow "Dialog" element has been inverted twice - it appears
    white. Also note that the text outside "Box" has been drawn in blurred black
    text over the top of un-blurred white text, and the black edge of "Box"
    shows through the now-white blurred edge of "Box". None of these visual
    effects is expected or intuitive.</p>
</div>
</div>

<div class="example">
<p>Opacity poses much the same problem that filter does. Using the example
  above, but with the <code>filter: invert(1);</code> style replaced with
  <code>opacity: 0.5;</code> the results are similarly unintuitive.
  <b>Important note:</b> the left image below does <b>not</b> represent how the
  Backdrop Root is defined - this example is intended to illustrate why a
  Backdrop Root is formed by elements containing opacity less than 1.</p></p>
<div class="figure">
  <img alt="ALT TEXT HERE" src="examples/opacity.png" style="width:30%"/>
  &nbsp;&nbsp;&nbsp;&nbsp;
  <img alt="ALT TEXT HERE" src="examples/opacity_correct.png" style="width:30%"/>
  <p class="caption">The left image is the final result of rendering, assuming
  the Backdrop Root includes everything on the page. Note that the blurred
  region inside the blue border is very faint. It is rendered at 0.25 opacity
  (double-counted), instead of the 0.5 opacity inherited from the "Box" element.
  The right image is the final result using the correct definition of Backdrop
  Root. The blurred region inside the blue border has opacity 0.5, as
  expected.</p>
</div>
</div>

<div class="example">
<p>Because masks effectively change the opacity of parts of the elements they
  are masking, the behavior for mask operations is quite similar to that for
  elements with opacity applied. Using the same example as above, but replacing
  the filter/opacity style line with <code>-webkit-mask-image:
  linear-gradient(to top, transparent 25%, black 75%);</code> the results are
  quite similar. The masked area is applied twice, meaning the alpha channel is
  effectively squared. <b>Important note:</b> the left image below does
  <b>not</b> represent how the Backdrop Root is defined - this example is
  intended to illustrate why a Backdrop Root is formed by elements containing
  masks.</p>
<div class="figure">
  <img alt="ALT TEXT HERE" src="examples/mask.png" style="width:30%"/>
  &nbsp;&nbsp;&nbsp;&nbsp;
  <img alt="ALT TEXT HERE" src="examples/mask_correct.png" style="width:30%"/>
  <p class="caption">The left image is the final result of rendering, assuming
  the Backdrop Root includes everything on the page. Note that the gradient
  applied to the blurred region inside the blue border (but not the border
  itself) is very faint at the bottom. The opacity of the mask has been applied
  twice, so the gradient is no longer a linear gradient, but is instead a
  quadratic gradient. The right image is the final result using the correct
  definition of Backdrop Root. The mask is applied correctly as a linear
  gradient for both the blue border and the blurred region inside.</p>
</div>
</div>

## Backdrop Root Triggers ## {#BackdropRootTriggers}

<em>This section is non-normative</em>

Issue: This specification does not yet have Working Group consensus, specifically on the definition of Backdrop Root.
See <a href="https://github.com/w3c/fxtf-drafts/issues/53">discussion in issue 53</a>.

As described in [[#BackdropRootMotivation]], several operations pose fundamental problems for operations that need to read back content that was painted before, and then re-paint that content (possibly filtered or blended) again. The list of element types that trigger a [[#BackdropRoot]] each pose one of these problems. Some are obvious and some are more nuanced. This section describes why each trigger is necessary.

Obvious triggers:
 * <b>The root element.</b> There is nothing above the root element, so it must form a Backdrop Root.
 * <b>Will-change values.</b> The will-change hint means that the element might take on a value that creates a Backdrop Root. For this reason, the Backdrop Root should be created immediately, to avoid changing it (and therefore changing the rendered appearance) during animation.

Less obvious triggers:
 * <b>An element with a filter.</b> Filters apply to an element and all of its children, rendered together. Allowing the Backdrop Root to read back content above a filtered element necessitates breaking the atomicity of the filter, with undefined results. See [[#BackdropRootMotivation]] for more a much more detailed explanation and some examples.
 * <b>An element with opacity < 1.</b> Elements with an opacity value less than 1.0 suffer from a very similar problem as elements with filters. Because the opacity applies to the element and all children, rendered together, the opacity value would be "applied twice" if the Backdrop Root Image were to read back content above the semi-transparent element. Effectively, the backdrop-filtered portion of the backdrop would be rendered with (Opacity)^2. See [[#BackdropRootMotivation]] for more details and an example.
 * <b>CSS Masks.</b> There are two reasons why a CSS mask triggers a Backdrop Root element. The first is that masks are exactly analogous to the opacity problem described in the bullet above. Masks can contain regions of partial opacity (e.g. <code>style="mask-image: linear-gradient(to bottom, transparent 25%, black 75%);"</code>). If the Backdrop Root Image were to read back content above the mask, those partially-opaque regions would be double-applied to the final backdrop image, which would result in incorrect rendering. See [[#BackdropRootMotivation]] for more details and an example.
 * <b>Elements with clip-path.</b> These elements are effectively rendered as if they were a mask. While in general that mask doesn't explicitly contain partially-opaque regions, it is usually rendered with antialiasing, which introduces semi-transparent portions around the edges of the clip. Therefore, for the same reasons as described above for masks, those semi-transparent regions will be incorrectly rendered, producing visual artifacts near the edges of the clip. Disabling antialiasing for clip-paths would produce poor rendering quality, and relaxing the Backdrop Root restriction on these elements would similarly lead to observable artifacts.
 * <b>Backdrop-filter and mix-blend-mode.</b> These elements themselves must form a Backdrop Root for two reasons. First, they cause the same problems that filters and opacity cause: there is an ambiguity about what constitutes the image to be filtered or blended, given the atomicity of the operations. Second, there would likely be an exponential performance degradation in the case of nested backdrop-filter or mix-blend-mode elements, due to the need to re-paint the content behind each element. Each nesting level will double the number of these required re-paint cycles, leading to significant performance problems.

For all of the above triggers, performance is also an important motivation. In all of these cases, relaxing the Backdrop Root constraint would lead to a potential doubling of the CPU/GPU memory and bandwidth. See [[#BackdropRootMotivation]] for a more detailed discussion.

## Mix Blend Mode ## {#mix-blend-mode}

<em>This section is non-normative</em>

The current definition of mix-blend-mode <a href="https://www.w3.org/TR/compositing-1/#csscompositingrules_CSS">defines</a> the input (backdrop) image for mix-blend-mode as being all of the underlying content of the parent stacking context. This definition could likely be relaxed to use the Backdrop Root definition instead, allowing more elements to be blended.


<h2 id="priv-sec">Privacy and Security Considerations</h2>

All of the same privacy and security concerns exist for backdrop-filter as do for "standard" filters. See [[filter-effects#tainted-filter-primitives]] for more details.

<h2 class=no-num id='acknowledgments'>Acknowledgments</h2>

The editors would like to thank Mason Freed, Marcus Stange, Matt Rakow, Simon Fraser, Amelia Bellamy-Royds, Dirk Schulze, and Tab Atkins for their careful reviews, comments, and corrections.