Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[filter-effects] Browsers don't match spec for feDisplacementMap #113

Open
AmeliaBR opened this issue Feb 25, 2017 · 19 comments
Open

[filter-effects] Browsers don't match spec for feDisplacementMap #113

AmeliaBR opened this issue Feb 25, 2017 · 19 comments

Comments

@AmeliaBR
Copy link

@AmeliaBR AmeliaBR commented Feb 25, 2017

The spec for feDisplacementMap says:

This filter primitive uses the pixels values from the image from in2 to spatially displace the image from in. This is the transformation to be performed:

P'(x,y) ← P( x + scale * (XC(x,y) - .5), y + scale * (YC(x,y) - .5))

where P(x,y) is the input image, in, and P'(x,y) is the destination. XC(x,y) and YC(x,y) are the component values of the channel designated by the xChannelSelector and yChannelSelector. For example, to use the R component of in2 to control displacement in x and the G component of Image2 to control displacement in y, set xChannelSelector to "R" and yChannelSelector to "G".

The displacement map, in2, defines the inverse of the mapping performed.

The input image in is to remain premultiplied for this filter primitive. The calculations using the pixel values from in2 are performed using non-premultiplied color values.

If I'm reading that correctly, I would expect that, when xChannelSelector and yChannelSelector are R,G,or B:

  • If the input map is solid white, the filtered graphic would be displaced in negative x&y directions by 0.5*scale.
  • If the input map is solid black, the filtered graphic would be displaced in a positive x&y directions by 0.5*scale.
  • If the input map is solid 50% gray (#888 if color-interpolation-filters is linearRGB), the filtered graphic would not be displaced at all.

If the input map is partially transparent, I'm not quite sure what should happen next. It depends on whether "premultiplied" only applies to scaling the color channel range, or whether it also includes scaling by alpha. If we do not scale the color channels by the alpha, then the alpha value should not have any effect unless the value of xChannelSelector or yChannelSelector is A. This is what I would expect as an author.

If you do scale the color channels by alpha, which I think is a more literal reading of the spec, then I would expect:

  • 50% opaque white to have the same effect as 50% solid gray, i.e. no displacement.
  • 50% opaque/50% gray input to have the same effect as 25% solid gray, aka positive displacement by 0.25*scale.
  • fully transparent (0% opaque) input to have the same effect as solid black, i.e. positive displacement by 0.5*scale.

None of the browsers tested (Chrome 56, Firefox 53, and Edge 14) lead to any of these sets of expected results.

Solid black and white input are treated as expected in all browsers: displacement by half the scale factor, in opposing directions. But a 50% gray input in all browsers does displace the image, halfway the distance of the displacement for a solid black input.

Chrome and Firefox treat a 50% transparent input the same as a solid input of the same color, but treat a completely transparent input the same as solid black. MS Edge linearly scales the amount of displacement by the alpha of the input color.

CodePen test case

@dirkschulze
Copy link
Contributor

@dirkschulze dirkschulze commented Dec 28, 2017

@AmeliaBR I didn't look at the spec text yet. For me it seems that Firefox and WebKit behave the same while Chromium doesn't displace anything. I have no comparison to Edge.

@AmeliaBR
Copy link
Author

@AmeliaBR AmeliaBR commented Dec 28, 2017

Screenshots of the CodePen on Windows 10, for comparison.

Chrome 63:
image

EdgeHTML 16:
image

Firefox 58:
image

As explained in the pen description:

The dark red bar shows the default position of the test green bar, the light red bars are offsets by +/- 10 or 20 units.

The feDisplacementMap elements have a scale of 40, which should mean maximum displacement of +/- 20 in each direction.

The feDisplacementMap elements use the G and B channels, which should (I hope?) mean they are unaffected by the alpha of the input.

Each row has a different input color (white, 50% gray, black). Each column has a different input opacity (solid, 0.5, fully transparent).

@dirkschulze
Copy link
Contributor

@dirkschulze dirkschulze commented Dec 28, 2017

So WebKit, Chrome and Firefox come to the same result.

Here the examples of

Adobe Photoshop:
fedisplacementmap-ps

Adobe Illustrator:
fedisplacementmap-ai

Note that Adobe Illustrator has some clipping and position issues with nested <svg> elements. The actual result should look like in Adobe Photoshop.

Here the pure SVG code that I used. @Tavmjong could you add the results of Inkscape please?

<svg width="600" height="600" viewBox="0 0 300 300" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<style>
* {
stroke-width: 10;
}
.ref {stroke: tomato; fill: none;}
.base-position {stroke: darkRed; }
.test {stroke: forestGreen; fill: none;}
</style>
<svg viewBox="0 0 100 100" width="100" height="100">
  <defs>
    <path id="p" d="M30,70L70,30" />
    <g id="refs" class="ref">
      <use xlink:href="#p" transform="translate(-20,-20)"/>
      <use xlink:href="#p" transform="translate(-10,-10)"/>
      <use xlink:href="#p" class="base-position"/>
      <use xlink:href="#p" transform="translate(10,10)"/>
      <use xlink:href="#p" transform="translate(20,20)"/>
    </g>
  </defs>
  <filter id="solid-white" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#fff"
             flood-opacity="1" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test" 
       filter="url(#solid-white)"/>
</svg>

<svg viewBox="0 0 100 100" x="100" width="100" height="100">
  <filter id="half-white" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#fff"
             flood-opacity="0.5" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#half-white)"/>
</svg>

<svg viewBox="0 0 100 100" x="200" width="100" height="100">
  <filter id="transparent-white" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#fff"
             flood-opacity="0" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#transparent-white)"/>
</svg>

<svg viewBox="0 0 100 100" transform="translate(0,100)" y="100" width="100" height="100">
  <filter id="solid-gray" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#888"
             flood-opacity="1" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#solid-gray)"/>
</svg>

<svg viewBox="0 0 100 100" x="100" y="100" width="100" height="100">
  <filter id="half-gray" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#888"
             flood-opacity="0.5" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#half-gray)"/>
</svg>

<svg viewBox="0 0 100 100" x="200" y="100" width="100" height="100">
  <filter id="transparent-gray" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#888"
             flood-opacity="0" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#transparent-gray)"/>
</svg>

<svg viewBox="0 0 100 100" y="200" width="100" height="100">
  <filter id="solid-black" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#000"
             flood-opacity="1" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#solid-black)"/>
</svg>

<svg viewBox="0 0 100 100" x="100" y="200" width="100" height="100">
  <filter id="half-black" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#000"
             flood-opacity="0.5" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#half-black)"/>
</svg>

<svg viewBox="0 0 100 100" x="200" y="200" width="100" height="100">
  <filter id="transparent-black" filterUnits="userSpaceOnUse">
    <feFlood flood-color="#000"
             flood-opacity="0" />
    <feDisplacementMap xChannelSelector="G" yChannelSelector="B" scale="40" in="SourceGraphic" />
  </filter>
  <use xlink:href="#refs" />
  <use xlink:href="#p" class="test"
       filter="url(#transparent-black)"/>
</svg>
</svg>
@AmeliaBR
Copy link
Author

@AmeliaBR AmeliaBR commented Dec 28, 2017

@dirkschulze @Tavmjong

It looks like Inkscape (at least, the stable 0.92.1 that I have) doesn't support filterUnits="userSpaceOnUse"; the green line just disappears.

To make the test work, I took Dirk's version and replaced every instance of filterUnits="userSpaceOnUse" with x="-5" y="-5" width="12" height="12" (PS, thanks for converting it to a single SVG test file, Dirk!). Here's the result (consistent with Chrome/Firefox):

fedisplacement-test

@dirkschulze
Copy link
Contributor

@dirkschulze dirkschulze commented Dec 31, 2017

@AmeliaBR To summarize your confusion with the spec text: You are asking if implementations should use premultiplied colors or non-premultiplied colors when selecting the channels R, G or B?

The introduction (https://drafts.fxtf.org/filter-effects/#FilterPrimitivesOverviewIntro) states that all primitives operate with premultiplied colors unless stated differently.

feDisplacementMap actually defines it pretty clear:

The input image in is to remain premultiplied for this filter primitive. The calculations using the pixel values from in2 are performed using non-premultiplied color values.

So no scaling by alpha from in2, the displacement map, for R, G or B channel. If implementations do so, then they are in mistake. Quite frankly, I personally would not think that we should change the specification text here. Using premultiplied colors from the displacement map is logically incorrect.

What we should do is writing tests (which you did) and report issues to browser vendors.

Edit: Ergo, your expectations are correct that gray should not displace the pixel of in according to the formula regardless of the alpha channels value.

@dirkschulze
Copy link
Contributor

@dirkschulze dirkschulze commented Dec 31, 2017

@AmeliaBR Note: colors with 0 alpha is a special case. In your example, feFlood uses flood-color="#888" flood-opacity="0" so you know that you have a gray flooded area.

That is not what the implementations see. If we assume premultiplied colors, then all colors with opacity of 0 result in a color value of 0. There is no way to compute the original unpremuliplied color anymore.

So your observation that for 0 alpha all implementations are operating on transparent black makes sense unless filter-effects would require to preserve the unpremultiplied color values of the input image which it currently doesn't. It just operates on un-premultiplied color values which means an implementation needs to convert premultiplied color values back to unpremultiplied color values.

Edit: This also explains the difference from Photoshop to browser implementations which does preserve un-premultiplied colors as much as possible here.

@dirkschulze
Copy link
Contributor

@dirkschulze dirkschulze commented Dec 31, 2017

Adding @tabatkins @svgeesus for their input: What do you think about specifying that all color values default to 0 when then alpha channel is 0 for filter primitive inputs? This allows implementations to store intermediate results in either premultiplied or un-premultiplied colors and still have consistent output regardless of the implementation details.

@AmeliaBR
Copy link
Author

@AmeliaBR AmeliaBR commented Dec 31, 2017

So your observation that for 0 alpha all implementations are operating on transparent black makes sense unless filter-effects would require to preserve the unpremultiplied color values of the input image which it currently doesn't. It just operates on un-premultiplied color values which means an implementation needs to convert premultiplied color values back to unpremultiplied color values.

That is consistent with other problems I've discovered with feTurbulence output.
Because browsers ignore the color channel of feTurbulence when alpha is zero, you end up with black patches where you lose all the random data from the other channels, no matter how much you try to manipulate the image later.

@AmeliaBR
Copy link
Author

@AmeliaBR AmeliaBR commented Dec 31, 2017

So, basically there are 2 distinct issues here:

  • Should implementations preserve color values of transparent pixels for filter primitive results? I've separated this out as #243

  • Why are browsers using non-linear and inconsistent conversions between color channel value and displacement (when color-interpolation-filters is the default linearRBG)? Is there any benefit in spec'ing the most common implementation? Otherwise, how can the text be clarified so that we can converge on consistent behavior?

@dirkschulze
Copy link
Contributor

@dirkschulze dirkschulze commented Jan 2, 2018

@AmeliaBR @smfr started investigating in color space issues with regards to SVG filters in WebKit.

@dirkschulze
Copy link
Contributor

@dirkschulze dirkschulze commented Nov 15, 2018

@smfr Since you did a lot of changes/fixes with regard for linearRGB in WebKit, any comment on the above?

@svgeesus
Copy link
Contributor

@svgeesus svgeesus commented Dec 14, 2018

@AmeliaBR wrote:

Because browsers ignore the color channel of feTurbulence when alpha is zero, you end up with black patches where you lose all the random data from the other channels, no matter how much you try to manipulate the image later.

Loosing color precision is a known issue with premultiplied alpha; at the limit, an alpha of zero looses all precision and the original color is completely gone.

This is unavoidable when only the premultiplied value is stored. It is avoidable and somewhat tragic, if the unpremultiplied values (separate alpha) are stored, but then "for consistency" an alpha of 0/255 results in the color channels being knocked out to black (while at an alpha of 1/255, the color channels are left as-is. IIRC this is what Photoshop does, and it bugs me).

@AmeliaBR
Copy link
Author

@AmeliaBR AmeliaBR commented Dec 15, 2018

@svgeesus Agreed, and there's no easy solution. But that's also a separate issue from the main problem being discussed here, so I pulled it out into #243

@faceless2
Copy link

@faceless2 faceless2 commented Mar 23, 2019

The premultiplied alpha question above has been covered by other commenters, but no-one has voiced any thoughts on the colorspace issue which I think is more interesting.

A few observations:

  • If you want 50% gray in the input it should be #808080 not #888 (which expands to #888888)

  • All of the implementations described above are converting the "displacement" input in "in2" to sRGB. This appears to be the right thing to do when the displacement comes from an feImage, as in filters-displace-01-f.html - because images created by feImage are (?) always sRGB.

Amelia, you asked whether the spec should change to match current behaviour, or whether there was anything that could be clarified.

I can think of 3 options.

  1. Require the displacement input into feDisplacementMap to be sRGB. This will work if the input is an image, but if you want it to work for anything else, like feFlood in your example, this becomes a problem. Most of the primitives work best in linearRGB, and a conversion to sRGB would make it harder to get an exact color like #808080.

  2. Allow the feImage primitive to define the color-space of the image. sRGB, as it is now, is ideal for images destined for display, but feDisplacementMap uses the color-channels of an image effectively as a data channel (I believe it's the only primitive that does). It makes a lot of sense here to have #808080 in a bitmap image mean "no displacement". Allowing feImage to define which space the image is loaded in covers all the bases. But it would mean any existing SVGs defining an feImage as input to an feDisplacementMap would suddenly become incorrect.

  3. Explicitly state that the displacement input into feDisplacementMap doesn't care about the colorspace of the input. Regardless of whether the input is linear (as in the feFlood example above) or sRGB (when it's loaded from an image), if it contains #808080 then there's no shift. This means color-interpolation-filters is ignored by feDisplacementMap.

On balance I'd go for the last option. feDisplacement is unique in using a color channel for something other than color, and applying any sort of gamma curve to this input makes no sense. So far I haven't thought of a case where this will do the wrong thing.

The other benefit is it means that the behaviour of all browsers when the feDisplacementMap input is an image continues to work. The only change required is to remove any color conversion on the input loaded from "in2" - if the input is an image this has no effect, and if the input is not, well that's the case that isn't working now.

Hope that makes sense.

@faceless2
Copy link

@faceless2 faceless2 commented Apr 29, 2019

To briefly follow myself up:

I was digging through the "filters-displace-01-f.html" referenced above, and the PNG images used as input have a gAMA block, which alters the ColorSpace from sRGB. If this were respected it would change the position of the "bump" in the displacement map.

It's not respected - an embedded ColorSpace in an image used for displacement is ignored by all current implementations. Further indication that option 3, ignoring the colorspace completely, is the right thing to do.

@svgeesus
Copy link
Contributor

@svgeesus svgeesus commented May 1, 2019

Explicitly state that the displacement input into feDisplacementMap doesn't care about the colorspace of the input. Regardless of whether the input is linear (as in the feFlood example above) or sRGB (when it's loaded from an image), if it contains #808080 then there's no shift. This means color-interpolation-filters is ignored by feDisplacementMap.

Yes. The data in the color channels for the in2 image is not treated as a color. It is treated as a 0 to 255 displacement amount. Thus alpha is ignored gamma and chromaticity and ICC profiles are ignored, because the raw image data is not being interpreted as a color.

The spec already states this regarding alpha, but could state the rest more clearly.

@faceless2
Copy link

@faceless2 faceless2 commented May 1, 2019

The feDisplacement map description states "The color-interpolation-filters property only applies to the in2 source image ...". To me, that implies that the intention is that "in2" is color corrected, to sRGB or linearRGB depending on the value of that attribute.

@svgeesus
Copy link
Contributor

@svgeesus svgeesus commented May 1, 2019

Huh. That seems wrong to me, and also doesn't match implementations, and should be corrected. I wonder if that text was always there or arrived more recently.

@MeirionHughes
Copy link

@MeirionHughes MeirionHughes commented Jul 20, 2020

I just want to chime in with a related issue I had with the current browsers. I attempted to populate an feImage to use as the displacement source, with a URL encoded copy of a local canvas buffer. I had the expectation that 128/256 - 0.5 = 0... but it wasn't. the mid-point was closer to 187.5; could be useful for future standards to make feImage and/or feDisplacementMap input source-generation bullet-proof - preferably too with the ability of feDisplacementMap to have a high-byte selector too i.e. dX = (R + B)/(2 << 16) - 0.5 as frankly the displacement map's current resolution is really poor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.