Skip to content
Permalink
Browse files
CoreImage Implementation of SourceGraphic and saturate(), hue-rotate(…
…), grayscale() and sepia()

https://bugs.webkit.org/show_bug.cgi?id=213673

Patch by Frank Yang <guowei_yang@apple.com> on 2020-09-03
Reviewed by Darin Adler, Simon Fraser.

Source/WebCore:

Tests: css3/filters/effect-grayscale-square.html
       css3/filters/effect-hue-rotate-square.html
       css3/filters/effect-saturate-square.html
       css3/filters/effect-sepia-square.html

Added CoreImage implementation of SourceGraphic and partial of FEColorMatrix,
which is used to implement sepia(), grayscale() hue-rotate() and saturate().
Also added CIFilter caching, by storing CIFilters in a hash map, where the key
is the level of traversal (an integer).

* platform/graphics/coreimage/FilterEffectRendererCoreImage.h:
    Imported FEColorMatrix.h and SourceGraphic.h; added function getCIContext()
    that returns a static copy of CIContext.
* platform/graphics/coreimage/FilterEffectRendererCoreImage.mm:
(WebCore::FilterEffectRendererCoreImage::getCIContext): Creating a
    CIContext is expensive. Instead of creating a CIContext on each frame
    update, we just use a static singleton on every draw
(WebCore::FilterEffectRendererCoreImage::supportsCoreImageRendering):
    the function now returns true if the filter chain contains SourceGraphic
    and FEColorMatrix of TYPE_MATRIX, TYPE_SATURATE and TYPE_HUEROTATE
(WebCore::FilterEffectRendererCoreImage::connectCIFilters):
    Made slight changes to wrap Obj-C pointers in RetainPtr
(WebCore::FilterEffectRendererCoreImage::imageForSourceGraphic):
    Use the source image to create a CIImage and return it. If the source image
    is backed by IOSurface, we create a CIImage directly from the IOSurface; otherwise
    we create a CIImage by getting the NativeImagePtr from source image.
(WebCore::FilterEffectRendererCoreImage::imageForFEColorMatrix):
    Based on the type of the matrix and input parameter,
    create different types of CIColorMatrix filers that are used to implement
    sepia, grayscale, hue-rotate, saturate
(WebCore::FilterEffectRendererCoreImage::output const):
    Added a logging line
(WebCore::FilterEffectRendererCoreImage::renderToImageBuffer):
    Minor refactors
(WebCore::FilterEffectRendererCoreImage::clearResult):
(WebCore::FilterEffectRendererCoreImage::FilterEffectRendererCoreImage):
* platform/graphics/filters/FEColorMatrix.h:
(isType): added SPECIALIZE_TYPE_TRAITS so that we can downcast to FEColorMatrix
* platform/graphics/filters/FilterEffect.cpp:
(WebCore::FilterEffect::requestedRegionOfInputImageData const): removed
    ASSERT(hasResult()); from the function, because CoreImage path wouldn't create
    image buffers inside FilterEffect, but CI path still needs to consult this function
    to obtain geometry info
* platform/graphics/filters/FilterEffect.h:
(WebCore::FilterEffect::filterEffectClassType const): Made the function a const
(WebCore::FilterEffect::normalizedFloats): Changed this function to public since it is
    consulted in FilterEffectCoreImageRenderer.
(WebCore::FilterEffect::filterEffectClassType): Deleted.
* platform/graphics/filters/SourceGraphic.h:
(isType): added SPECIALIZE_TYPE_TRAITS so that we can downcast to SourceGraphic
* rendering/CSSFilter.cpp:
(WebCore::CSSFilter::build):
    Checks whether m_filterRenderer is already created, no need to re-create if the pointer
    is non-null
(WebCore::CSSFilter::allocateBackingStoreIfNeeded):
    If we are using m_filterRenderer, which is currently using CoreImage and relies on
    IOSurfaces, we ask source image to use IOSurface for performance concerns.

LayoutTests:

* css3/filters/effect-grayscale-square-expected.html: Added.
* css3/filters/effect-grayscale-square.html: Added.
* css3/filters/effect-hue-rotate-square-expected.html: Added.
* css3/filters/effect-hue-rotate-square.html: Added.
* css3/filters/effect-saturate-square-expected.html: Added.
* css3/filters/effect-saturate-square.html: Added.
* css3/filters/effect-sepia-square-expected.html: Added.
* css3/filters/effect-sepia-square.html: Added.
* fast/filter-image/background-filter-image-expected.html:
    This is the expected result for the background image filter. The background image
    is not upsampled for retina screens due to a bug, so the test runner is comparing
    an unscaled image with an upscaled-then-downscaled image. CoreImage handles upscaling
    and downscaling differently, so the pixel diff will fail. Thus, turning off CI for this
    particular test case until the background image filter not upsampling bug is fixed.
* fast/filter-image/background-filter-image.html:
    ditto

Canonical link: https://commits.webkit.org/228944@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@266542 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
Frank Yang authored and webkit-commit-queue committed Sep 3, 2020
1 parent 8a49d54 commit bbe023e7a7d9d96baa4b41f8f1ef1b33c6ef3b85
@@ -1,3 +1,27 @@
2020-09-03 Frank Yang <guowei_yang@apple.com>

CoreImage Implementation of SourceGraphic and saturate(), hue-rotate(), grayscale() and sepia()
https://bugs.webkit.org/show_bug.cgi?id=213673

Reviewed by Darin Adler, Simon Fraser.

* css3/filters/effect-grayscale-square-expected.html: Added.
* css3/filters/effect-grayscale-square.html: Added.
* css3/filters/effect-hue-rotate-square-expected.html: Added.
* css3/filters/effect-hue-rotate-square.html: Added.
* css3/filters/effect-saturate-square-expected.html: Added.
* css3/filters/effect-saturate-square.html: Added.
* css3/filters/effect-sepia-square-expected.html: Added.
* css3/filters/effect-sepia-square.html: Added.
* fast/filter-image/background-filter-image-expected.html:
This is the expected result for the background image filter. The background image
is not upsampled for retina screens due to a bug, so the test runner is comparing
an unscaled image with an upscaled-then-downscaled image. CoreImage handles upscaling
and downscaling differently, so the pixel diff will fail. Thus, turning off CI for this
particular test case until the background image filter not upsampling bug is fixed.
* fast/filter-image/background-filter-image.html:
ditto

2020-09-03 Kimmo Kinnunen <kkinnunen@apple.com>

Mark uniform samplers in an array unused per element
@@ -0,0 +1,19 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: rgb(154, 27, 27);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: red;
filter: grayscale(50%);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -0,0 +1,19 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: rgb(0, 91, 0);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: red;
filter: hue-rotate(90deg);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -0,0 +1,19 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: rgb(154, 27, 27);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: red;
filter: saturate(50%);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -0,0 +1,19 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: rgb(24, 21, 144);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html>
<head>
<title>CSS Filter Test: Test for Saturate short hand</title>
<style type="text/css">
div {
width: 600px;
height: 600px;
background: blue;
filter: sepia(50%);
}

</style>
</head>
<body>
<p>You should see a green rectangle.</p>
<div></div>

</body>
</html>
@@ -2,6 +2,10 @@
<html>
<head>
<title>This tests that background images with filters scale properly.</title>
<script>
if (window.internals)
internals.settings.setCoreImageAcceleratedFilterRenderEnabled(false);
</script>
<style>
img {
width: 100px;
@@ -2,6 +2,10 @@
<html>
<head>
<title>This tests that background images with filters scale properly.</title>
<script>
if (window.internals)
internals.settings.setCoreImageAcceleratedFilterRenderEnabled(false);
</script>
<style>
div {
background-image: filter(url(../replaced/resources/compass.jpg), sepia(1));
@@ -1,3 +1,68 @@
2020-09-03 Frank Yang <guowei_yang@apple.com>

CoreImage Implementation of SourceGraphic and saturate(), hue-rotate(), grayscale() and sepia()
https://bugs.webkit.org/show_bug.cgi?id=213673

Reviewed by Darin Adler, Simon Fraser.

Tests: css3/filters/effect-grayscale-square.html
css3/filters/effect-hue-rotate-square.html
css3/filters/effect-saturate-square.html
css3/filters/effect-sepia-square.html

Added CoreImage implementation of SourceGraphic and partial of FEColorMatrix,
which is used to implement sepia(), grayscale() hue-rotate() and saturate().
Also added CIFilter caching, by storing CIFilters in a hash map, where the key
is the level of traversal (an integer).

* platform/graphics/coreimage/FilterEffectRendererCoreImage.h:
Imported FEColorMatrix.h and SourceGraphic.h; added function getCIContext()
that returns a static copy of CIContext.
* platform/graphics/coreimage/FilterEffectRendererCoreImage.mm:
(WebCore::FilterEffectRendererCoreImage::getCIContext): Creating a
CIContext is expensive. Instead of creating a CIContext on each frame
update, we just use a static singleton on every draw
(WebCore::FilterEffectRendererCoreImage::supportsCoreImageRendering):
the function now returns true if the filter chain contains SourceGraphic
and FEColorMatrix of TYPE_MATRIX, TYPE_SATURATE and TYPE_HUEROTATE
(WebCore::FilterEffectRendererCoreImage::connectCIFilters):
Made slight changes to wrap Obj-C pointers in RetainPtr
(WebCore::FilterEffectRendererCoreImage::imageForSourceGraphic):
Use the source image to create a CIImage and return it. If the source image
is backed by IOSurface, we create a CIImage directly from the IOSurface; otherwise
we create a CIImage by getting the NativeImagePtr from source image.
(WebCore::FilterEffectRendererCoreImage::imageForFEColorMatrix):
Based on the type of the matrix and input parameter,
create different types of CIColorMatrix filers that are used to implement
sepia, grayscale, hue-rotate, saturate
(WebCore::FilterEffectRendererCoreImage::output const):
Added a logging line
(WebCore::FilterEffectRendererCoreImage::renderToImageBuffer):
Minor refactors
(WebCore::FilterEffectRendererCoreImage::clearResult):
(WebCore::FilterEffectRendererCoreImage::FilterEffectRendererCoreImage):
* platform/graphics/filters/FEColorMatrix.h:
(isType): added SPECIALIZE_TYPE_TRAITS so that we can downcast to FEColorMatrix
* platform/graphics/filters/FilterEffect.cpp:
(WebCore::FilterEffect::requestedRegionOfInputImageData const): removed
ASSERT(hasResult()); from the function, because CoreImage path wouldn't create
image buffers inside FilterEffect, but CI path still needs to consult this function
to obtain geometry info
* platform/graphics/filters/FilterEffect.h:
(WebCore::FilterEffect::filterEffectClassType const): Made the function a const
(WebCore::FilterEffect::normalizedFloats): Changed this function to public since it is
consulted in FilterEffectCoreImageRenderer.
(WebCore::FilterEffect::filterEffectClassType): Deleted.
* platform/graphics/filters/SourceGraphic.h:
(isType): added SPECIALIZE_TYPE_TRAITS so that we can downcast to SourceGraphic
* rendering/CSSFilter.cpp:
(WebCore::CSSFilter::build):
Checks whether m_filterRenderer is already created, no need to re-create if the pointer
is non-null
(WebCore::CSSFilter::allocateBackingStoreIfNeeded):
If we are using m_filterRenderer, which is currently using CoreImage and relies on
IOSurfaces, we ask source image to use IOSurface for performance concerns.

2020-09-03 Chris Dumez <cdumez@apple.com>

BiquadFilterNode.getFrequencyResponse() should return NaN for out-of-bounds frequencies
@@ -27,7 +27,9 @@

#if USE(CORE_IMAGE)

#import "FEColorMatrix.h"
#import "FilterEffectRenderer.h"
#import "SourceGraphic.h"
#import <wtf/HashMap.h>
#import <wtf/Vector.h>

@@ -42,25 +44,25 @@ class FilterEffectRendererCoreImage : public FilterEffectRenderer {

public:
static std::unique_ptr<FilterEffectRendererCoreImage> tryCreate(FilterEffect&);

RetainPtr<CIContext> sharedCIContext();
void applyEffects(FilterEffect&) final;
bool hasResult() const final { return m_outputImage; }
ImageBuffer* output() const final;
FloatRect destRect(const FilterEffect&) const final;
void clearResult() final;

FilterEffectRendererCoreImage();

private:
CIImage* connectCIFilters(FilterEffect&);
RetainPtr<CIImage> connectCIFilters(FilterEffect&);
void renderToImageBuffer(FilterEffect&) final;
static bool supportsCoreImageRendering(FilterEffect&);
static bool canRenderUsingCIFilters(FilterEffect&);

RetainPtr<CIImage> imageForSourceGraphic(SourceGraphic&);
RetainPtr<CIImage> imageForFEColorMatrix(const FEColorMatrix&, const Vector<RetainPtr<CIImage>>&);

std::unique_ptr<ImageBuffer> m_outputImageBuffer;
HashMap<Ref<FilterEffect>, Vector<RetainPtr<CIFilter>>> m_ciFilterStorageMap;
RetainPtr<CIImage> m_outputImage;
RetainPtr<CIContext> m_context;
};

} // namespace WebCore

0 comments on commit bbe023e

Please sign in to comment.