Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a layer and filter interface in the 2D canvas #9537

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

graveljp
Copy link

@graveljp graveljp commented Jul 19, 2023

This adds new beginLayer and endLayer functions to open and close layers
in the canvas. While layers are active, draw calls operate on a separate
texture that gets composited to the parent output bitmap when the layer
is closed. An optional filter can be specified in beginLayer, allowing
effects to be applied to the layer's texture when it's composited its
parent.

Fixes #8476

(See WHATWG Working Mode: Changes for more details.)


/acknowledgements.html ( diff )
/canvas.html ( diff )
/index.html ( diff )
/infrastructure.html ( diff )

Copy link
Member

@Kaiido Kaiido left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Impressive work on this.
I still have some concerns about the layer rendering state being picked from the global settings instead of being defined along the filter as a parameter, and about the "when a canvas is presented" concept, but otherwise it all seems to make sense, I think.

Also, I've been a bit quick over the CanvasFilters part as I suppose it's mostly what had already been reviewed on #6763 by Aaron.

source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated
has one, is always just an alias to a <code>canvas</code> element's bitmap. When layers are
opened, implementations must behave as if draw calls operate on a separate <span>output
bitmap</span> that gets composited to the parent <span>output bitmap</span> when the layer is
closed. If the <code>canvas</code> element's bitmap needs to be presented while layers are opened,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"presented" is unfortunately a bit unclear as a concept, should it be clearly defined?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, this has actually caused me some confusion as I was trying to understand how the canvas gets consumed by the event loop. I added a "concept-canvas-presented" definition above to clarify this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great.
Hopefully if/when the "update the rendering" steps are rewritten to finally match the implementations we'll be able to add a line about that in there too?
My only remaining concern about this would be about other canvas consumers that don't go through the "check the usability of the image" algo. On the top of my head I can only think of the mediacapture-fromelement (canvas.captureStream()), where they never defined when the canvas frame should be moved to the stream.
I guess having this definition would help them to specify the apparent implementation which seems to be to wait for the next "update the rendering". (c.f. w3c/mediacapture-fromelement#94 and linked).

source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated

<div w-nodev>

<p>Before any operations could access the <code>canvas</code> element's bitmap pixels, the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that reading operations do throw if a layer is open, this leaves only the ill-defined "when the canvas is presented" case that could reach it, right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the intent. But I removed this section. The CanvasRenderingContext2D now has a top level output bitmap and a current output bitmap. The top level output bitmap can always be presented to the user even when layers are opened. When layers are opened, draw operations are performed in a separate bitmap via current output bitmap.

source Outdated
Comment on lines 64618 to 64630
was flushed by running the steps for <dfn>restoring the drawing state stack</dfn>:</p>

<ol>
<li><p><span data-x="list iterate">For each</span> <var>stateEntry</var> of
<var>stateBackup</var>:</p></li>

<ol>
<li><p><span data-x="stack push">Push</span> <var>stateEntry</var> onto the <span>drawing state
stack</span>.</p></li>
</ol>

<li><p>Restore the context's current <span>drawing state</span> by running the <code
data-x="dom-context-2d-restore">restore()</code> method steps.</p></li>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this doesn't restore any of the bitmaps, right? This means that if we have something like

ctx.globalAlpha = 0.2;
ctx.beginLayer();
ctx.fillRect(0, 0, 50, 50);

The rectangle will keep getting more opaque every time the canvas is "presented". Which is... every frame while it's visible on the page? Or every time the page has to be repainted? Or something else?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section has been removed.

The idea was that when presenting a frame, all the layers would have been closed, the draw calls would have been rasterized, and then, all layers would have been re-opened. This algorithm was describing the logic for re-opening the layers. Only the states were to be re-populated, not the draw calls. On later frame, only new draw calls would get drawn.

This was all replaced with an alternative strategy where unclosed layers are never presented. Instead, only the draw calls up to the first beginLayer() gets rasterized. The content of the layer is preserved for the next frame, where layers can be closed and rasterized as a whole.

source Outdated
Comment on lines 64319 to 64320
<li><p>Set all current <span data-x="drawing state">drawing states</span> to the values they have
in <var>parentDrawingStates</var>.</p></li>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably exclude the canvas layer state.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is tricky to phrase. Let me know what you think.

source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated
<span>CanvasFilterInput</span>? <dfn data-x="dom-context-2d-beginLayer-options-filter">filter</dfn> = null;
};

interface mixin <dfn interface>CanvasLayers</dfn> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will/should PaintRenderingContext2D also get this mixin?

Copy link
Author

@graveljp graveljp Aug 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should probably. Since it's spec'ed in a different place, I can file a ticket to track this separately. Would that make sense?

source Outdated Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Show resolved Hide resolved
source Outdated

<li><p>Set all current <span data-x="drawing state">drawing states</span> to the values they have
in <var>previousDrawingStates</var>.</p></li>
</ol>

<p>The <dfn method for="CanvasState"><code data-x="dom-context-2d-reset">reset()</code></dfn>
method steps are to <span>reset the rendering context to its default state</span>.</p>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resetting a context should also close/discard all currently opened layers right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. But I think that's already covered in step 3 and 4 below:

"To reset the rendering context to its default state:
....
3 - Clear the context's drawing state stack
4 - Set the context's layer-count to zero."

Isn't it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that should be done first. Here canvas's bitmap in step 1 may still be an alias to an open layer, and stating clearly what happens to pending layers may be a good thing too.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was interpreting "canvas's bitmap" to be the "canvas element's bitmap" (for element canvas), which is different than the "context output bitmap". For instance, in section 4.12.5.1.1, we have:
"The top level output bitmap of a rendering context, when it has one, is always just an alias to a canvas element's bitmap."

But you are right that it whould be made clear that the "context's current output bitmap", which changes when layers are opened, should be restored to point to the canvas element bitmap.

Copy link

@tuankiet65 tuankiet65 Aug 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I interpret step 3 and 4 as: literally clear the drawing state stack and not unwinding the stack (like say, if the drawing state stack is a stack variable, just call some clear() method to clear it), and set the layer-count variable to 0 without unwinding opened layers. I think for clarity purposes, it'd be helpful to explicitly specify it like "close all opened layers", or "while layer count is not zero, perform the steps in endLayer()" or even "discard all opened layers".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is, closing all layers is redundant if you are to clear the state stack and reset all states to their default value. In practice, implementations would not close all layers, compositing up all layer outputs and applying filters, just to discard everything in the end. The current spec already uses the step 3 (clear the context's drawing state stack). It's not saying "call restore() in a loop until the state stack is empty.

Maybe I can add a non-normative note saying that these steps have the effect of closing all layers?

Copy link
Member

@Kaiido Kaiido Aug 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There would be an observable difference though: flushing the layers (or calling endLayer) would update the origin-clean flag of the canvas bitmap, while if you don't something like the below snippet would not taint the canvas:

ctx.beginLayer()
ctx.drawImage(crossOriginImage, x, y)
ctx.reset()

I'm not quite sure what security risks this would involve (since the cross-origin resource would technically never have touched the bitmap), but that would be observable.

(Ps: Testing on the current Canary's implementation the above snippet does taint the canvas, so somehow at least part of the flushing might be done?)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a bullet point specifying that the current output bitmap has to be reset to point to the top level output bitmap.

@Kaiido, thanks for pointing out the origin-clean issue. You are correct that closing all layers would give a different result than just resetting the state stack.

The reset() spec should actually say that origin-clean must be set to true. The context does reset the origin-clean flag if the canvas is reset by a size change:

“The flag can be reset in certain situations; for example, when changing the value of the width or the height content attribute of the canvas element to which a CanvasRenderingContext2D is bound, the bitmap is cleared and its origin-clean flag is reset.”
https://html.spec.whatwg.org/multipage/canvas.html#security-with-canvas-elements

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I never noticed that sentence before, but I agree with your reading.
FWIW, this is not interoperable today, only Chrome does reset the flag when changing the canvas size. Can probably be left for another issue.

chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 13, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 14, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 16, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 18, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
aarongable pushed a commit to chromium/chromium that referenced this pull request Oct 20, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212692}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 20, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212692}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 20, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212692}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 20, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
aarongable pushed a commit to chromium/chromium that referenced this pull request Oct 20, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212741}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 20, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212741}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Oct 20, 2023
This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212741}
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this pull request Nov 1, 2023
…map if canvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in transferToImageBitmap if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212692}

--

wpt-commits: 3375400712353d2c9b011ed3dbb24c8d756b784f
wpt-pr: 42544
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this pull request Nov 1, 2023
…nvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in putImageData if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212741}

--

wpt-commits: 6e6f9fbb0746983001d7fe80ab717567c2f73bfc
wpt-pr: 42662
ErichDonGubler pushed a commit to ErichDonGubler/firefox that referenced this pull request Nov 2, 2023
…map if canvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in transferToImageBitmap if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212692}

--

wpt-commits: 3375400712353d2c9b011ed3dbb24c8d756b784f
wpt-pr: 42544
ErichDonGubler pushed a commit to ErichDonGubler/firefox that referenced this pull request Nov 2, 2023
…nvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in putImageData if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserb@chromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravel@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1212741}

--

wpt-commits: 6e6f9fbb0746983001d7fe80ab717567c2f73bfc
wpt-pr: 42662
Copy link
Author

@graveljp graveljp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I uploaded a commit addressing all review comments. I also made these important changes:

  • beginLayer now accepts CSS filter strings.

  • The CanvasRenderingContext2D now has a "top level output bitmap" and a "current output bitmap". The "top level output bitmap" can always be presented to the user, even when layers are open. The content of layers only gets rasterized to the "top level output bitmap" when all layers are closed. This is done by having layers replace and restore the "current output bitmap" to which context draws.

source Outdated Show resolved Hide resolved
source Outdated Show resolved Hide resolved
source Outdated
<span>CanvasFilterInput</span>? <dfn data-x="dom-context-2d-beginLayer-options-filter">filter</dfn> = null;
};

interface mixin <dfn interface>CanvasLayers</dfn> {
Copy link
Author

@graveljp graveljp Aug 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should probably. Since it's spec'ed in a different place, I can file a ticket to track this separately. Would that make sense?

source Outdated Show resolved Hide resolved
source Outdated
keeping track of the number of opened nested layers. To access the content of the context's
<span>output bitmap</span>, the <span>steps for reading the context output bitmap</span> must be
used.

<p>The <span>output bitmap</span> has an <span
data-x="concept-canvas-origin-clean">origin-clean</span> flag, which can be set to true or false.
Initially, when one of these bitmaps is created, its <span
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Layers behave the same as other draw calls for {alpha: false} contexts. The layer content and the layer itself is alpha-blended up to the point where it gets written to the top level output bitmap. I updated the paragraph below to clarify this, and added WPT tests validating this (https://crrev.com/c/5006000).

source Show resolved Hide resolved
source Show resolved Hide resolved
source Outdated
Comment on lines 64618 to 64630
was flushed by running the steps for <dfn>restoring the drawing state stack</dfn>:</p>

<ol>
<li><p><span data-x="list iterate">For each</span> <var>stateEntry</var> of
<var>stateBackup</var>:</p></li>

<ol>
<li><p><span data-x="stack push">Push</span> <var>stateEntry</var> onto the <span>drawing state
stack</span>.</p></li>
</ol>

<li><p>Restore the context's current <span>drawing state</span> by running the <code
data-x="dom-context-2d-restore">restore()</code> method steps.</p></li>
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section has been removed.

The idea was that when presenting a frame, all the layers would have been closed, the draw calls would have been rasterized, and then, all layers would have been re-opened. This algorithm was describing the logic for re-opening the layers. Only the states were to be re-populated, not the draw calls. On later frame, only new draw calls would get drawn.

This was all replaced with an alternative strategy where unclosed layers are never presented. Instead, only the draw calls up to the first beginLayer() gets rasterized. The content of the layer is preserved for the next frame, where layers can be closed and rasterized as a whole.

source Show resolved Hide resolved
source Outdated Show resolved Hide resolved
This adds new beginLayer and endLayer functions to open and close layers
in the canvas. While layers are active, draw calls operate on a separate
texture that gets composited to the parent output bitmap when the layer
is closed. An optional filter can be specified in beginLayer, allowing
effects to be applied to the layer's texture when it's composited its
parent.

Tests:
 https://github.com/web-platform-tests/wpt/tree/master/html/canvas/element/layers
 https://github.com/web-platform-tests/wpt/tree/master/html/canvas/offscreen/layers

Fixes whatwg#8476
This update addresses 3 main things:
- Support for CSS filter strings was added to the beginLayer API
- Unclosed layers are now never rasterized when the canvas is presented
  to the user. Instead, the content of the layer is preserved and will
  be rasterized in a later frame, if/when the layer is closed.
- Replied to the first round of review comments.
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified-and-comments-removed that referenced this pull request Nov 8, 2023
…map if canvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in transferToImageBitmap if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravelchromium.org>
Reviewed-by: Fernando Serboncini <fserbchromium.org>
Cr-Commit-Position: refs/heads/main{#1212692}

--

wpt-commits: 3375400712353d2c9b011ed3dbb24c8d756b784f
wpt-pr: 42544

UltraBlame original commit: 5dcacd6ceadca3d78dab262fdd726b9646edbe7e
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified-and-comments-removed that referenced this pull request Nov 8, 2023
…nvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in putImageData if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserbchromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravelchromium.org>
Cr-Commit-Position: refs/heads/main{#1212741}

--

wpt-commits: 6e6f9fbb0746983001d7fe80ab717567c2f73bfc
wpt-pr: 42662

UltraBlame original commit: 685e2ff9c9f305fea2dfd442a95067594d05a6d1
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified that referenced this pull request Nov 8, 2023
…map if canvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in transferToImageBitmap if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravelchromium.org>
Reviewed-by: Fernando Serboncini <fserbchromium.org>
Cr-Commit-Position: refs/heads/main{#1212692}

--

wpt-commits: 3375400712353d2c9b011ed3dbb24c8d756b784f
wpt-pr: 42544

UltraBlame original commit: 5dcacd6ceadca3d78dab262fdd726b9646edbe7e
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified that referenced this pull request Nov 8, 2023
…nvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in putImageData if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserbchromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravelchromium.org>
Cr-Commit-Position: refs/heads/main{#1212741}

--

wpt-commits: 6e6f9fbb0746983001d7fe80ab717567c2f73bfc
wpt-pr: 42662

UltraBlame original commit: 685e2ff9c9f305fea2dfd442a95067594d05a6d1
gecko-dev-updater pushed a commit to marco-c/gecko-dev-comments-removed that referenced this pull request Nov 8, 2023
…map if canvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in transferToImageBitmap if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

OffscreenCanvas.transferToImageBitmap however is supposed to release
the canvas content, leaving the offscreen canvas empty. We cannot
release the recording if layers are incomplete, and if we kept the
layer content alive for later, we would not be leaving the canvas
empty as the spec requires.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Bug: 1484741
Change-Id: Ic770b51a0343faf0b2c7477624d69f59187ce97f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4939633
Commit-Queue: Jean-Philippe Gravel <jpgravelchromium.org>
Reviewed-by: Fernando Serboncini <fserbchromium.org>
Cr-Commit-Position: refs/heads/main{#1212692}

--

wpt-commits: 3375400712353d2c9b011ed3dbb24c8d756b784f
wpt-pr: 42544

UltraBlame original commit: 5dcacd6ceadca3d78dab262fdd726b9646edbe7e
gecko-dev-updater pushed a commit to marco-c/gecko-dev-comments-removed that referenced this pull request Nov 8, 2023
…nvas layers are opened, a=testonly

Automatic update from web-platform-tests
Throw an exception in putImageData if canvas layers are opened

This API is incompatible with how the 2D canvas is rasterized when
it contains unclosed layers. Because layers can have filters that get
applied on their final content, they can't be presented until they are
closed. Instead, we normally keep the layer content alive after a
flush, so that it can be presented in a later frame when the layer is
finally closed.

putImageData however is supposed to write to the canvas bitmap
wholesale, bypassing all render states. This means that we can't write
to the layer's content because the written pixels would then get
filtered when the layer is closed. We can't write to the main canvas
either because these pixels would later be overwritten by the layer's
result with draw calls that potentially happened before (e.g. in the
sequence `beginLayer(); fillRect(); putImageData(); endLayer();`,
`fillRect()` would write over `putImageData()`.

This behavior is part of the current 2D Canvas Layer spec draft:
Explainer: https://github.com/fserb/canvas2D/blob/master/spec/layers.md
Spec draft: whatwg/html#9537

Change-Id: I266a3155c32919a68dbbb093e4aff9b1dd13a3b5
Bug: 1484741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4943172
Reviewed-by: Fernando Serboncini <fserbchromium.org>
Commit-Queue: Jean-Philippe Gravel <jpgravelchromium.org>
Cr-Commit-Position: refs/heads/main{#1212741}

--

wpt-commits: 6e6f9fbb0746983001d7fe80ab717567c2f73bfc
wpt-pr: 42662

UltraBlame original commit: 685e2ff9c9f305fea2dfd442a95067594d05a6d1
Copy link
Member

@domenic domenic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some editorial feedback, but it all looks reasonable otherwise.

Can you add citations for the other two implementers support? In particular, have they reviewed this specific API proposal and API design choices, e.g. the ones @Kaiido is questioning?

source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
source Show resolved Hide resolved
Copy link
Member

@Kaiido Kaiido left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new "top level output bitmap" structure is a lot cleaner IMO and seems to work very well.
Thanks for including the CSS <filter-value-list>. I'd definitely have loved to be able to set it directly as part of the sequence<CanvasFilterPrimitive> but this can certainly wait for a future iteration. I just have a few nits about the new CSS filters.

Then, you know my concerns about the whole "global states" + "reset states" things, but it's best to keep that discussion in the issue.

Ps: Thanks again for your great work on this.

<var>image</var>.</p></li>

<li><p>Let <var>filteredImage</var> be the result of applying the <var>resolvedFilter</var>
<span>&lt;filter-value-list&gt;</span> to <var>image</var> in the same manner as SVG
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds a bit vague. Wouldn't it be better to link to how filter handles it currently?

<li><p>If any property-independent style sheet syntax like 'inherit' or 'initial' is present
in <var>parsedValue</var>, then return "<code data-x="">none</code>".</p></li>

<li><p>If <var>parsedValue</var> references filters externally-defined in documents that
Copy link
Member

@Kaiido Kaiido Nov 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that's interesting. So if we have something like

ctx.beginLayer({ filter: "url(./image-that-takes-1s-to-load.svg)#filter" });
// ...
await wait(3000); // make sure the filter is loaded by now
ctx.endLayer();

the layer would not be filtered.
To be honest, I'm a bit torn by that. On the one hand I like very much that it's clear that if it's not loaded, it's not rendered. On the other hand, the loading of that filter is done through this exact param. So this means we always have at least one beginLayer() call where the filter wouldn't be rendered before being able to actually use any external SVG filter.
External SVG filters and their asynchronicity are already buggy in the current .filter anyway, and I don't think we have to block on that, just wanted to note it here. Also, this probably will require tests.

<li><p>If <var>filter</var> is "<code data-x="">none</code>", then return "<code
data-x="">none</code>".</p></li>

<li><p>Let <var>parsedValue</var> be the result of <span data-x="parse something according
Copy link
Member

@Kaiido Kaiido Nov 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we need to pass the linked canvas element here, so that relative units and stuff like currentColor can be resolved.
[edit]: Actually looking at "parse something according to a grammar" there is no context element argument there. So we may still need to handle all the special cases manually. I guess a hook to what is done in https://html.spec.whatwg.org/multipage/canvas.html#filters would be the easiest, but I'm not sure.

@domenic domenic added the agenda+ To be discussed at a triage meeting label Jan 17, 2024
@past past removed the agenda+ To be discussed at a triage meeting label Jan 26, 2024
shallawa added a commit to shallawa/WebKit that referenced this pull request May 16, 2024
https://bugs.webkit.org/show_bug.cgi?id=273923
rdar://127789082

Reviewed by NOBODY (OOPS!).

This is a draft patch for the canvas layers API. The discussion of this API is
in whatwg/html#9537.

Unlike what is stated in the discussion above, this PR adds the filter to the
layer rendering states. This means like other members in layer rendering state,
the filter will be cleared out from the context state once beginLayer() is called.
It will restored to the context state once endLayer() is called. Then the restored
filter will be applied to the layer before compositing it to the canvas context.

* LayoutTests/fast/canvas/canvas-layer-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-filter-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-layer-filter-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-nested-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-nested-layer-filter-drawing.html: Added.
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/CMakeLists.txt:
* Source/WebCore/DerivedSources-input.xcfilelist:
* Source/WebCore/DerivedSources-output.xcfilelist:
* Source/WebCore/DerivedSources.make:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/html/canvas/CanvasFilterContextSwitcher.cpp: Added.
(WebCore::CanvasFilterContextSwitcher::create):
(WebCore::CanvasFilterContextSwitcher::CanvasFilterContextSwitcher):
(WebCore::CanvasFilterContextSwitcher::~CanvasFilterContextSwitcher):
(WebCore::CanvasFilterContextSwitcher::expandedBounds const):
* Source/WebCore/html/canvas/CanvasFilterContextSwitcher.h: Copied from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.h.
* Source/WebCore/html/canvas/CanvasLayerContextSwitcher.cpp: Renamed from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.cpp.
(WebCore::CanvasLayerContextSwitcher::create):
(WebCore::CanvasLayerContextSwitcher::CanvasLayerContextSwitcher):
(WebCore::CanvasLayerContextSwitcher::~CanvasLayerContextSwitcher):
(WebCore::CanvasLayerContextSwitcher::drawingContext const):
(WebCore::CanvasLayerContextSwitcher::outsets const):
* Source/WebCore/html/canvas/CanvasLayerContextSwitcher.h: Renamed from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.h.
* Source/WebCore/html/canvas/CanvasLayers.idl: Added.
* Source/WebCore/html/canvas/CanvasRenderingContext2D.cpp:
(WebCore::CanvasRenderingContext2D::createFilter const):
* Source/WebCore/html/canvas/CanvasRenderingContext2D.idl:
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp:
(WebCore::CanvasRenderingContext2DBase::beginLayer):
(WebCore::CanvasRenderingContext2DBase::endLayer):
(WebCore::CanvasRenderingContext2DBase::fillInternal):
(WebCore::CanvasRenderingContext2DBase::strokeInternal):
(WebCore::CanvasRenderingContext2DBase::fillRect):
(WebCore::CanvasRenderingContext2DBase::strokeRect):
(WebCore::CanvasRenderingContext2DBase::drawImage):
(WebCore::CanvasRenderingContext2DBase::drawingContext const):
(WebCore::CanvasRenderingContext2DBase::drawTextUnchecked):
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.h:
(WebCore::CanvasRenderingContext2DBase::setFilterTargetSwitcher): Deleted.
* Source/WebCore/html/canvas/OffscreenCanvasRenderingContext2D.idl:
* Source/WebCore/platform/graphics/GraphicsContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterTargetSwitcher.cpp.
(WebCore::GraphicsContextSwitcher::create):
(WebCore::GraphicsContextSwitcher::GraphicsContextSwitcher):
* Source/WebCore/platform/graphics/GraphicsContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterTargetSwitcher.h.
* Source/WebCore/platform/graphics/ImageBuffer.cpp:
(WebCore::ImageBuffer::filteredNativeImage):
* Source/WebCore/platform/graphics/ImageBufferContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterImageTargetSwitcher.cpp.
(WebCore::ImageBufferContextSwitcher::ImageBufferContextSwitcher):
(WebCore::ImageBufferContextSwitcher::drawingContext const):
(WebCore::ImageBufferContextSwitcher::beginClipAndDrawSourceImage):
(WebCore::ImageBufferContextSwitcher::endClipAndDrawSourceImage):
(WebCore::ImageBufferContextSwitcher::endDrawSourceImage):
* Source/WebCore/platform/graphics/ImageBufferContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterImageTargetSwitcher.h.
* Source/WebCore/platform/graphics/TransparencyLayerContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterStyleTargetSwitcher.cpp.
(WebCore::TransparencyLayerContextSwitcher::TransparencyLayerContextSwitcher):
(WebCore::TransparencyLayerContextSwitcher::beginClipAndDrawSourceImage):
(WebCore::TransparencyLayerContextSwitcher::beginDrawSourceImage):
(WebCore::TransparencyLayerContextSwitcher::endDrawSourceImage):
* Source/WebCore/platform/graphics/TransparencyLayerContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterStyleTargetSwitcher.h.
* Source/WebCore/rendering/RenderLayerFilters.cpp:
(WebCore::RenderLayerFilters::beginFilterEffect):
* Source/WebCore/rendering/RenderLayerFilters.h:
* Source/WebCore/rendering/svg/legacy/LegacyRenderSVGResourceFilter.cpp:
(WebCore::LegacyRenderSVGResourceFilter::applyResource):
* Source/WebCore/rendering/svg/legacy/LegacyRenderSVGResourceFilter.h:
shallawa added a commit to shallawa/WebKit that referenced this pull request May 20, 2024
https://bugs.webkit.org/show_bug.cgi?id=273923
rdar://127789082

Reviewed by NOBODY (OOPS!).

This is a draft patch for the canvas layers API. The discussion of this API is
in whatwg/html#9537.

Unlike what is stated in the discussion above, this PR adds the filter to the
layer rendering states. This means like other members in layer rendering state,
the filter will be cleared out from the context state once beginLayer() is called.
It will restored to the context state once endLayer() is called. Then the restored
filter will be applied to the layer before compositing it to the canvas context.

* LayoutTests/fast/canvas/canvas-layer-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-filter-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-layer-filter-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-nested-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-nested-layer-filter-drawing.html: Added.
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/CMakeLists.txt:
* Source/WebCore/DerivedSources-input.xcfilelist:
* Source/WebCore/DerivedSources-output.xcfilelist:
* Source/WebCore/DerivedSources.make:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/html/canvas/CanvasFilterContextSwitcher.cpp: Added.
(WebCore::CanvasFilterContextSwitcher::create):
(WebCore::CanvasFilterContextSwitcher::CanvasFilterContextSwitcher):
(WebCore::CanvasFilterContextSwitcher::~CanvasFilterContextSwitcher):
(WebCore::CanvasFilterContextSwitcher::expandedBounds const):
* Source/WebCore/html/canvas/CanvasFilterContextSwitcher.h: Copied from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.h.
* Source/WebCore/html/canvas/CanvasLayerContextSwitcher.cpp: Renamed from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.cpp.
(WebCore::CanvasLayerContextSwitcher::create):
(WebCore::CanvasLayerContextSwitcher::CanvasLayerContextSwitcher):
(WebCore::CanvasLayerContextSwitcher::~CanvasLayerContextSwitcher):
(WebCore::CanvasLayerContextSwitcher::drawingContext const):
(WebCore::CanvasLayerContextSwitcher::outsets const):
* Source/WebCore/html/canvas/CanvasLayerContextSwitcher.h: Renamed from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.h.
* Source/WebCore/html/canvas/CanvasLayers.idl: Added.
* Source/WebCore/html/canvas/CanvasRenderingContext2D.cpp:
(WebCore::CanvasRenderingContext2D::createFilter const):
* Source/WebCore/html/canvas/CanvasRenderingContext2D.idl:
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp:
(WebCore::CanvasRenderingContext2DBase::beginLayer):
(WebCore::CanvasRenderingContext2DBase::endLayer):
(WebCore::CanvasRenderingContext2DBase::fillInternal):
(WebCore::CanvasRenderingContext2DBase::strokeInternal):
(WebCore::CanvasRenderingContext2DBase::fillRect):
(WebCore::CanvasRenderingContext2DBase::strokeRect):
(WebCore::CanvasRenderingContext2DBase::drawImage):
(WebCore::CanvasRenderingContext2DBase::drawingContext const):
(WebCore::CanvasRenderingContext2DBase::drawTextUnchecked):
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.h:
(WebCore::CanvasRenderingContext2DBase::setFilterTargetSwitcher): Deleted.
* Source/WebCore/html/canvas/OffscreenCanvasRenderingContext2D.idl:
* Source/WebCore/platform/graphics/GraphicsContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterTargetSwitcher.cpp.
(WebCore::GraphicsContextSwitcher::create):
(WebCore::GraphicsContextSwitcher::GraphicsContextSwitcher):
* Source/WebCore/platform/graphics/GraphicsContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterTargetSwitcher.h.
* Source/WebCore/platform/graphics/ImageBuffer.cpp:
(WebCore::ImageBuffer::filteredNativeImage):
* Source/WebCore/platform/graphics/ImageBufferContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterImageTargetSwitcher.cpp.
(WebCore::ImageBufferContextSwitcher::ImageBufferContextSwitcher):
(WebCore::ImageBufferContextSwitcher::drawingContext const):
(WebCore::ImageBufferContextSwitcher::beginClipAndDrawSourceImage):
(WebCore::ImageBufferContextSwitcher::endClipAndDrawSourceImage):
(WebCore::ImageBufferContextSwitcher::endDrawSourceImage):
* Source/WebCore/platform/graphics/ImageBufferContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterImageTargetSwitcher.h.
* Source/WebCore/platform/graphics/TransparencyLayerContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterStyleTargetSwitcher.cpp.
(WebCore::TransparencyLayerContextSwitcher::TransparencyLayerContextSwitcher):
(WebCore::TransparencyLayerContextSwitcher::beginClipAndDrawSourceImage):
(WebCore::TransparencyLayerContextSwitcher::beginDrawSourceImage):
(WebCore::TransparencyLayerContextSwitcher::endDrawSourceImage):
* Source/WebCore/platform/graphics/TransparencyLayerContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterStyleTargetSwitcher.h.
* Source/WebCore/rendering/RenderLayerFilters.cpp:
(WebCore::RenderLayerFilters::beginFilterEffect):
* Source/WebCore/rendering/RenderLayerFilters.h:
* Source/WebCore/rendering/svg/legacy/LegacyRenderSVGResourceFilter.cpp:
(WebCore::LegacyRenderSVGResourceFilter::applyResource):
* Source/WebCore/rendering/svg/legacy/LegacyRenderSVGResourceFilter.h:
shallawa added a commit to shallawa/WebKit that referenced this pull request May 23, 2024
https://bugs.webkit.org/show_bug.cgi?id=273923
rdar://127789082

Reviewed by NOBODY (OOPS!).

This is a draft patch for the canvas layers API. The discussion of this API is
in whatwg/html#9537.

Unlike what is stated in the discussion above, this PR adds the filter to the
layer rendering states. This means like other members in layer rendering state,
the filter will be cleared out from the context state once beginLayer() is called.
It will restored to the context state once endLayer() is called. Then the restored
filter will be applied to the layer before compositing it to the canvas context.

* LayoutTests/fast/canvas/canvas-layer-alpha-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-alpha-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-layer-composite-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-composite-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-layer-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-layer-filter-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-layer-filter-drawing.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-nested-layer-filter-drawing-expected.html: Added.
* LayoutTests/fast/canvas/canvas-multiple-nested-layer-filter-drawing.html: Added.
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/CMakeLists.txt:
* Source/WebCore/DerivedSources-input.xcfilelist:
* Source/WebCore/DerivedSources-output.xcfilelist:
* Source/WebCore/DerivedSources.make:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/html/canvas/CanvasFilterContextSwitcher.cpp: Copied from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.h.
(WebCore::CanvasFilterContextSwitcher::create):
(WebCore::CanvasFilterContextSwitcher::CanvasFilterContextSwitcher):
(WebCore::CanvasFilterContextSwitcher::~CanvasFilterContextSwitcher):
(WebCore::CanvasFilterContextSwitcher::expandedBounds const):
* Source/WebCore/html/canvas/CanvasFilterContextSwitcher.h: Copied from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.h.
* Source/WebCore/html/canvas/CanvasLayerContextSwitcher.cpp: Renamed from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.cpp.
(WebCore::CanvasLayerContextSwitcher::create):
(WebCore::CanvasLayerContextSwitcher::CanvasLayerContextSwitcher):
(WebCore::CanvasLayerContextSwitcher::~CanvasLayerContextSwitcher):
(WebCore::CanvasLayerContextSwitcher::drawingContext const):
(WebCore::CanvasLayerContextSwitcher::outsets const):
* Source/WebCore/html/canvas/CanvasLayerContextSwitcher.h: Renamed from Source/WebCore/html/canvas/CanvasFilterTargetSwitcher.h.
* Source/WebCore/html/canvas/CanvasLayers.idl: Added.
* Source/WebCore/html/canvas/CanvasRenderingContext2D.cpp:
(WebCore::CanvasRenderingContext2D::createFilter const):
* Source/WebCore/html/canvas/CanvasRenderingContext2D.h:
* Source/WebCore/html/canvas/CanvasRenderingContext2D.idl:
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp:
(WebCore::CanvasRenderingContext2DBase::beginLayer):
(WebCore::CanvasRenderingContext2DBase::endLayer):
(WebCore::CanvasRenderingContext2DBase::fillInternal):
(WebCore::CanvasRenderingContext2DBase::strokeInternal):
(WebCore::CanvasRenderingContext2DBase::fillRect):
(WebCore::CanvasRenderingContext2DBase::strokeRect):
(WebCore::CanvasRenderingContext2DBase::drawImage):
(WebCore::CanvasRenderingContext2DBase::drawingContext const):
(WebCore::CanvasRenderingContext2DBase::drawTextUnchecked):
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.h:
(WebCore::CanvasRenderingContext2DBase::createFilter const):
(WebCore::CanvasRenderingContext2DBase::setFilterTargetSwitcher): Deleted.
* Source/WebCore/html/canvas/OffscreenCanvasRenderingContext2D.idl:
* Source/WebCore/platform/graphics/GraphicsContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterTargetSwitcher.cpp.
(WebCore::GraphicsContextSwitcher::create):
(WebCore::GraphicsContextSwitcher::GraphicsContextSwitcher):
* Source/WebCore/platform/graphics/GraphicsContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterTargetSwitcher.h.
* Source/WebCore/platform/graphics/ImageBuffer.cpp:
(WebCore::ImageBuffer::filteredNativeImage):
* Source/WebCore/platform/graphics/ImageBufferContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterImageTargetSwitcher.cpp.
(WebCore::ImageBufferContextSwitcher::ImageBufferContextSwitcher):
(WebCore::ImageBufferContextSwitcher::drawingContext const):
(WebCore::ImageBufferContextSwitcher::beginClipAndDrawSourceImage):
(WebCore::ImageBufferContextSwitcher::endClipAndDrawSourceImage):
(WebCore::ImageBufferContextSwitcher::endDrawSourceImage):
* Source/WebCore/platform/graphics/ImageBufferContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterImageTargetSwitcher.h.
* Source/WebCore/platform/graphics/TransparencyLayerContextSwitcher.cpp: Renamed from Source/WebCore/platform/graphics/filters/FilterStyleTargetSwitcher.cpp.
(WebCore::TransparencyLayerContextSwitcher::TransparencyLayerContextSwitcher):
(WebCore::TransparencyLayerContextSwitcher::beginClipAndDrawSourceImage):
(WebCore::TransparencyLayerContextSwitcher::beginDrawSourceImage):
(WebCore::TransparencyLayerContextSwitcher::endDrawSourceImage):
* Source/WebCore/platform/graphics/TransparencyLayerContextSwitcher.h: Renamed from Source/WebCore/platform/graphics/filters/FilterStyleTargetSwitcher.h.
* Source/WebCore/rendering/RenderLayerFilters.cpp:
(WebCore::RenderLayerFilters::beginFilterEffect):
* Source/WebCore/rendering/RenderLayerFilters.h:
* Source/WebCore/rendering/svg/legacy/LegacyRenderSVGResourceFilter.cpp:
(WebCore::LegacyRenderSVGResourceFilter::applyResource):
* Source/WebCore/rendering/svg/legacy/LegacyRenderSVGResourceFilter.h:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Canvas2D layers with filter support
6 participants