Avoid going back to the GPU process for canvas ImageData when possible #16028

heycam · 2023-07-24T01:18:58Z

`5466cd2`

Avoid going back to the GPU process for canvas ImageData when possible
https://bugs.webkit.org/show_bug.cgi?id=259436
rdar://problem/112746729

Reviewed by Cameron McCormack and Myles C. Maxfield.

On a 2D canvas, if a recent putImageData() was made with no other
intervening drawing commands, we can use its data to fulfill a
subsequent getImageData(). Start with some simple conditions for
ImageData size and drawing dimensions, which can be extended later if it
proves useful.

* LayoutTests/fast/canvas/canvas-ImageData-cache-expected.txt: Added.
* LayoutTests/fast/canvas/canvas-ImageData-cache.html: Added.
* Source/WebCore/html/ImageData.cpp:
(WebCore::ImageData::clone const):
* Source/WebCore/html/ImageData.h:
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp:
(WebCore::CanvasRenderingContext2DBase::reset):
(WebCore::CanvasRenderingContext2DBase::didDraw):
(WebCore::CanvasRenderingContext2DBase::cacheImageDataIfPossible):
(WebCore::CanvasRenderingContext2DBase::takeCachedImageDataIfPossible const):
(WebCore::CanvasRenderingContext2DBase::getImageData const):
(WebCore::CanvasRenderingContext2DBase::putImageData):
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.h:

Canonical link: https://commits.webkit.org/266632@main

be9ac84

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🛠 gtk
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	~~🧪 gtk-wk2~~
	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-wk2-stress
✅ 🛠 🧪 merge	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2023-07-24T01:19:10Z

EWS run on previous version of this PR (hash 9a09fc1)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🛠 gtk
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	✅ 🧪 gtk-wk2
	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🧪 api-gtk
	✅ 🛠 tv	❌ 🧪 mac-AS-debug-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-wk2-stress
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2023-08-04T18:04:32Z

EWS run on previous version of this PR (hash d710d27)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	⏳ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🛠 gtk
	✅ 🧪 ios-wk2-wpt	~~🧪 mac-wk1~~	~~🧪 gtk-wk2~~
⏳ 🛠 🧪 jsc	✅ 🧪 api-ios	~~🧪 mac-wk2~~	~~🧪 api-gtk~~
	⏳ 🛠 tv	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 tv-sim	~~🧪 mac-wk2-stress~~
	✅ 🛠 watch
	⏳ 🛠 watch-sim

litherum · 2023-08-04T18:12:47Z

I think there's still a bit more to do here, even if EWS is green:

Only take this codepath for small canvases
Make an eviction policy, so we don't keep the data cached forever

webkit-early-warning-system · 2023-08-04T18:35:48Z

EWS run on previous version of this PR (hash 918c1f5)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	⏳ 🧪 wpe-wk2
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🛠 gtk
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	~~🧪 gtk-wk2~~
	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-wk2-stress
	✅ 🛠 watch
	✅ 🛠 watch-sim

heycam · 2023-08-04T20:39:28Z

And the conversion to premultiplied alpha and back.

webkit-early-warning-system · 2023-08-04T22:19:16Z

EWS run on previous version of this PR (hash 4bfd474)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	~~🛠 ios~~	~~🛠 mac~~	~~🛠 wpe~~	~~🛠 wincairo~~
~~🧪 bindings~~	~~🛠 ios-sim~~	~~🛠 mac-AS-debug~~	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🛠 gtk
	~~🧪 ios-wk2-wpt~~	~~🧪 mac-wk1~~	⏳ 🧪 gtk-wk2
	~~🧪 api-ios~~	~~🧪 mac-wk2~~	~~🧪 api-gtk~~
	~~🛠 tv~~	~~🧪 mac-AS-debug-wk2~~
	✅ 🛠 tv-sim	~~🧪 mac-wk2-stress~~
	~~🛠 watch~~
	~~🛠 watch-sim~~

litherum · 2023-08-04T22:28:56Z

This is not ready yet. When you get/put image data, the data is presented as unpremultiplied, but needs to round-trip through premultiplied, back to unpremultiplied. This isn't actually the identity function, because it's slightly lossy (consider round-tripping the color [0, 131, 0, 32] -> it ends up becoming [0, 128, 0, 32])

webkit-early-warning-system · 2023-08-04T23:08:41Z

EWS run on previous version of this PR (hash 59629b5)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	❌ 🧪 wpe-wk2
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🛠 gtk
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	❌ 🧪 gtk-wk2
	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-wk2-stress
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2023-08-05T07:24:56Z

EWS run on previous version of this PR (hash 3b419c8)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🛠 gtk
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	✅ 🧪 gtk-wk2
✅ 🛠 🧪 jsc	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🧪 api-gtk
✅ 🛠 🧪 jsc-arm64	✅ 🛠 tv	✅ 🧪 mac-AS-debug-wk2	✅ 🛠 jsc-armv7
	✅ 🛠 tv-sim	✅ 🧪 mac-wk2-stress	✅ 🧪 jsc-armv7-tests
	✅ 🛠 watch		✅ 🛠 jsc-mips
	✅ 🛠 watch-sim		✅ 🧪 jsc-mips-tests

litherum · 2023-08-05T18:45:43Z

I guess I can review (and approve) @heycam's part of this. Someone else (maybe @heycam?) will have to review my part of this.

litherum · 2023-08-05T18:47:22Z

This leads to a 4.53% performance improvement on MotionMark.

Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp

heycam · 2023-08-05T23:06:59Z

(Since GitHub thinks I am still the PR owner I'm unable to r+ or r- the PR.)

webkit-early-warning-system · 2023-08-07T08:24:35Z

EWS run on previous version of this PR (hash 478d2f2)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🛠 gtk
	~~🧪 ios-wk2-wpt~~	~~🧪 mac-wk1~~	~~🧪 gtk-wk2~~
✅ 🛠 🧪 jsc	~~🧪 api-ios~~	~~🧪 mac-wk2~~	~~🧪 api-gtk~~
✅ 🛠 🧪 jsc-arm64	✅ 🛠 tv	~~🧪 mac-AS-debug-wk2~~	✅ 🛠 jsc-armv7
	✅ 🛠 tv-sim	~~🧪 mac-wk2-stress~~	~~🧪 jsc-armv7-tests~~
	✅ 🛠 watch		✅ 🛠 jsc-mips
	✅ 🛠 watch-sim		~~🧪 jsc-mips-tests~~

webkit-early-warning-system · 2023-08-07T08:28:33Z

EWS run on previous version of this PR (hash 2a4beec)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🛠 gtk
	~~🧪 ios-wk2-wpt~~	~~🧪 mac-wk1~~	~~🧪 gtk-wk2~~
	~~🧪 api-ios~~	~~🧪 mac-wk2~~	~~🧪 api-gtk~~
	✅ 🛠 tv	~~🧪 mac-AS-debug-wk2~~
	⏳ 🛠 tv-sim	~~🧪 mac-wk2-stress~~
~~🛠 🧪 merge~~	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2023-08-07T08:35:40Z

EWS run on current version of this PR (hash be9ac84)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🛠 gtk
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	~~🧪 gtk-wk2~~
	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-wk2-stress
✅ 🛠 🧪 merge	✅ 🛠 watch
	✅ 🛠 watch-sim

https://bugs.webkit.org/show_bug.cgi?id=259436 rdar://problem/112746729 Reviewed by Cameron McCormack and Myles C. Maxfield. On a 2D canvas, if a recent putImageData() was made with no other intervening drawing commands, we can use its data to fulfill a subsequent getImageData(). Start with some simple conditions for ImageData size and drawing dimensions, which can be extended later if it proves useful. * LayoutTests/fast/canvas/canvas-ImageData-cache-expected.txt: Added. * LayoutTests/fast/canvas/canvas-ImageData-cache.html: Added. * Source/WebCore/html/ImageData.cpp: (WebCore::ImageData::clone const): * Source/WebCore/html/ImageData.h: * Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp: (WebCore::CanvasRenderingContext2DBase::reset): (WebCore::CanvasRenderingContext2DBase::didDraw): (WebCore::CanvasRenderingContext2DBase::cacheImageDataIfPossible): (WebCore::CanvasRenderingContext2DBase::takeCachedImageDataIfPossible const): (WebCore::CanvasRenderingContext2DBase::getImageData const): (WebCore::CanvasRenderingContext2DBase::putImageData): * Source/WebCore/html/canvas/CanvasRenderingContext2DBase.h: Canonical link: https://commits.webkit.org/266632@main

webkit-commit-queue · 2023-08-07T09:56:27Z

Committed 266632@main (5466cd2): https://commits.webkit.org/266632@main

Reviewed commits have been landed. Closing PR #16028 and removing active labels.

heycam requested review from cdumez and rniwa as code owners July 24, 2023 01:18

heycam self-assigned this Jul 24, 2023

heycam added the Canvas Bugs related to the canvas element. label Jul 24, 2023

webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Jul 24, 2023

litherum removed the merging-blocked Applied to prevent a change from being merged label Aug 4, 2023

litherum force-pushed the imagedata-cache branch from 9a09fc1 to d710d27 Compare August 4, 2023 18:04

litherum self-assigned this Aug 4, 2023

litherum marked this pull request as draft August 4, 2023 18:04

litherum force-pushed the imagedata-cache branch from d710d27 to 918c1f5 Compare August 4, 2023 18:35

litherum marked this pull request as ready for review August 4, 2023 18:35

litherum force-pushed the imagedata-cache branch from 918c1f5 to 4bfd474 Compare August 4, 2023 22:14

litherum marked this pull request as draft August 4, 2023 22:29

litherum force-pushed the imagedata-cache branch from 4bfd474 to 59629b5 Compare August 4, 2023 23:06

webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Aug 5, 2023

litherum removed the merging-blocked Applied to prevent a change from being merged label Aug 5, 2023

litherum force-pushed the imagedata-cache branch from 59629b5 to 3b419c8 Compare August 5, 2023 07:24

litherum marked this pull request as ready for review August 5, 2023 07:25

litherum approved these changes Aug 5, 2023

View reviewed changes

heycam commented Aug 5, 2023

View reviewed changes

Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp Outdated Show resolved Hide resolved

Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp Outdated Show resolved Hide resolved

litherum force-pushed the imagedata-cache branch from 3b419c8 to 478d2f2 Compare August 7, 2023 08:24

litherum force-pushed the imagedata-cache branch from 478d2f2 to 2a4beec Compare August 7, 2023 08:28

litherum added merge-queue Applied to send a pull request to merge-queue and removed merge-queue Applied to send a pull request to merge-queue labels Aug 7, 2023

litherum force-pushed the imagedata-cache branch from 2a4beec to be9ac84 Compare August 7, 2023 08:35

litherum added the merge-queue Applied to send a pull request to merge-queue label Aug 7, 2023

webkit-commit-queue force-pushed the imagedata-cache branch from be9ac84 to 5466cd2 Compare August 7, 2023 09:56

webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Aug 7, 2023

webkit-commit-queue merged commit 5466cd2 into WebKit:main Aug 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid going back to the GPU process for canvas ImageData when possible #16028

Avoid going back to the GPU process for canvas ImageData when possible #16028

heycam commented Jul 24, 2023 •

edited by webkit-early-warning-system

webkit-early-warning-system commented Jul 24, 2023 •

edited

webkit-early-warning-system commented Aug 4, 2023 •

edited

litherum commented Aug 4, 2023

webkit-early-warning-system commented Aug 4, 2023 •

edited

heycam commented Aug 4, 2023

webkit-early-warning-system commented Aug 4, 2023 •

edited

litherum commented Aug 4, 2023

webkit-early-warning-system commented Aug 4, 2023 •

edited

webkit-early-warning-system commented Aug 5, 2023 •

edited

litherum commented Aug 5, 2023

litherum commented Aug 5, 2023

heycam commented Aug 5, 2023

webkit-early-warning-system commented Aug 7, 2023 •

edited

webkit-early-warning-system commented Aug 7, 2023 •

edited

webkit-early-warning-system commented Aug 7, 2023 •

edited

webkit-commit-queue commented Aug 7, 2023

Avoid going back to the GPU process for canvas ImageData when possible #16028

Avoid going back to the GPU process for canvas ImageData when possible #16028

Conversation

heycam commented Jul 24, 2023 • edited by webkit-early-warning-system

5466cd2

webkit-early-warning-system commented Jul 24, 2023 • edited

webkit-early-warning-system commented Aug 4, 2023 • edited

litherum commented Aug 4, 2023

webkit-early-warning-system commented Aug 4, 2023 • edited

heycam commented Aug 4, 2023

webkit-early-warning-system commented Aug 4, 2023 • edited

litherum commented Aug 4, 2023

webkit-early-warning-system commented Aug 4, 2023 • edited

webkit-early-warning-system commented Aug 5, 2023 • edited

litherum commented Aug 5, 2023

litherum commented Aug 5, 2023

heycam commented Aug 5, 2023

webkit-early-warning-system commented Aug 7, 2023 • edited

webkit-early-warning-system commented Aug 7, 2023 • edited

webkit-early-warning-system commented Aug 7, 2023 • edited

webkit-commit-queue commented Aug 7, 2023

heycam commented Jul 24, 2023 •

edited by webkit-early-warning-system

`5466cd2`

webkit-early-warning-system commented Jul 24, 2023 •

edited

webkit-early-warning-system commented Aug 4, 2023 •

edited

webkit-early-warning-system commented Aug 4, 2023 •

edited

webkit-early-warning-system commented Aug 4, 2023 •

edited

webkit-early-warning-system commented Aug 4, 2023 •

edited

webkit-early-warning-system commented Aug 5, 2023 •

edited

webkit-early-warning-system commented Aug 7, 2023 •

edited

webkit-early-warning-system commented Aug 7, 2023 •

edited

webkit-early-warning-system commented Aug 7, 2023 •

edited