[web] canvaskit performance issue

[laurensdewaele]

Details

1. To reproduce: put scaled images in a Stack wrapped with an InteractiveViewer (see gist)
   The problem is that the performance is fine on macOS desktop target, however on Chrome with canvaskit, the performance really suffers.
   Test was done with --release and --web-renderer canvaskit

2. Sample app

3. Video showcasing osx vs chrome

**Target Platform: macOS / Web
**Target OS version/browser: 11.5.2 / 92

Logs

Logs

[✓] Flutter (Channel master, 2.5.0-7.0.pre.188, on macOS 11.5.2 20G95 darwin-arm, locale en-BE)
    • Flutter version 2.5.0-7.0.pre.188 at /Users/laurens/Library/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 369a0e2bc0 (65 minutes ago), 2021-08-23 09:17:04 -0400
    • Engine revision 710af46d53
    • Dart version 2.15.0 (build 2.15.0-42.0.dev)

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/laurens/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 12.5.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio 4.2 Preview.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 11.5.2 20G95 darwin-arm
    • Chrome (web)    • chrome • web-javascript • Google Chrome 92.0.4515.159

! Doctor found issues in 1 category.

[danagbemava-nc]

Issue is reproducible on master 2.5.0-7.0.pre.213 when using the canvaskit web-renderer. The html web-renderer has considerably better performance. On stable the performance with both renderers is about the same, which makes it seem like this issue may have been fixed for the html renderer on master. The performance is fine on macOS on both stable and master.

See profile reports below:

canvaskit-stable.json.zip
html-renderer stable.json.zip

html-renderer.json.zip
canvas-kit.json.zip

macOS
dart_devtools_stable.json.zip
dart_devtools_master.json.zip

flutter doctor -v

[✓] Flutter (Channel master, 2.5.0-7.0.pre.213, on macOS 11.5.1 20G80
    darwin-arm, locale en-GB)
    • Flutter version 2.5.0-7.0.pre.213 at /Users/nexus/dev/sdks/flutters
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6dc18ef74e (7 hours ago), 2021-08-24 11:09:04 +0530
    • Engine revision 926ce0d855
    • Dart version 2.15.0 (build 2.15.0-43.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.5.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code (version 1.59.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.25.0

[✓] Connected device (3 available)
    • M2007J20CG (mobile) • 5dd3be00 • android-arm64  • Android 11 (API 30)
    • macOS (desktop)     • macos    • darwin-arm64   • macOS 11.5.1 20G80
      darwin-arm
    • Chrome (web)        • chrome   • web-javascript • Google Chrome
      92.0.4515.159

• No issues found!

[✓] Flutter (Channel stable, 2.2.3, on macOS 11.5.1 20G80 darwin-arm, locale
    en-GB)
    • Flutter version 2.2.3 at /Users/nexus/dev/sdks/flutter
    • Framework revision f4abaa0735 (8 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code (version 1.59.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.25.0

[✓] Connected device (3 available)
    • M2007J20CG (mobile) • 5dd3be00 • android-arm64  • Android 11 (API 30)
    • macOS (desktop)     • macos    • darwin-arm64   • macOS 11.5.1 20G80
      darwin-arm
    • Chrome (web)        • chrome   • web-javascript • Google Chrome
      92.0.4515.159

• No issues found!

[yjbanov]

The images are very big (4000x4000, which requires 64MB of memory assuming 4-bytes per pixel). I'm guessing that loading them all at the same time maxes out the GPU memory limit causing the images to be kicked out of the cache and reuploaded again on every frame. Flutter currently does not automatically downscale images. Chrome probably does (when using the HTML renderer).

@laurensdewaele Can you please try increasing the GPU memory limit and see if that improves performance? To increase the limit call first thing in the main function:

await SystemChannels.skia.invokeMethod('Skia.setResourceCacheMaxBytes', 1000000000);

[yjbanov]

Once we confirm the hypothesis that this is a GPU memory limit issue, we can start thinking about solutions. For example, one solution (and one that's generally recommended on the web) is to use smaller images for thumbnails. However, it's also fair to say that Flutter shouldn't be worse than HTML, and we could attempt a heuristic similar to what browsers do, such as downscale the image automatically.

[laurensdewaele]

@yjbanov,
Thank you for investigating this further.

Increasing the GPU mem limit does solve the performance problem for the reproducible demo app.
Since it's fine on other embedders apart from CanvasKit, I'm hoping we don't have to open #56482 again?

The app we're building, ideally, needs to be capable to handle a lot of high-res images in a sort of InteractiveViewer widget.
The way the framework works at the moment, we're hitting the default limit of ImageCache (100MB) rapidly after only 2 or 3 large images. Apart from that, it's also possible to overload the GPU cache.

What would be the ideal implementation here?

• On the framework side, cache the encoded images in the ImageCache instead of the decoded ones. #47378 and #50024
• Account for the amount of 'zoom' the user currently has on the canvas, to determine what image granularity they actually need. We could then set cacheHeight and cacheWidth to limit what gets sent to the gpu.
• Implement deferred image loading when scrolling too fast.

Any thoughts? (@dnfield probably knows more about this)

[dnfield]

@yjbanov are you sure the resource cache in Skia is used for images? We don't use it on the VM side, we actually disable the resource cache on the thread responsible for image decoding.

You are right that these images are really big and using a lot of memory. You might try to increase the image cache's limit, but it's probably not relevant here since the widget tree contains all the images anyway up front, and even if they don't fit in the limits of the cache they'll get tracked that way.

I'd recommend trying to add a cacheHeight at the largest setting you expect the images to be rendered at. That will make it take much less memory and should speed things up.

[yjbanov]

cacheHeight/cacheWidth are translated to targetHeight/targetWidth on the engine side. I will tentatively assign this issue to @alanwutang11 who is working on this in flutter/engine#38028. I suspect to fully fix the issue, a follow-up in the framework will be needed. Currently the framework uses this code to decode network images on the web for both HTML and CanvasKit renderers, and that code ignores cacheHeight/cacheWidth. We will need to change the framework to always use the if branch for CanvasKit. We only need webOnlyInstantiateImageCodecFromUrl in HTML mode. For that, we'll need the framework to know that the app is running in CanvasKit mode. However, the current implementation of isCanvasKit is incorrect. We'll need to fix that too.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.