Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use toPictureSync for faster zoom page transition #106621

Merged
merged 65 commits into from
Jul 22, 2022

Conversation

jonahwilliams
Copy link
Member

@jonahwilliams jonahwilliams commented Jun 26, 2022

Rather than using transform layers, snap off a texture of the current/next route using toGpuImage, and then draw the scale/opacity operations using these images while the animation is ongoing.

Edit: Updated because I realized I was accidentally drawing one of the layers twice + dpr

Pixel 6

  toGpuImage ToT   Delta
average_frame_rasterizer_time_millis 2.704333333 6.932666667   4.228333333
90th_percentile_frame_rasterizer_time_millis 3.108333333 13.91833333   10.81
99th_percentile_frame_rasterizer_time_millis 13.19466667 21.64666667   8.452
worst_frame_rasterizer_time_millis 16.031 24.299   8.268

Wembly

  toGpuImage ToT   Delta
average_frame_rasterizer_time_millis 6.576966868 10.93125447   4.354287606
90th_percentile_frame_rasterizer_time_millis 8.874 15.01333333   6.139333333
99th_percentile_frame_rasterizer_time_millis 10.67266667 20.96766667   10.295
worst_frame_rasterizer_time_millis 33.301 39.167   5.866

Fixes #104680
Fixes #100972

@flutter-dashboard flutter-dashboard bot added f: material design flutter/packages/flutter/material repository. framework flutter/packages/flutter repository. See also f: labels. labels Jun 26, 2022
@jonahwilliams
Copy link
Member Author

FYI @dnfield , this is awesome

@@ -250,18 +253,137 @@ class _ZoomPageTransition extends StatelessWidget {
}
}

class _ZoomEnterTransition extends StatelessWidget {
abstract class _RadicallyAwesomeRenderObject extends RenderProxyBox {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonahwilliams
Copy link
Member Author

Ahh shit, web

@jonahwilliams
Copy link
Member Author

Updated numbers with pixel 6 benchmark, the improvement is even more dramatic, partially due to a new benchmark I wrote which isolates the page transition better

@jonahwilliams
Copy link
Member Author

The fidelity of this is a little off, but promising results so far. Hopefully fixing the fidelity doesn't tank the performance...

@flutter-dashboard flutter-dashboard bot added the team Infra upgrades, team productivity, code health, technical debt. See also team: labels. label Jun 26, 2022
// found in the LICENSE file.

import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart' hide TypeMatcher, isInstanceOf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw I am curious why isInstanceOf is hidden?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remember, its just a copy paste from the other test

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it causes type definition clashes because flutter_test defined this before it was availalbe in package:test or something.

@jonahwilliams
Copy link
Member Author

I also investigated total frame count after @dnfield raised some concerns about potentially hidden costs of toGpuImage. I found that with toGpuImage, the attached benchmark pumps about 208 frames. With the old version, we get about 65 frames - on a pixel 6.

Comment on lines 402 to 408
final Rect src = area;
final double newWidth = src.width * scale / pixelRatio;
final double newHeight = src.height * scale / pixelRatio;
final double leftOffset = (src.width / pixelRatio - newWidth) / 2;
final double topOffset = (src.height / pixelRatio - newHeight) / 2;
final Rect dst = Rect.fromLTWH(src.left + leftOffset, src.top + topOffset, newWidth, newHeight);
context.canvas.drawImageRect(image, src, dst, paint);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this math can be a little confusing if you're not thinking about dp vs physical pixels. I think this could be more easily done by putting a scale on the canvas itself and just using drawImage

@jonahwilliams
Copy link
Member Author

Flutter DevTools - Google Chrome 6_27_2022 1_34_24 PM

I'm observing some odd GC behavior, it seems to trigger consistently when calling toGpuImage

@jonahwilliams
Copy link
Member Author

Filled #106698

}
final bool updateImage = delegate.willPaint(animation);
if (_childImage == null && updateImage) {
_childImage = PaintingContext.paintAndDetachToGpuImage(
Copy link
Contributor

@fzyzcjy fzyzcjy Jun 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we have trouble if the children is changing during animation? For example, there may be a loading indicator that is animating during this zoom page transition. Or there may be a splash effect happening because user has just tapped a button. Since zoom page transition is a very general transition, we may not be able to assume all things like loading indicator or splash effect never happen to be a child in the subtree.

In those cases, the content (i.e. childImage) actually should be updated, but we are caching it and never update it except when animation status changes.

Therefore, users may see the loading indicator or splash "freezed" during this zoom IMHO.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a price I am willing to pay

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. But maybe we should have some flags allowing users to customize it? Or maybe some documentation saying this is expected behavior - or people may file a bug about it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@goderbauer did we ever change route transitions to turn the ticker mode off, or was that just in customer: money's app?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect that would not have improved raster performance at all for zoom page, given that the scale transform would invalidate all raster caches

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was for the opacity one. Animation was busting the raster caching there

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense. We could apply this sort of fix to the other route transitions but since the current zoom page transition doesn't work with any raster cache heuristics, then changing this transition will have an oversized impact.

Not clear to me if we have other route transitions that are as heavily used

Copy link
Member

@goderbauer goderbauer Jun 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did we ever change route transitions to turn the ticker mode off, or was that just in customer: money's app?

We did something like that for customer:money for some transition types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a price I am willing to pay

I am going back and forth on this. On the one hand, this is also how iOS handles transitions, I believe. On the other hand, this was a cool feature of flutter that we could do animations while transitioning...

At the very least, we need to document this behavior somewhere. I also wonder if we need a flag to opt out of this behavior?

One example where you absolutely want animations to run during transitions are Hero animations. It's the whole point of a Hero that they fly around while you transition between routes. How does this change affect Heros? (Maybe they are unaffected, because the Hero is placed in a separate Overlay?)

Also, there are transitions where when you trigger them manually (e.g. by swiping back) things actually animate on the screen (e.g. a back-arrow animates back to a Hamburger Menu button). Are any of these use cases impacted?

@jonahwilliams jonahwilliams marked this pull request as ready for review June 28, 2022 16:01
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022
camsim99 pushed a commit to camsim99/flutter that referenced this pull request Aug 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: text input Entering text in a text field or keyboard related problems autosubmit Merge PR when tree becomes green via auto submit App customer: money (g3) f: cupertino flutter/packages/flutter/cupertino repository f: material design flutter/packages/flutter/material repository. f: routes Navigator, Router, and related APIs. framework flutter/packages/flutter repository. See also f: labels. team Infra upgrades, team productivity, code health, technical debt. See also team: labels. will affect goldens Changes to golden files
Projects
None yet
8 participants