Flutter adds additional one frame touch latency compared to native

[knopp]

Bad	Expected
tracking-bad.mov	tracking-expected.mov

Repository: https://github.com/knopp/flutter_scroll_overlay

When touch event is received, instead of immediately beginning a frame Flutter waits for next vsync. But that skips an entire frame. By the time vsync comes, the frame should be already rendered and ready to go. But instead it has just began. On iOS touch events seem to be delivered within few milliseconds of the display link callback so there should be enough time.

You can pause the "bad" video to see that during scrolling Flutter lags behind native by one frame exactly.

The expected video is with quick and dirty engine modification that forces BeginFrame immediately on input event.

The way vsync is implemented in Flutter might need some work. It should make sure that repaints are synchronized with display updates, but it shouldn't be causing additional latency.

[knopp]

To clarify - in first video there are frames like this:

where the flutter scroll view doesn't keep up with UITableView (on the right) because is always one frame behind. The video on the right stays aligned at all times.

[knopp]

cc @iskakaushik

[theniceboy]

https://www.youtube.com/watch?v=N66MdEDHh2Q

This is a slo-mo video recorded that can demonstrate this issue.

Up top is the Wonderous app, and down below is a swift app.

[pedromassango]

@knopp were you able to reproduce this on Android? Seems iOS specific, can't reproduce on Android Emulator so far.

[jmagman]

cc @cyanglaz

[Hixie]

@knopp btw please don't hesitate to land your test app in https://github.com/flutter/platform_tests if you want to. Collecting test apps like this is what that repo was made for.

[cyanglaz]

I'm not currently working on this so I'll un-assign myself. I might circle back to this eventually.
@knopp Do you mind to open a draft PR with your engine modification that results the right video?

[alidcast]

@cyanglaz did you see this as being reasonably in-reach to resolve?

There's comments about Flutter being in an "uncanny valley" - close to native, yet slightly different in an uncanny way. Outside of just saying "performance" or "jank" I couldn't identify why scrolling feels off in every flutter app I tried. So I find relief at finding issues like this since they identify why that's the case, and fixing these issues overtime make true native experience feel feasible! 🤞

If I may suggest, this feels like a higher priority issue as scrolling is a major interaction in any app.

[yashwant1999]

@knopp do you have any workaround regarding this or any PR or may any sample repo with your modification. I wish I could help but due to limited knowledge, i can't. And this is first think i always notice when i run my app . As a perfectionist, you foolishly try to split flutter app and other app side by side and try to compare but there is always a feeling of something being "off" but can't figure out. i tried all best performance pratice but i think this is the issue if i am not wrong.

[Juliotati]

@yashwant1999 meanwhile the requested PR is not here, I can share something that might be related to this which I saw back them.

flutter/packages/flutter/lib/src/widgets/scroll_physics.dart

Line 441 in e3429dd

double? get dragStartDistanceMotionThreshold => parent?.dragStartDistanceMotionThreshold;

flutter/packages/flutter/lib/src/widgets/scroll_physics.dart

Line 751 in e3429dd

double get dragStartDistanceMotionThreshold => 3.5;

[knopp]

@Juliotati, this particular issue needs to be fixed in the engine, not framework.

[movaid7]

@knopp is this behaviour present when using an Impeller runtime?

[jessetabak]

I’ve had this experience for years, scrolling always feels off, muddy, laggy. It’s the main reason I never got into flutter. I need snappy scrolling 😉

[cyanglaz]

@knopp I tried to repro but I'm seeing a different behavior. The scrolling acceleration seems different between iOS and Flutter. Could you post your workaround so I can try it too?

[knopp]

@knopp I tried to repro but I'm seeing a different behavior. The scrolling acceleration seems different between iOS and Flutter. Could you post your workaround so I can try it too?

This is not about scroll physics. This is just about the tracking phase (following finger while the finger is down). The physics only comes into play after lift (and is indeed slightly different but that's not the point here).

I don't have the changes anymore, it was just a quick and dirty hack to trigger beginning frame immediately after event (instead of scheduling vsync), but i didn't pus it anywhere.

[theniceboy]

Seeing this behavior on iPad scrolling with the Apple Pencil. You can see that the iOS recognizes the pencil first and triggers the scribble API (which is a Flutter bug), and a few frames after that Flutter would react.

I'm on Flutter 3.7.5

Upload.from.GitHub.for.iOS.MOV

Video recorded at 240fps. Sorry the video is short, it helps if you scrub through the timeline

[aytunch]

@ slowmotionbot

Here is your slowed down video with 5x
https://user-images.githubusercontent.com/6442915/221829162-1fdeb20c-7c08-45cf-b52c-83e552f89feb.mp4

[bobmoff]

@knopp Is there any chance that you could redo that change you made and share it with us so that we might experiment and perhaps make an example repo for the Flutter team to try?

If you dont have time, perhaps you could just point us in the right direction? Any help is appreciated 😁.

[qfournetworks]

Maybe helpful additional info: This issue seems to only apply to IOS running 60hz.

I'll look further into resolving this issue soon.

Some of our customers noticed this issue running our app on "60hz IPhones"

[knopp]

Couldn't it be just that is much less obvious at 120Hz? As far as I can tell the frame cycle should be the same, the latency should be still present just shorter in absolute numbers.