We don't pump frames on Android when the screen is split

[Hixie]

If you run a Flutter app in split-screen mode, we don't pump frames once you tap the other app.

[Hixie]

This is a new-user-experience issue that affects everyone.

The problem is probably that we use the wrong app lifecycle state for the split-screen mode on Android. We should be using "inactive".

[Hixie]

cc @amirh who said he might be interested in this bug.

[cbracken]

/cc @jason-simmons

[jonahwilliams]

@jason-simmons If you're not looking at this right now I could take a look, since I'm blocked on my other work for a while...

[jonahwilliams]

This also requires us to add screenLayout to android:configChanges, otherwise the app will restart when switching into and out of split screen

[jonahwilliams]

Looked at this today - It seems the only change we would need to make is to also consider the lifecycle Activity#onMultiWindowModeChanged() and use Activity#isInMultiWindowMode() to avoid pausing and resuming the flutter application. These methods don't show up until API level >= 24, so I think we might need to bump the version of the android_support library.

[amirh]

Hi @jonahwilliams :)
I believe the current Android Activity lifecycle documentation for onPause reads more as the activity lost focus rather than is being terminated or is invisible.

While multi-window is the most obvious case for being not in focus but visible, they also mention dialogs that can pause your activity while keeping it visible.

I believe we shouldn't stop pumping frames when the activity loses focus. e.g think of a stopwatch application, you probably still want to see the stopwatch running when losing focus due to e.g multi-window or a dialog. While the documentation say that onPause is a good place to stop animations, I don't believe it should always stop all animations, and apps should probably still be able to update the screen while not in focus but visible.

For these reasons (and probably for future compatibility as onPause now doesn't mean invisible) I would suggest we consider to not restrict the solution to multi window detection, and allow apps to keep updating the screen even when not in focus.

[jonahwilliams]

Thanks for the background @amirh. I believe @Hixie and @cbracken could give more background on why the decision was made original to stop pumping frames entirely in onPause.

Ideally we would stop pumping frames only when the app was "invisible." That concept is poorly defined in the Android world, but onPause and onResume are a good proxy. Except of course for all the times that they aren't, like you mentioned:

• Splitscreen
• Under a dialog
• Picture in picture?
• probably many more

I don't think we should necessarily give up just because there are edge cases though, we can fix them. As the ultimate work around, you can always override onPause and onResume in your FlutterActivity so rendering is never paused.

[amirh]

Would it be reasonable to stop pumping frames onStop ("Called when you are no longer visible to the user")?

[cbracken]

Keep in mind that due to significant differences in the Android and iOS lifecycle platform callbacks these map to somewhat different concepts on the two platforms. On iOS it's particularly important not to pump frames once backgrounded, since the result is an abort(). IIRC onPaused maps to the inactive foreground state on iOS -- (examples of this state are the app is current, then you open the app switcher, or the app is current but you're in the in-call/touchID screen). onStop, on the other hand, is invoked right as the app is backgrounded on iOS, and almost never invoked on Android (or at least that's my over-1-year-old recollection -- wire it up and test)

[Hixie]

why the decision was made original to stop pumping frames entirely in onPause

Probably because we didn't know what we were doing.

[Hixie]

I agree with @amirh's interpretation, that we shouldn't stop painting on Android when we're still visible. We continue painting on iOS. We are probably just mapping the Android lifecycle events to the wrong values of the lifecycle enum.

[cbracken]

We are probably just mapping the Android lifecycle events to the wrong values of the lifecycle enum.

This is pretty likely; I certainly didn't have split-screen in mind when I originally added this code.

[jonahwilliams]

Okay - I will do some more investigation into onStop and onPause in the android world. If I can simplify that code then I can probably avoid using the more specific split screen APIs

[cbracken]

I'd definitely look at instrumenting the two calls and gathering some empirical evidence for what they do. IIRC the documentation at the time was quite vague on when each was called. It's entirely possible they've added more detail to the docs now with split-screen in mind.

[jonahwilliams]

Some more investigation -

If you use picture in picture mode with another app, the flutter app will also pause if you tap into that

[jonahwilliams]

The original addition of the onPause and onResume hooks is here flutter/engine@6f0c2d0983. It seems like the intention was to ensure we weren't needlessly pumping frames.

[jonahwilliams]

VsyncWaiter may be handling this for us - we don't get doFrame callbacks when the App is completely invisible

[Hixie]

@chinmaygarde can speak to this more, but I believe the plan is to get the engine out of this business and have the Dart code entirely in charge of when we push frames.

[jonahwilliams]

We might need to look at the iOS side too, though android.view.Choreographer doesn't seem to push any frames when the app is completely invisible - even when I remove the onPause/onResume hooks.

[chinmaygarde]

It is entirely possible (and likely) that the choreographer on Android is making decisions about when to give the engine vsync pulses. You can try switching to a timer based fallback vsync waiter to check to see if this indeed the issue. Patch in this method on Android and return VsyncWaiterFallback instead like in the base platform view implementation.

If it turns out that the we depend on the vsync waiter to give us frames even when the application is inactive, we might need to introduce a switching mechanism to use the fallback when the platform vsync waiter refuses to give us events.

[amirh]

@chinmaygarde if the choreographer doesn't call the callback because we're invisible, why should we pump frames?
If we can rely on that than maybe on Android we just don't need to stop frames based on lifecycle, just rely on the choreographer not calling the frame callback?

[chinmaygarde]

If we can rely on that than maybe on Android we just don't need to stop frames based on lifecycle, just rely on the choreographer not calling the frame callback?

After initially skimming this thread, I thought that the choreographer was not giving us frames even when it was on screen but the user had focused another application in split screen mode. If this is not the case and the choreographer works as expected when the application is not focused but still present on screen, then lets not stop the animator based on lifecycle.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.