Flutter 3.22.0 -- iOS crash on app launch: 0x0000000108967408 NativeCanvas._restore$Method$FfiNative + 124 #149340
Comments
|
We have hundreds of similar crashes on 3.22, only on iOS. NativeCanvas._restore$Method$FfiNative 1 App 0x13b500 NativeCanvas._restore$Method$FfiNative + 1232384 (painting.dart:1232384) |
|
Hi @acoutts ➜ ~ dart pub global run symbolizer:symbolize https://github.com/flutter/flutter/issues/149340\#issue-2326350155 "flutter#5dcb86f arm64 release force ios"
...
symbolized using symbols for f6344b75dcf861d8bf1f1322780b8811f982e31a ios-arm64-release
#00 000000010272c2d4 Flutter (Missing)
#01 0000000108967408 App NativeCanvas._restore$Method$FfiNative + 124 (painting.dart:)
#02 0000000108d341b4 App _NativeCanvas.restore + 16 (painting.dart:580)
#03 0000000108d341b4 App BaseLayer.draw + 1716 (base_layer.dart:24)
#04 0000000108d5d904 App CompositionLayer.drawLayer + 480 (composition_layer.dart:10)
#05 0000000108d33d70 App BaseLayer.draw + 624 (base_layer.dart:19)
#06 0000000108d5d904 App CompositionLayer.drawLayer + 480 (composition_layer.dart:10)
#07 0000000108d33d70 App BaseLayer.draw + 624 (base_layer.dart:19)
#08 00000001089797c0 App LottieDrawable.draw + 768 (lottie_drawable.dart:23)
#09 00000001089794a8 App RenderLottie.paint + 140 (render_lottie.dart:26)
#10 0000000108db3350 App RenderObject._paintWithContext + 148 (object.dart:323)
#11 0000000108967f64 App PaintingContext.paintChild + 172 (object.dart:25)
#12 000000010896d74c App RenderProxyBoxMixin.paint (#2) + 60 (proxy_box.dart:13)
#13 0000000108db3350 App RenderObject._paintWithContext + 148 (object.dart:323)
#14 0000000108907130 App PaintingContext._repaintCompositedChild + 308 (object.dart:16)
#15 0000000108906dbc App PaintingContext.repaintCompositedChild + 176 (object.dart:10)
#16 0000000108906dbc App PipelineOwner.flushPaint + 288 (object.dart:118)
#17 0000000108906eb4 App PipelineOwner.flushPaint + 536 (object.dart:119)
#18 00000001088fe058 App RendererBinding.drawFrame + 88 (binding.dart:57)
#19 00000001088fdb14 App WidgetsBinding.drawFrame + 180 (binding.dart:113)
#20 00000001088fd460 App RendererBinding._handlePersistentFrameCallback (#2) + 40 (binding.dart:44)
#21 00000001088fd424 App RendererBinding._handlePersistentFrameCallback + 36 (binding.dart:44)
#22 0000000108d980d8 App SchedulerBinding._invokeFrameCallback + 56 (binding.dart:139)
#23 0000000108d97e7c App SchedulerBinding.handleDrawFrame + 320 (binding.dart:131)
#24 000000010887eac4 App SchedulerBinding._handleDrawFrame (#2) + 276 (binding.dart:117)
#25 000000010887e99c App SchedulerBinding._handleDrawFrame + 32 (binding.dart:115)
#26 000000010887f564 App invoke + 128 (hooks.dart:31)
#27 0000000108881080 App PlatformDispatcher._drawFrame + 36 (platform_dispatcher.dart:41)
#28 0000000108881044 App drawFrame + 56 (hooks.dart:28)
#29 00000001088810b0 App drawFrame (#2) + 24 (hooks.dart:28)
#30 0000000108857bf8 App stub InvokeDartCode + 216
#31 00000001029e94a4 Flutter (Missing)
#32 0000000102b01044 Flutter (Missing)
#33 00000001027eb8f0 Flutter (Missing)
#34 00000001027cb2c4 Flutter (Missing)
#35 00000001027fe230 Flutter (Missing)
#36 00000001026cf47c Flutter (Missing)
#37 00000001026d29f4 Flutter (Missing)
#38 000000019690f6e4 CoreFoundation __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 32 (CFRunLoop.c:179)
#39 000000019690f388 CoreFoundation __CFRunLoopDoTimer + 1004 (CFRunLoop.c:240)
#40 0000000196899184 CoreFoundation __CFRunLoopDoTimers + 288 (CFRunLoop.c:256)
#41 000000019689618c CoreFoundation __CFRunLoopRun + 1856 (CFRunLoop.c:312)
#42 0000000196895968 CoreFoundation CFRunLoopRunSpecific + 608 (CFRunLoop.c:342)
#43 00000001026d2ae0 Flutter (Missing)
#44 00000001026d2728 Flutter (Missing)
#45 00000001026d2438 Flutter (Missing)
#46 00000001f2845a90 libsystem_pthread.dylib _pthread_start + 136 (pthread.c:92)
#47 00000001f2844fcc libsystem_pthread.dylib thread_start + 8 (:-)I found an issue with a similar trace at #130478 which happened before 3.22.0 and still needs reproduction info to investigate. To confirm if this is a similar issue, does this happen before you upgrade Flutter? What time did you add Lottie package to your project? |
huycozy
added
the
waiting for customer response
|
Thanks @huycozy - we did not receive any reports until after we upgraded to 3.22 recently. We have ran lottie since day 1 and have not seen this crash since before we upgraded flutter. Since upgrading we've have 2 different users on different devices report the issue. |
github-actions
bot
removed
the
waiting for customer response
|
I found the cause. example The problem only occurs in version 3.22 and later. Currently, I'm using Flutter 3.22.1 in production after removing the renderCache and frameRate options from lottie. Additional |
|
@synstin I tried using lottie example with two params as above but couldn't reproduce the crash in @acoutts Are you also using two params as @synstin found above? It would be helpful if you could share a sample code (in case @synstin couldn't accomplish it) |
huycozy
added
the
waiting for customer response
github-actions
bot
removed
the
waiting for customer response
|
@synstin Ok after speaking to a coworker, we indeed are using lottie with the framerate set to |
|
We use lottie.asset without renderCache and framRate, but the issue still exists. By the way, we don't have lottie in logs Logs |
|
Labeling the issue for other's input. |
huycozy
added
c: crash
|
This does not happen in the simulator, only on the real device. Using only one of the renderCache and frameRate options will result in an error. |
|
@huycozy sadly I cannot reproduce it on my device either but I think we should leave it open for discussion because several other people here have reported it too. We downgraded to flutter 3.19 and the user who had repeated crashes is now able to open and use the app. We tried disabling impeller and that did not fix it but downgrading flutter did fix it. |
|
The DLBuilder code is here: https://github.com/flutter/engine/blob/main/display_list/dl_builder.cc#L587-L614 There were changes to restore that I believe landed in 3.22: flutter/engine@4b8218d#diff-600d175a73a116cd33a67521705e1f0115a0b03217683298d22d3fb4b53891ee @flar , can you take a look at this and see if there are any cases where perhaps a mismatched save/restore could cause a native crash? |
jonahwilliams
added
P1
|
Is there a minimum reproducible test case available? Has anyone who experienced the issue tried using Flutter ToT? |
|
I poked around a second. I suspect void DisplayListBuilder::Restore() {
if (save_stack_.size() <= 1) {
return;
}
if (!current_info().has_deferred_save_op) {
SaveOpBase* op = reinterpret_cast<SaveOpBase*>(storage_.get() +
current_info().save_offset);
FML_CHECK(op->type == DisplayListOpType::kSave ||
op->type == DisplayListOpType::kSaveLayer ||
op->type == DisplayListOpType::kSaveLayerBackdrop); |
The intent is that a regular |
|
I believe this is the version in 3.22.0? |
flar
added
the
waiting for customer response
|
@flar is there anything you're waiting on me for? Saw the customer response label get added. |
github-actions
bot
removed
the
waiting for customer response
I was chatting with him earlier today. He's hoping that reproduction code could be isolated. I'm not sure how feasible that is given your project. |
|
Ah i see. Yea the thing is, nobody on our team can reproduce it either - there seems to be something specific about the users and their devices. The only common thing I've seen is the 2 users who had crashes had very low disk space on their phones (2-3gb available). Could free disk space be relevant here? |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
@RoyGuanyu that seems very unrelated to this bug, which is a crash. You need to file a new bug |
As @gaaclarke mentioned a reproducible test case would be best as we could diagnose the underlying problem. But the other thing I'm still waiting on is whether more recent releases (even developer channel releases) fixes the problem. The 3.22 release contained a change to the indicated code, but immediately after that we rewrote that code - the rewrites did not make that release so testing against the ToT rather than a production release might give us an indication if the rewrite solved the problem. |
|
If it doesn't reproduce on ToT then bisecting to find out when it was fixed might help us identify a change we can cherry-pick into a dot release. |
|
Adding some additional stack traces to this. Also getting a number of crashes in iOS with the same error. We're getting a huge number from a very specific screen though which uses a very specific package called |
|
Hi, I have a very similar problem in #149971 (closed, to discuss it here). I can not reproduce it. But I'm using the So far I have only heard about this crash from blind users who use VoiceOver. I know that VoiceOver needs much RAM. |
|
@flar I have a new theory. I think |
|
What is the right thing to do if realloc fails? it seems like the best we could do at that point would be to crash anyway |
I think you can |
|
I was was wrong, I'm not sure if the reproduction for this is to the point of being unreasonable, like drawing a billion things. |
Not that I am certain its the case that the app is out of memory. But if we're out of memory to the point that malloc is failing we'll just crash somewhere else. We can't reasonable guard all allocations (plus what do STL classes do in that case)? |
|
We should still try and find a repro for this though, because I'm not certain this is an OOM anyway. |
|
Could this be a case of the dependent packages (Lottie? Confetti?) creating a DL that uses so much memory that reallocating from N bytes to N + page_size bytes is the most likely allocation to run out of room? But why does this suddenly happen with that one PR? It didn't increase the space consumption of the DLs by much. Potentially if they were using unbounded saveLayers like candy then that PR started always recording a bounds for every saveLayer which adds an SkRect to each saveLayer, but I'd find it hard to believe that an additional 16 bytes per saveLayer would be enough to push apps to crash so suddenly. |
|
A repro would be best so we can diagnose how to respond. But, barring that, a retest by someone who experiences the crash on more recent releases/builds (especially ToT) would help to know if the recent redesign of this code fixed the problem... |
Just to clarify, it's possible the display list can fail to realloc but that we are not out of memory. We could be requesting a 4MB region of memory and there doesn't exist one because of fragmentation, but there is well over 4MB available in aggregate. I double checked the algorithm for our realloc size and it isn't growing exponentially, so that's good.
Waiting for more information sounds good to me. Chunking could have better performance if I will say though, if the bug requires memory to get into a certain state of fragmentation to fail, it's not going to be easy to reproduce since it may require using the app for a certain amount of time to get in the state where it fails.
I'm less sure about definitively pinning this to one PR without a reproduction. There's a lot of reasons why this could all of a sudden just start getting reported now outside of the engine's control. |
That depends. If the malloc implementation isn't good about consolidating freed memory then the fragmentation by growing linearly could end up causing more problems. Each page should be good for a lot of ops, though, so growing by the page size (4k or 8k?) shouldn't happen very often. |
|
I'm having this issue with Flutter stable 3.22.1 I'm not using lottie or confetti. I had not seen the problem during development, but after distributing the app, and installing from Appstore in the same device, it started to close immediately. I commented and uncommented the code several times to verify that it was causing the issue, and the app only crashed when it was on. I thought it may have been related to app distribution process and certificates, because it is a new app and the error went away on its own. Today I saw some more reports, and they are coincident with the release of a second version of the app. Heres a stack trace of the issue: Regards |
|
Some more info hoping to give some valuable info to someone investigating this: i keep receiving this crashes, always with this structure: painting.dart:6006 is the implementation of drawPath: MapaBackPainter.dibujarMedio is the one that always triggers the error in my app. Is fairly simple: It's doing saveLayer and restore. However at the time I could test this problem locally, I found that removing the call to _dibujarEstacion stopped the crash. That function calls save and restore again: |
|
Keep receiving this crash when app starts video play using video_player ibrary. iOS: 17.5.1 Logs:Screenshot
|
Steps to reproduce
We have received beta feedback in testflight that the app is crashing for some users right after they install it for the first time and try to launch it. We recently upgraded to Flutter 3.22.
Unfortunately I cannot reproduce it on any of my devices but we've received 3 reports so it is affecting several people now. I attached the stack traces below:
stack traces.zip
Expected results
App should not crash from flutter
Actual results
App crashes immediately on launch for some users.
Code sample
Code sample
[Paste your code here]Screenshots or Video
Screenshots / Video demonstration
Logs
Logs
Flutter Doctor output
Doctor output
The text was updated successfully, but these errors were encountered: