Flutter app freezes and VM service stops responding when adding breakpoints to multiple isolates at the same time #54650
Comments
|
Nope, no ideas off the top of my head. We'll need a repro to make any progress on this, unfortunately. |
|
@DanTup Willing to share the project as it is, just needs docker to fully reproduce. But I'll try strip as much from it as I can tomorrow too and see what happens. |
|
@mellowcello77 can you strip out the bits that require docker? If the freeze just occurs when setting breakpoints (and having isolates), seems like it shouldn't be necessary. Please also don't share anything sensitive/confidential. The simpler the repo the better. Can you also confirm your OS, Flutter version and the device you're running on (if a physical device, can you confirm whether the issue also repros on the desktop device or a simulator)? Thanks! |
a-siva
added
area-vm
|
/cc @aam |
|
Thanks to info posted at Dart-Code/Dart-Code#4926 (comment) by @QCIPaulCardno I think I've managed to get a reliable repro for this. Currently running this on M1 Mac OS desktop device (so far have been unable to repro on Intel Mac or Windows) using current Flutter stable. The code spawns some isolates that just count, with the counts shown on the app: import 'dart:isolate';
import 'package:flutter/material.dart';
void main() {
runApp(const MainApp());
}
class MainApp extends StatefulWidget {
const MainApp({super.key});
@override
State<MainApp> createState() => _MainAppState();
}
class _MainAppState extends State<MainApp> {
final _counters = List.filled(10, 0);
@override
void initState() {
super.initState();
for (var i = 0; i < _counters.length; i++) {
// BP 1
var myReceivePort = ReceivePort();
// BP 2
Isolate.spawn<SendPort>(_count, myReceivePort.sendPort);
// BP 3
myReceivePort.listen((message) {
setState(() {
_counters[i] = message;
});
});
}
}
static void _count(SendPort mySendPort) async {
for (var i = 0;; i++) {
mySendPort.send(i);
await Future.delayed(const Duration(milliseconds: 100));
}
}
@override
Widget build(BuildContext context) {
return MaterialApp(
home: Scaffold(
body: Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
for (var (index, counter) in _counters.indexed)
Text('Isolate $index: $counter'),
],
),
),
);
}
}When the app is running, I add breakpoints to the line under breakpoint_freeze.movComplete DAP + VM log here: |
|
I wasn't able to repro with the sample above on Intel Mac or on Windows. I don't know if that means it's specific to ARM Mac though or if there might just be some timing issue or something that differs between them. |
|
@DanTup I just ran your repro code and doing exact same on my M2 as in your video. Tested on both iOS and MacOS. |
|
@aam do you think you can take a look? I don't have access to an M1 device. |
I would be surprised if it's mac or m1 specific. I will try to reproduce it though. |
|
It's the macOS Desktop device where I can repro this, no Android involved. It'll probably be faster if you can reproduce it, but otherwise I'm happy to run commands and provide more info if I can (note: I've not used lldb/gdb much so I might need explicit steps). |
|
@DanTup do you think you will be able to reproduce the same using devtools (want to get vscode out of the equation) |
|
I can certain give it a go. |
a-siva
removed
the
P1
|
@DanTup wrote
Ah, then can you try |
|
Yep, I can repro with DevTools (with the app run from the terminal, no VS Code running). I had to toggle the second breakpoint a few times (I did not do this particularly quickly): devtools_repro.mov |
|
@aam does this look right? |
|
@DanTup thanks, but I expect to see more threads (named DartWorker and such) in the flutter dart vm process. You should be able see them if you say |
|
Ah, seems like Here's And the full output of https://gist.github.com/DanTup/cea9c51d1ad75472d01767c8c0c88528 |
aam
removed
the
needs-info
|
The deadlock is caused by thread |
|
A user reported this issue in the vs-code channel on the Flutter discord and believes they don't have multiple isolates, so it might be that multiple isolates is not required to trigger this (but perhaps it makes it more likely, as we'll be sending more add/remove breakpoint requests). |
|
I'm not sure why this was not encountered before, people started to report it only recently. |
|
Yeah, I wondered that too. At first I thought maybe it was due to more people using the new SDK debug adapters in the latest release (up from 50% to 100%) and maybe the behaviour is slightly different there, however reports are that it occurs using the legacy adapters too, and the poster on Discord suggested it was happening in both Android Studio and VS Code. So far all reports I've seen (and the only place I can repro) is still ARM Macs.. I don't know whether that's coincidence or there's something making it more likely to trigger there. |
|
This is also impacting Android Studio. https://discord.com/channels/608014603317936148/958861820507078686/1196120595172171877 |
bkonyi
added
P1
|
Is there anything we can do in the meantime, maybe a downgrade? I can't touch breakpoints in my projects, which is, of course, making it super hard to debug this way. Thanks again for looking into this. |
Unfortunately not. This bug has seemingly been present for awhile, so it's not clear why developers are encountering it frequently all of a sudden. We do have a fix ready and we should see if we can get it into a hotfix release. |
|
Is this a cherry pick candidate ? |
I think so, it seems like it's impacting quite a few users. |
|
I think this should be cherry-picked given 6 people have now thumbs up the VSCode issue and the severity of the issue when it occurs. |
|
The fix for this landed in flutter flutter/flutter@676e322, @DanTup would you be able to give it a try? |
|
A cherry pick request for this has been filed here #54699 |
…ions locks and Ensure setting breakpoints is lock-safe. TEST=DartAPI_BreakpointLockRace and DeoptimizeFramesWhenSettingBreakpoint Fixes #54650 flutter/flutter#140878 Acquire reload opreation scope when deoptimizing the world to ensure locks can be acquired for compilation. Set up scope for operations that can be run while the world is deoptimized and stopped to avoid races. Ensure code stays unoptizimed when single stepping, prevent other isolates to reoptimize. Bug: #54650 and flutter/flutter#140878 Change-Id: I9a88096f15a34b645281e5b2b3805a73dd93672e Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/347420 and https://dart-review.googlesource.com/c/sdk/+/345743 Cherry-pick-request: #54699 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/347650 Reviewed-by: Alexander Aprelev <aam@google.com> Commit-Queue: Kevin Chisholm <kevinjchisholm@google.com>
I just tested with the change before that and confirmed I could still reproduce the issue (although it did require toggling the breakpoint a few times). Then I tried on latest and I was not able to reproduce the issue even with a significant amount of very quick breakpoint toggling. As far as I can tell, the issue is definitely fixed. Thanks! :-) |
…ns locks and Ensure setting breakpoints is lock-safe. Acquire reload opreation scope when deoptimizing the world to ensure locks can be acquired for compilation. Set up scope for operations that can be run while the world is deoptimized and stopped to avoid races. Ensure code stays unoptizimed when single stepping, prevent other isolates to reoptimize it. Fixes: #54650 flutter/flutter#140878 TEST=DartAPI_BreakpointLockRace and DeoptimizeFramesWhenSettingBreakpoint Bug: #54650 and flutter/flutter#140878 Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/347420 and https://dart-review.googlesource.com/c/sdk/+/345743 Cherry-pick-request: #54699 Change-Id: Ia4bc883121dac978fbb76027906a810000ef1138 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/347760 Reviewed-by: Alexander Aprelev <aam@google.com> Commit-Queue: Siva Annamalai <asiva@google.com>
|
I just verified the issue is no longer reproducible for me using the new Flutter stable (3.16.9). If you were hitting this, please run |
This was raised at Dart-Code/Dart-Code#4926 and seems to be affecting a number of users. I haven't managed to reproduce it yet.
The users can initially add breakpoints fine, but at some point when they try to modify the breakpoints, the Flutter app freezes and stops responding. In the logs, it seems that the VM Service never responds to the calls to
addBreakpointWithScriptUri.Here are the interesting parts of a log that logged the DAP + VM Service traffic from @mellowcello77:
My guess is that there is some issue triggered by concurrent requests to add breakpoints to each isolate. I don't know if it's a Flutter-specific issue or Dart so I thought I'd start here.
I'll try again to see if I can get a repro now I have a slightly better understanding of the issue.
(@bkonyi any ideas here?)
The text was updated successfully, but these errors were encountered: