Using dart:ui from non-main-thread Isolates crashes #10647
Comments
|
We're currently in the process of making some improvements to our engine to allow exposing |
|
Ok, please let me know when something like this would be possible! |
|
This will also help another part of our project where we currently spawn another process and communicate via HTTP as a workaround. So +1 to making this work in an isolate. |
|
@GaryQian is currently working on wiring up |
|
Running code that uses |
Hixie
changed the title
|
@chinmaygarde Any idea what the current status of this is? We will have to come to a decision on what needs to be supported here by 1.0. |
|
Launching multiple Flutter views in their own isolates when their own dedicated threads is now supported (and is the default). Having multiple isolates talk to the same view is not currently supported and is not a priority. We should probably not crash when |
|
I need this so much, especially for a heavy things like Path.combine.
|
This comment was marked as abuse.
This comment was marked as abuse.
|
Same problem |
|
Did you guys find any solution to open UI from isolate ? |
|
Issue replicable on latest beta. Although moving flutter doctor -v |
darshankawar
added
found in release: 1.22
could you give a small example , please? thank you |
|
From a Flutter application developer’s perspective, dart:ui is no different from any other general purpose dart library. For example, an application developer might use dart:math for literally infinite many use cases, some of which might need to run inside an isolate for performance reasons. The developer would be surprised and disappointed to find that after successfully compiling their Flutter application, a call to dart:math breaks when called from an isolate at runtime. This is exactly what happens with dart:ui as evidenced by the number of issues I have stumbled across that are all asking to use some component of dart:ui in an isolate. Within each of those issues, there are many additional use cases contributed by other users via comments. Most of these use cases are attempting to use dart:ui as a generic drawing or image processing tool. I gather that there are some “unsafe” components in dart:ui that should not be called from an isolate that can impact application rendering and frame scheduling. But, to reiterate, many of these use cases are looking to use the “safe” components of dart:ui (Canvas, Picture, PictureRecorder, Paint, Path, Radius, Rect, Offset, Image, Gradient, etc.), using dart:ui as a generic drawing or image manipulation tool. But then, why are these "safe" generic drawing utilities packaged with “unsafe” Flutter components? Alternatively, why are the “safe” generic drawing utilities indiscriminately restricted from isolates along with the “unsafe” Flutter components. To go back to the analogy, would it make sense for the Flutter team to restrict dart:math from isolates if there was some special Flutter code packaged with dart:math. Of course it doesn't, but it appears that that is exactly what is happening with dart:ui. |
|
Use case: My app generates and saves thousands of images using To speed things up, running multiple isolates is necessary. As mentioned above, dart:ui is not available in isolates. |
|
Reproduces on the latest versions of flutter. Updating labels. code sampleimport 'dart:isolate';
import 'package:flutter/material.dart';
void echo(SendPort sender) {
print("echo() the Isolate has been spawned!");
ReceivePort receivePort = ReceivePort();
sender.send(receivePort.sendPort);
receivePort.listen((data) {
String msg = data[0];
SendPort replyTo = data[1];
print("Isolate received: $msg");
WidgetsFlutterBinding.ensureInitialized();
replyTo.send("Greetings from echo() the Isolate!");
});
}
Future sendReceive(SendPort port, String msg) {
ReceivePort receivePort = ReceivePort();
port.send([msg, receivePort.sendPort]);
return receivePort.first;
}
void main() {
ReceivePort receivePort = ReceivePort();
Future<Isolate> remote = Isolate.spawn(echo, receivePort.sendPort);
remote.then((_) => receivePort.first).then((sendPort) {
sendReceive(sendPort, "message sent from main()").then((msg) {
print("Main received: $msg");
});
});
runApp(const MyApp());
}
class MyApp extends StatelessWidget {
const MyApp({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Material App',
theme: ThemeData.dark(),
home: const Home(),
);
}
}
class Home extends StatelessWidget {
const Home({super.key});
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: const Text('Material App Bar'),
),
body: const Center(
child: Text('some text'),
),
floatingActionButton: FloatingActionButton(
child: const Icon(Icons.add),
onPressed: () {},
),
);
}
}
logsLaunching lib/main.dart on macOS in debug mode...
Building macOS application...
--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:macOS, arch:arm64, id:00008103-000329201AD1001E }
{ platform:macOS, arch:x86_64, id:00008103-000329201AD1001E }
Debug service listening on ws://127.0.0.1:51624/HWY-7KvoUsw=/ws
Syncing files to device macOS...
flutter: echo() the Isolate has been spawned!
flutter: Isolate received: message sent from main()
[ERROR:flutter/runtime/dart_isolate.cc(1097)] Unhandled exception:
UI actions are only available on root isolate.
#0 FfiTrampoline____nativeSetNeedsReportTimings$Method$FfiNative$Ptr (dart:ffi)
#1 PlatformDispatcher.__nativeSetNeedsReportTimings (dart:ui/platform_dispatcher.dart:546:24)
#2 PlatformDispatcher._nativeSetNeedsReportTimings (dart:ui/platform_dispatcher.dart:543:52)
#3 PlatformDispatcher.onReportTimings= (dart:ui/platform_dispatcher.dart:535:29)
#4 SchedulerBinding.addTimingsCallback (package:flutter/src/scheduler/binding.dart:308:26)
#5 SchedulerBinding.initInstances (package:flutter/src/scheduler/binding.dart:240:7)
#6 ServicesBinding.initInstances (package:flutter/src/services/binding.dart:37:11)
#7 PaintingBinding.initInstances (package:flutter/src/painting/binding.dart:20:11)
#8 SemanticsBinding.initInstances (package:flutter/src/semantics/binding.dart:18:11)
#9 RendererBinding.initInstances (package:flutter/src/rendering/binding.dart:30:11)
#10 WidgetsBinding.initInstances (package:flutter/src/widgets/binding.dart:249:11)
#11 new BindingBase (package:flutter/src/foundation/binding.dart:151:5)
#12 new _WidgetsFlutterBinding&BindingBase&GestureBinding (package:flutter/src/widgets/binding.dart)
#13 new _WidgetsFlutterBinding&BindingBase&GestureBinding&SchedulerBinding (package:flutter/src/widgets/binding.dart)
#14 new _WidgetsFlutterBinding&BindingBase&GestureBinding&SchedulerBinding&ServicesBinding (package:flutter/src/widgets/binding.dart)
#15 new _WidgetsFlutterBinding&BindingBase&GestureBinding&SchedulerBinding&ServicesBinding&PaintingBinding (package:flutter/src/widgets/binding.dart)
#16 new _WidgetsFlutterBinding&BindingBase&GestureBinding&SchedulerBinding&ServicesBinding&PaintingBinding&SemanticsBinding (package:flutter/src/widgets/binding.dart)
#17 new _WidgetsFlutterBinding&BindingBase&GestureBinding&SchedulerBinding&ServicesBinding&PaintingBinding&SemanticsBinding&RendererBinding (package:flutter/src/widgets/binding.dart)
#18 new _WidgetsFlutterBinding&BindingBase&GestureBinding&SchedulerBinding&ServicesBinding&PaintingBinding&SemanticsBinding&RendererBinding&WidgetsBinding (package:flutter/src/widgets/binding.dart)
#19 new WidgetsFlutterBinding (package:flutter/src/widgets/binding.dart)
#20 WidgetsFlutterBinding.ensureInitialized (package:flutter/src/widgets/binding.dart:1260:7)
#21 echo.<anonymous closure> (package:ent/main.dart:13:27)
#22 _RootZone.runUnaryGuarded (dart:async/zone.dart:1593:10)
#23 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#24 _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#25 _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:774:19)
#26 _StreamController._add (dart:async/stream_controller.dart:648:7)
#27 _StreamController.add (dart:async/stream_controller.dart:596:5)
#28 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:189:12)
Application finished.flutter doctor -v |
danagbemava-nc
added
found in release: 3.7
|
If the Flutter team does not want to fix this issue, can they provide a workaround with a working example? |
I second that. In my point of view, there are two main usecases:
The 1st one is harder to implement, since it requires changing the engine, and is related to other limitations of the current state of threading in Dart. The 2nd one is simpler, and could be easier to implement. I imagine Dart can easily choose to allow importing files from Personally, more than a couple times I had to implement code related to rendering, either as a Dart library or to run in an Isolate, not neccessarily on Flutter. Being unable to use |
This comment was marked as duplicate.
This comment was marked as duplicate.
|
Would one way of potentially working around this be to use Lightweight flutter engines in a pure Flutter app? (ie. notAdd-to-App). Since instances of the Lightweight flutter engines have their own threads but share some resources they would be fast to start up and perhaps could be used for offscreen rendering? (Please note: I'm a Flutter newbie, so there could be gaps in my understanding, so pardon if it is a stupid question). There are so many use cases which require being able to draw to a canvas in an isolate is required to get Flutter to perform any where near native solutions. |
This comment was marked as duplicate.
This comment was marked as duplicate.
flutter-triage-bot
bot
added
team-engine
This comment was marked as duplicate.
This comment was marked as duplicate.
|
@Medit8r I hit this issue back in 2021, not sure if I commented here, but seems like this issue is not going to be fixed unless there is a change in the flutter design itself, something I doubt they will pull anytime soon, the project I started back then in flutter I had to rewrite twice to go around this issue with no luck sadly, I went to discover different technologies and as far as I know Xamarin is Xml based, not much what a flutter dev would like to move to, the technology I recommend is Jetpack compose and Compose Multiplatform, developed by JetBrains themselves, I worked with it, they have coroutines where u can pull some ui logic into another thread (if safe to call it a thread), but basically what we exactly want! which is to offload some ui code into another isolate. Kotlin is also super cool as well, so no complains so far, apart from being relatively new, back then I doubted the move from flutter, but looking into 2023, Although flutter is very promising and I like it, I simply don't regret the move I made. So I recommend it to everyone.👌 |
|
Isolates from the Flutter Isolates package works fine with calls to dart.ui on both IOS and Android. I have been using it to generate lots of images from a pool of Flutter Isolates. |
|
Just want to add my use case: I use dart:ui & Canvas to render thousands of video frames. I'd like to use multiple isolates to increase speed. |
|
I've released a new plugin. https://pub.dev/packages/dart_ui_isolate It is very similar to the Due to that, it supports FlutterEngineGroup which saves 100s of megabytes of memory. |
|
@chipweinberger Thank you for that! We dropped a project a couple of years ago because of this issue. We're planning to give it another chance in the next couple of months, so I'll be testing your plugin then. |
Can you show an example of how to use it with widgets? |
|
What I understood from @chipweinberger, ui isolates can be used from Android, iOS and OSX. Are there any plans to support the other platforms? Independently, even when using this functionality on supported platforms, I'm either holding it wrong or it's pretty clunky. Let's say I rasterize an image with dart:ui on a child ui isolate, I then have to copy the image bytes into the dart isolate to then send it over, to then decode the image which will copy the bytes back to native. Naively, I would have hope to use ImageDescriptor, i.e. just sending a pointer to the image in memory, however ImageDescriptor doesn't seem to be serializable1. With this new functionality, should it be?
Non-authoritatively: I don't think that's an intended use-case Footnotes
|
and others
Latest update (tl;dr: no current plans): #10647 (comment)
Steps to Reproduce
In a fresh codebase generated by
flutter create, I added the following functions and amended themain()function. Code taken from this example:When I comment out the
WidgetsFlutterBinding.ensureInitialized();call, everything runs smoothly. Leaving it in, the app crashes. When I run it in SkyShell with that call left in, I get a seg fault:What is happening here? Should
WidgetsFlutterBinding.ensureInitialized()be able to be called from a spawned Isolate?Flutter Doctor
The text was updated successfully, but these errors were encountered: