Receiving data from an isolate locks the main thread for a very long time

[Hixie]

The following little flutter program creates an isolate, and times how long the main thread locks while receiving a 100MB of zero bytes from an isolate. It does this by having a timer run continuously on the main thread, timing how long since it last ran, and printing any time that it took more than 5ms between invocations ("since last checkin"). If the main thread is never locked, this should never print anything. At the same time, in a Future-mediated loop on the main thread, it sends a single byte to the isolate, and then receives a 100MB ByteData buffer of zeros in response, and prints the total round-trip time.

The isolate merely runs a Future-mediated loop that waits for a message, then sends a 100MB buffer of zero bytes back.

For flutter, what matters is primarily that the total overhead of receiving something from a thread is small (small single-digits of milliseconds at most, ideally microseconds). Total round-trip time is only of academic interest so long as it's not measured in minutes.

import 'dart:async';
import 'dart:isolate';
import 'dart:typed_data';

// true = time the in-thread overhead for sending a big message out of the main thread                                                  
// false = time the in-thread overhead for receiving a big message into the main thread                                                 
const bool benchmarkSend = false;

const int toIsolateSize = benchmarkSend ? 100 * 1024 * 1024 : 1;
const int fromIsolateSize = benchmarkSend ? 1 : 100 * 1024 * 1024;

Future<Null> main() async {
  Timer.run(idleTimer);
  ReceivePort port = new ReceivePort();
  StreamIterator<dynamic> inbox = new StreamIterator<dynamic>(port);
  Isolate.spawn(isolateMain, port.sendPort);
  await inbox.moveNext();
  SendPort outbox = inbox.current;
  Stopwatch workWatch = new Stopwatch();
  ByteData data = new ByteData(toIsolateSize);
  while (true) {
    print('sending...');
    workWatch.start();
    outbox.send(data);
    await inbox.moveNext();
    workWatch.stop();
    int time = workWatch.elapsedMilliseconds;
    print('${time}ms for round-trip');
    workWatch.reset();
  }
}
 
Future<Null> isolateMain(SendPort outbox) async {
  ReceivePort port = new ReceivePort();
  StreamIterator<dynamic> inbox = new StreamIterator<dynamic>(port);
  outbox.send(port.sendPort);
  ByteData data = new ByteData(fromIsolateSize);
  while (true) {
    await inbox.moveNext();
    outbox.send(data);
  }
}

Stopwatch idleWatch = new Stopwatch();

void idleTimer() {
  idleWatch.stop();
  int time = idleWatch.elapsedMilliseconds;
  if (time > 5)
    print('${time}ms since last checkin');
  idleWatch.reset();
  idleWatch.start();
  Timer.run(idleTimer);
}

Here is some representative output for this script running on a Pixel XL 2. The total overhead for receiving 100MB from another isolate appears to be in the triple-digit* millisecond range, which means that receiving 100MB from another isolate guarantees that the application will miss many frames, animations will stutter noticeably, etc.

I/flutter ( 3647): sending...
I/flutter ( 3647): 126ms since last checkin
I/flutter ( 3647): 145ms for round-trip
I/flutter ( 3647): sending...
I/flutter ( 3647): 125ms since last checkin
I/flutter ( 3647): 146ms for round-trip
I/flutter ( 3647): sending...
I/flutter ( 3647): 126ms since last checkin
I/flutter ( 3647): 147ms for round-trip
I/flutter ( 3647): sending...
I/flutter ( 3647): 134ms since last checkin
I/flutter ( 3647): 220ms for round-trip
I/flutter ( 3647): sending...
I/flutter ( 3647): 129ms since last checkin
I/flutter ( 3647): 148ms for round-trip
I/flutter ( 3647): sending...
I/flutter ( 3647): 126ms since last checkin
I/flutter ( 3647): 145ms for round-trip
I/flutter ( 3647): sending...
I/flutter ( 3647): 123ms since last checkin
I/flutter ( 3647): 145ms for round-trip
I/flutter ( 3647): sending...

Receiving pure byte data should be something we can implement with near-zero overhead on the receiving thread. It can be placed in memory then the memory handed off without the main thread being particularly involved. When receiving data structures (not benchmarked here), I could see this being more complicated. That is also something we will need to make fast, as it will be common for people to want to create Widgets off the main thread then hand them over efficiently.

See also #31960, which is a less serious problem with sending data taking double-digit milliseconds.

[a-siva]

cc @rmacnak-google