Feature request: Sync call to listen to a ReceivePort that releases current DartVM thread

[maks]

Given the changes that have happened over the last couple of years in bringing light-weight Isolates to Dart to make using them in larger numbers much more feasible, I would like to propose the ability for making a sync call to receive messages from a ReceivePort that importantly does not block the DartVM's current thread pool thread and instead releases the thread to service other Isolates.

This would help address a number of use cases that are currently not possible.

Firstly it would allow interpreters and VM's built in Dart, eg LuaDardo to be able to make calls to async Dart functions without itself having to have its whole codebase be polluted by the blue/red color problem that @munificent highlighted given the current async implementation in Dart.

Secondly this would help with the FFI callback situation, as it would mean that the Dart user programmer could guarantee that no DartVM thread would enter a specific Isolate until they explicitly "unblocked" a specific Isolate, so it should be safe for that Isolate to get direct FFI callbacks from non Dart threads. I think I remember @mkustermann mentioning that somewhat simliar functionality was proposed on the native side of FFI for releasing the current thread, but unfortuntately I can't seem to find which issue that comment appeared in.

Thirdly I think it would help implementations that have previously used waitFor() or would in the future want similar functionality to it get the functionality without again having to make existing or even new codebases be riddled with async both due to code complexity and the performance implications of having to "buy into" using async throughout a whole codebase that otherwise would be completely sync in nature.

I'm not familiar with all the details of how this could be done in the DartVM or even if this has already been discussed in other issues, but I thought this could serve as starting point for further discussion.

[lrhn]

Sounds something like #44804, which was suggested as a migration path for waitFor.

could guarantee that no DartVM thread would enter a specific Isolate until they explicitly "unblocked" a specific Isolate, so it should be safe for that Isolate to get direct FFI callbacks from non Dart threads.

If the isolate is blocked, no Dart code should run until it unblocks. That should block FFI callbacks to the isolate as well, unless they are calling directly using the blocking port. (Which is a possibility, the NativeCallback having a waitForCall() method which blocks until an FFI calback to it happens, then it runs that callback synchronously as if inside the waitForCall() invocation, and finally returns from waitForCall().)

Blocking on any other port would not allow native callbacks to run through an unblocked port. It's too dangerous to run Dart code that might initiate asynchronous computations when we are blocking the event loop.

[mraleph]

Yes this is effectively a duplicate of #44804 sans "releasing current thread" part.

I think the releasing part comes from the confusion around maximum number of active mutators which can enter the same isolate. The core of this limitation has nothing to do with threads, thread pools or etc. It's limitation originates from the structure of the heap and how new space is partitioned into thread local allocation buffers.

Note that you can't really release the thread back to the thread pool when you are doing a synchronous call - the thread stack is occupied by a synchronous part of the call and can't be reused. Threads can only be released back to the pool if the synchronous function they are running has completed.

What you can do is to exit the isolate (call Dart_ExitIsolate from your native code) in this case this thread is not going to count as a mutator towards the limit. This is something you can do already: we have exposed Dart_ExitIsolate through DL C API in a251281.

Effectively this means that you can already build a type of synchronous communication channel yourself by writing a bit of native code which exits isolate and blocks on a condition variable. We also want to eventually add this capability directly into FFI (see #51261).

Firstly it would allow interpreters and VM's built in Dart, eg LuaDardo to be able to make calls to async Dart functions without itself having to have its whole codebase be polluted by the blue/red color problem that @munificent highlighted given the current async implementation in Dart.

I am not sure this is connected. Interpreter which manages its own stack (rather than using Dart's stack) can already call Dart's asynchronous functions without being affected by this problem. The only problem here is sandwich stacks like "Lua -> Dart -> Lua -> Dart" where the innermost call wants to be asynchronous. But that's problematic in any language e.g. in C version of Lua you will get an error if you try to suspend a coroutine across a C function invocation.

[maks]

@lrhn thanks so much for looking at this! Yes on a quick read, my request does sound like #44804 but I'll want to take some time to ready through that issue more carefully to make sure I understand it properly.

@mraleph I wanted to follow on your comment as I think I may have missed or misunderstood something fundamental. In the case of a interpreter implemented in Dart that has its own stack, eg. afaik thats an exact description of LuaDardo that is written in completely sync Dart code, how can it call out to Dart async functions from sync code??
Its my current understanding that even a single call to a Dart async function means that the whole Interpreter ends up being riddled with async code, because I tried to do just that and I ended up with quite a mess going all the way into the interpreters for(,,) loop.

I'm only looking for Lua->Dart calls. For example something as basic as a sleep(), where inside Lua code, I can have an opcode that calls Dart's Future.delayed() blocking the Lua execution but not blocking the underlying Dart Isolate thread. If I've missed something obvious on how to do this, I'll be very happy to hear that! as it immediately solves my most pressing need here.

[mraleph]

@maks

In the case of a interpreter implemented in Dart that has its own stack, eg. afaik thats an exact description of LuaDardo that is written in completely sync Dart code, how can it call out to Dart async functions from sync code??

Interpreter with its own stack is suspendable/resumable at any point during the execution. Let me try to illustrate it with some sketch of the code:

class Interpreter {
  var callstack = <({LuaFunction function, int pc})>[];
  
  FutureOr<Object?> run() {
    var (:function, :pc) = callstack.removeLast();  // Continue from the last frame
    while (true) {
      final instr = function.body[pc++];  // Next instruction
      switch (instr.opcode) {  // Execute instruction
        case Opcode.CALL:
          final target = ...;
          if (target is LuaFunction) { 
            callstack.push((function: function, pc: pc));
            // We also need to deal with registers, copy arguments, etc.
            // but I am omitting all this stuff for brevity. 
            function = target;
            pc = 0;
            continue;
          } else if (target is DartFunction) {
            final int resultReg = ...;  // Index of the register in which result is expected
            final result = target.f.apply(...);
            // Got result of the Dart function. If it is a Future we want to auto-await it.
            if (result is Future) { 
              callstack.push((function: function, pc: pc));  // Remember where to resume.
              return result.then((v) {
                setRegister(resultReg, v);
                return run();  // Continue interpreting from the suspended frame.
              });  
            } else {
              setRegister(resultReg, result); 
              // Continue interpreting.
            }
          }
        } 
      }
    }
  }
}

Of course asynchrony leaks into the Dart caller which calls into the interpreter, but that's expected:

// Underlying execution might be asynchronous so we need to await luaState evaluate.
final result = await luaState.evaluate("return callSomething()");

Bad news though - I looked at LuaDardo code and unfortunately it is not a stackless interpreter. It uses Dart stack when doing calls. So naturally it does not have the properties I describe here and that explains why it does not implement coroutines - unfortunate/incorrect implementation choice.

[maks]

Thanks for the detailed explanation @mraleph !

I think I can see now what you mean about stackless interpreter implementation.

And yes I'd expect asynchrony to leak out into the VM's caller, that's not an issue from my point of view.

The approach you show in your code is I think fairly close to what I actually tried to do earlier in LuaDardo and of course failed. But now thanks to your explanation, I now realise why that didnt work and how I could go about trying to refactor LuaDardo's implementation to fix it, though it looks like it will require a lot of rework there.

I know it's not the Dart teams job to do CS lessons, but given the nature of Darts async/await implementation with the compiler splitting functions on the awaits and given just how many interpreter/VM implementations there are written in Dart (I personally came across half a dozen so far) it may be worthwhile to document the low-level details of async code, though @mraleph your comment above already does a pretty good job of that for future readers.

I think at this point, I'll close this issue as I know now that what essentially I'm asking for what is in #44804 BUT I think being able to free the underlying Dart thread is a very important distinction so I'll continue the discussion there.