Can I pass an Object as Dart_Handle for callback context

[kenfred]

I've used the code in samples/ffi/async code to help me accomplish callbacks from native code to dart. However, these examples call to static functions in dart. I would like the callback to take a handle to a dart class instance so the static function can use it as context to call a class method.

I've seen some similar things in vmspecific_handle_test.dart, which leads me to believe this should be possible.

I tried something like this. However, when the callback happens, the instance argument of the static method cannot be resolved to an instance of the class.

typedef NativesCallbackFunc = Void Function(Handle);
typedef NativeFunc = Void Function(Handle, Pointer<NativeFunction<NativeCallbackFunc>>);
typedef Func = void Function(Object, Pointer<NativeFunction<NativeCallbackFunc>>);

class SomeClass {
  SomeClass() {
    _func = dylib.lookupFunction<NativeFunc , Func>('Func');
    _func(this, _staticFunc);
  }

  void _staticFunc(Object instance) {
    // This fails
    Expect.isTrue(instance is SomeClass);
  }
}

I have also tried to print the identityHashCode on both ends. In the static method, the hash is usually 0. Sometimes it has a value, but does not equal the hash code of the class instance.

What is the expected way to pass an instance across the FFI boundary so native code can hand it back to Dart to reference a class instance?

[dcharkes]

Could you please also provide the relevant C code, does it immediately do a callback? That should work. If it doesn't, that sounds like a bug.

[kenfred]

In my latest test, the c code doesn't callback immediately but gets dispatched to a different thread and then follows the delegating "work" concept in your asynch example. Unfortunately, it wouldn't be practical to paste all of that code.

I have tried the direct callback as well, and it failed. However, I will need to try it again with that simple case to say so confidently.

[dcharkes]

If it is on a different thread, it needs to use the native ports mechanism. How are you passing the object through the native port message?

[kenfred]

I just tried it with the simple, direct callback and it works. So indeed, I must be corrupting the handle in some way.

You can see a diagram of what I'm doing here. It is the last diagram, on the bottom.

Here is some basic psuedo-code on what I do:

void Func(Dart_Handle instance, Callback callback_ptr) {
  // There is a singleton of MyClass and I'm passing the handle on to it
  MyClass::GetInstance()->Func(instance, callback_ptr);
}

void MyClass::Func(Dart_Handle instance, Callback callback_ptr) {
  // Dispatch to a different thread.
  // Here I must capture by value because these local variables will go out of scope right after I dispatch and this method returns.
  _dispatcher.Dispatch([instance, callback_ptr]() {
    // "DartCallbackExecutor" is a faithful implementation of your sample code. It takes this lamba, wraps it in an opaque Work type, and posts it via
    // Dart_PostCObject_DL. The ReceivePort in dart delegates that call to a different C function on the main thread, which runs the lambda.
    // Note, here I can capture the variables by reference because "ExecuteBlocking" blocks waiting for all that to happen, so
    // the local variables will not go out of scope.
    DartCallbackExecutor::GetInstance()->ExecuteBlocking([&instance, &callback]() {
      callback(instance);
    }
  }
}

During the course of this, it seems my Dart_Handle is invalidated. Does any of this give you a clue on where that happens?

Thanks for your help!

[dcharkes]

Side note: Why not pass the callback closure as Dart_Handle as well? rather than a Pointer? Then you can unwrap both objects in the message handler and invoke the closure. (Or you're already doing that and Callback is typedeffed to Dart_Handle.)

Dart_Handles live on the stack in FFI calls, so they get destroyed when returning from the first FFI call. If you want to pass a handle to another thread, you should make a Dart_PersistentHandle, which will live until deleted (so be sure to delete it after sending the message from the other thread).

[kenfred]

I didn't realize until recently that I could represent the closure with a Dart_Handle. I can see that, if I'm already passing around a Dart_Handle, I can get rid of the pointer argument. Good suggestion. However, I'm not sure what dart api I should use to invoke a closure from a Dart_Handle.

Can you point me to an example of converting a Dart_Handle to a Dart_PersistentHandle and then deleting it?

[dcharkes]

https://github.com/dart-lang/sdk/blob/b514b74dee88b84b9e2bd82442d80009fb14e1c7/runtime/bin/ffi_test/ffi_test_functions_vmspecific.cc#L1088-L1112

////////////////////////////////////////////////////////////////////////////////
// Example for doing closure callbacks with help of `dart_api.h`.
//
// sample_ffi_functions_callbacks_closures.dart

void (*callback_)(Dart_Handle);
Dart_PersistentHandle closure_to_callback_;

DART_EXPORT void RegisterClosureCallbackFP(void (*callback)(Dart_Handle)) {
  callback_ = callback;
}

DART_EXPORT void RegisterClosureCallback(Dart_Handle h) {
  closure_to_callback_ = Dart_NewPersistentHandle_DL(h);
}

DART_EXPORT void InvokeClosureCallback() {
  Dart_Handle closure_handle =
      Dart_HandleFromPersistent_DL(closure_to_callback_);
  callback_(closure_handle);
}

DART_EXPORT void ReleaseClosureCallback() {
  Dart_DeletePersistentHandle_DL(closure_to_callback_);
}

Corresponding Dart file: samples/ffi/sample_ffi_functions_callbacks_closures.dart

It is using an immediate callback rather than native ports, but it illustrates how to save a persistent handle.

[kenfred]

Great! I will try this and report back and close the issue. Thank you!

[kenfred]

This is now working based on advice from @dcharkes. Thanks!

One thing I found is that Dart_DeletePersistentHandle_DL must be called from the main thread or an exception will happen. (Presumably, this is true for the other persistent handle api funcs as well). I found this to be counterintuitive because the whole reason I needed the persistent version of the handle was to pass a handle to another thread. It was unexpected that I would then have to delegate back to the main thread to dispose of the handle.

[dcharkes]

One thing I found is that Dart_DeletePersistentHandle_DL must be called from the main thread or an exception will happen.

/**
 * Deallocates a persistent handle.
 *
 * Requires there to be a current isolate group.
 */
DART_EXPORT void Dart_DeletePersistentHandle(Dart_PersistentHandle object);

It doesn't have to be the main thread, but it needs to have an isolate group.

You can check if the current thread has one with:

/**
 * Returns the current isolate group. Will return NULL if there is no
 * current isolate group.
 */
DART_EXPORT Dart_IsolateGroup Dart_CurrentIsolateGroup(void);

And you can let a thread enter an isolate group by having it enter an isolate from that group:

/**
 * Enters an isolate. After calling this function,
 * the current isolate will be set to the provided isolate.
 *
 * Requires there to be no current isolate. Multiple threads may not be in
 * the same isolate at once.
 */
DART_EXPORT void Dart_EnterIsolate(Dart_Isolate isolate);

For more info see dart_api.h.

[xiaoxiaowesley]

One thing I found is that Dart_DeletePersistentHandle_DL must be called from the main thread or an exception will happen.

/**
 * Deallocates a persistent handle.
 *
 * Requires there to be a current isolate group.
 */
DART_EXPORT void Dart_DeletePersistentHandle(Dart_PersistentHandle object);

It doesn't have to be the main thread, but it needs to have an isolate group.

You can check if the current thread has one with:

/**
 * Returns the current isolate group. Will return NULL if there is no
 * current isolate group.
 */
DART_EXPORT Dart_IsolateGroup Dart_CurrentIsolateGroup(void);

And you can let a thread enter an isolate group by having it enter an isolate from that group:

/**
 * Enters an isolate. After calling this function,
 * the current isolate will be set to the provided isolate.
 *
 * Requires there to be no current isolate. Multiple threads may not be in
 * the same isolate at once.
 */
DART_EXPORT void Dart_EnterIsolate(Dart_Isolate isolate);

For more info see dart_api.h.

By referring dart_api.h in independent C so/dll/dyllib project like this below. We have to link dylib/so library of dart/flutter in case the link error Undefined symbols for architecture xxxxx. Does any solution fix it? Thank you!

#include <dart_api.h>
Dart_IsolateGroup currentIsolateGroup = Dart_CurrentIsolateGroup();