Cannot find documentation: How to use dart_api_dl.h correctly (espeically in Flutter Android and Flutter iOS)?

[fzyzcjy]

Hi thanks for the language! I wonder how to use dart_api_dl.h correctly (espeically in Flutter Android and Flutter iOS)? I have a Flutter app with some native C/C++/Rust(but you can just consider C) code. I want to call Dart_PostCObject from the native code, such that my dart (Flutter) program can receive some data.

I have tried to search on Google and dart doc, but do not find out anything useful.

My naive guess: In my_code.c, just #include "dart_api_dl.h". Then simply call Dart_PostCObject_DL(my_port, my_data);.

However, is this the correct way? Thanks!

P.S. Of course, the Flutter app will be run on Android and iOS. Thus here are some very simple background if you are not familiar with how Dart is used in Flutter Android and iOS: For android, the native code is transformed into my_native_lib.so. In iOS, it seems to be static linked (not sure). So imho we need to consider the dynamic link and static linked cases. (I am not an expert in this, so there can be problems in what I said above.)

[a-siva]

//cc @kwalrath

[fzyzcjy]

Hi @kwalrath May I know the suggestions? Thanks

[kwalrath]

I don't know the answer, but I'd be happy to help with communicating the answer... once we have it.

[fzyzcjy]

Thank you! Imho this api is quite important, so it show be documented how people should use it.

[Dampfwalze]

I guess you have already found the include directory inside the dart sdk. There you can find dart_api_dl.h and dart_api_dl.c.
Simply add that include directory to your application and copy the dart_api_dl.c file into your projects, so that it gets build into it.

Then you have to call Dart_InitializeApiDL. Because you need data supplied in dart, you have to call it from a native function that gets called from dart and pass in NativeApi.initializeApiDLData from dart:ffi. This will load the function pointers from the dart binary, so you can call them. Make sure, 0 is returned from Dart_InitializeApiDL, otherwise it failed.

Note, that this will only load a subset of the api functions. This includes only these, that a postfixed with _DL!

I found this information in the first comment of dart_api_dl.h, but it's a shame, that this isn't really documented anywhere. Sadly Google won't help you much with this!

But you can find an example implementation:

sdk/samples/ffi/async/sample_async_callback.dart

Lines 31 to 33 in ec829af

	final initializeApi = dl.lookupFunction<IntPtr Function(Pointer<Void>),
	int Function(Pointer<Void>)>("InitDartApiDL");
	Expect.isTrue(initializeApi(NativeApi.initializeApiDLData) == 0);

sdk/runtime/bin/ffi_test/ffi_test_functions_vmspecific.cc

Lines 411 to 413 in ec829af

	DART_EXPORT intptr_t InitDartApiDL(void* data) {
	return Dart_InitializeApiDL(data);
	}

[Dampfwalze]

And while we are at it, why does it only load such a small subset of functions? I would like to be able, to directly build dart objects from C, which should be possible, with the whole set of functions! I think I will open a new Issue for this.

Edit

Issue can be found here: #48497

[rogurotus]

@dcharkes, Can you help with this? Do I understand correctly that we can use functions from dart_api_dl.h on the c/c++/rust side to manipulate Dart_Hadle? Is it intended for this and is not deprecated?

[dcharkes]

Do I understand correctly that we can use functions from dart_api_dl.h on the c/c++/rust side to manipulate Dart_Handle?

Yes, but only a subset of them, rather than using the Dart API, you can do callbacks back into Dart. Right now, there's a limitation that you have to be on the Dart thread to do those callbacks. Callbacks from other threads are being worked on #37022.

So you cannot manipulate a handle such as asking the field of an object with the Dart API.

The subset it runtime/include/dart_api_dl.h

Is it intended for this and is not deprecated?

Our end goal is to use callbacks instead of the Dart API. The Dart API is reflective and much slower than FFI. So, we want to get rid of this API if possible.

However, some parts will stay.

• The ability to turn an Object in Dart to a Dart_Handle in C through FFI is required if you want to have some kind of mapping from C/C++/Rust/Go to Dart objects in a GCable way. (You could do the same with a map from integers to objects in Dart, and keep the integer in the native language, but then you'd never know when that object can be removed from the map.)
• The ability to make Finalizable/WeakPersistent handles. If your mapping requires holding on to Dart objects weakly, you need this.
• Errors will likely stay as long as the aboves stay.
• The ports part of the API stays for the foreseeable future, it's in use by people. But with async callbacks those use cases can likely be migrated.

Wait, that is all the parts of runtime/include/dart_api_dl.h. 😄

Please let me know if you have more questions.

[rogurotus]

Do I understand correctly that we can use functions from dart_api_dl.h on the c/c++/rust side to manipulate Dart_Handle?

Yes, but only a subset of them, rather than using the Dart API, you can do callbacks back into Dart. Right now, there's a limitation that you have to be on the Dart thread to do those callbacks. Callbacks from other threads are being worked on #37022.

So you cannot manipulate a handle such as asking the field of an object with the Dart API.

The subset it runtime/include/dart_api_dl.h

Is it intended for this and is not deprecated?

Our end goal is to use callbacks instead of the Dart API. The Dart API is reflective and much slower than FFI. So, we want to get rid of this API if possible.

However, some parts will stay.

• The ability to turn an Object in Dart to a Dart_Handle in C through FFI is required if you want to have some kind of mapping from C/C++/Rust/Go to Dart objects in a GCable way. (You could do the same with a map from integers to objects in Dart, and keep the integer in the native language, but then you'd never know when that object can be removed from the map.)
• The ability to make Finalizable/WeakPersistent handles. If your mapping requires holding on to Dart objects weakly, you need this.
• Errors will likely stay as long as the aboves stay.
• The ports part of the API stays for the foreseeable future, it's in use by people. But with async callbacks those use cases can likely be migrated.

Wait, that is all the parts of runtime/include/dart_api_dl.h. smile

Please let me know if you have more questions.

Thanks for your reply. Do I understand correctly that we should not rely on

• Dart_NewPersistentHandle
• Dart_DeletePersistentHandle
• Dart_HandleFromPersistent

but only on WeakPersistent and GC.

[dcharkes]

Thanks for your reply. Do I understand correctly that we should not rely on

• Dart_NewPersistentHandle
• Dart_DeletePersistentHandle
• Dart_HandleFromPersistent

but only on WeakPersistent and GC.

You can also rely on PersistentHandle. (The normal Dart_Handles live on the stack/in the Dart_Scope.)

For the use case with strong references from C/C++/Rust/Go you need persistent handles.

(Sorry for missing those in my previous post.)

[rogurotus]

@dcharkes, сan you help using dart_api_dl.c?
Let's say we integrate dart_api_dl.c into our library. After that, this library will be integrated somewhere where dart_api_dl.c was also integrated, will this lead to any problems if so, is there a way to resolve the conflict?

[dcharkes]

After that, this library will be integrated somewhere where dart_api_dl.c was also integrated, will this lead to any problems?

How will you integrate, by source or compiling these to dynamic libraries?

It doesn't matter if dart_api_dl.c is compiled and linked into a final native library multiple times. The one part of your code will use the first copy (and initialize the first copy) and the other part of your code the second copy (and initialize the second copy).

Of course it's fine too to make sure all your code uses a single copy.