-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dart:ffi GC finalizers #35770
Comments
As discussed with @jonasfj and @mkustermann, maybe it would be better to provide a class Pointer {
attachFinalizer(Pointer<NativeFunction<void Function(Pointer)>> finalizerPointer) {
}
} |
You might also want to include a |
We should implement this in the initial release since so many users seem to need it. |
I think we should estimate 3 weeks for this, given that we need:
|
Yeah, we really need it. |
Looking at https://dart-review.googlesource.com/c/sdk/+/123662, I don't think this solves all issues, especially with long lived structures or strings in a static method. For example, libpng's png_create_read_struct needs the version string of libpng from link-time to ensure that ABI-incompatible changes haven't occurred (https://github.com/glennrp/libpng/blob/libpng16/png.h#L922). Yes, you could also wrap the function with another C library to forward the version correctly, but then you have to make sure that Dart, the auxiliary native library, and libpng are all in sync and are ABI compatible. I'm just hoping that that CL isn't the final design for pointer finalizers. |
When will this feature be available? |
Somewhere in the next quarters, I cannot give you a precise date. |
Samples for managing native memory without finalizers. Design: go/dart-ffi-resource-lifetime Related issue: #35770 Change-Id: I2d0ac1acb65a78db9f57aea3dd5f25b4948ef6d6 Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-mac-release-simarm_x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123662 Commit-Queue: Daco Harkes <dacoharkes@google.com> Reviewed-by: Martin Kustermann <kustermann@google.com> Reviewed-by: Erik Ernst <eernst@google.com>
Required for finalizers: #35770 Change-Id: Ic512a4efd81cbd38cd836a8e8ad80464d2a3481f Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,analyzer-nnbd-linux-release-try,dart2js-nnbd-linux-x64-chrome-try,ddc-nnbd-linux-release-chrome-try,front-end-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-release-x64-try Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135906 Reviewed-by: Martin Kustermann <kustermann@google.com>
The reachability fence keeps a value alive and reachable. Required for finalizers: #35770 Design: go/dart-ffi-finalizers (See "Premature Cleanup (Single Object)".) Change-Id: I9742889f0f8d8b15bbcb5dca47f2a4231899dd59 Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136188 Reviewed-by: Vyacheslav Egorov <vegorov@google.com> Reviewed-by: Martin Kustermann <kustermann@google.com> Commit-Queue: Daco Harkes <dacoharkes@google.com>
Been over a quarter since the last update. Any news on this front? I believe Mongo, who acquired Realm, would be able to move forward with Realm support for Flutter which many of us want once this issue is resolved. |
What's the progress of that? I think this feature might be what I needed. simolus3/drift#835 (comment) |
@drriguz The status is given in the issue body itself. See the text after Update 2020-06-26. |
Supported now ? Is there an example with more details ? |
I'm already using this in |
Regarding the workaround for premature finalization: "Workaround: do an ffi call with a Dart_Handle passing that object to native, that will keep it alive." Does that apply to any call (e.g. attaching a finalizer in object's constructor), or must such a call happen in any method using the native pointer, after the use of the native pointer - to keep it alive until that point? |
It works like a reachability fence, it keeps it alive util that point. So yes, at the end of any method using the native pointer. Side note, we might end up adding finalizers to Dart in which we keep |
Hmm. Is there, by any chance, any way to do that without defining my own native method and distributing it as a (or part of an existing) shared library? (It's a little awkward to do that because of the way I get the libs for all the supported platforms... takes time to get built and releases out...) I've looked at dart_api/_dl.h for any "useful" function and // TODO windows? flutter has `flutter_windows.dll` which should hopefully work
// but is there something for plain Dart?
final DynamicLibrary _dartLib = DynamicLibrary.process();
/// Must be called after last native pointer call in a method to prevent `this`
/// from being "finalized" preliminarily (before the current method finishes).
final dart_keepalive =
_dartLib.lookupFunction<Uint8 Function(Handle), int Function(Object)>(
'Dart_IsError'); |
@vaind I think the best you can do right now is the following: @pragma('vm:never-inline')
Object reachabilityFence(Object obj) {
return obj;
} and use this as a replacement for |
Many thanks @mraleph, you saved my day! |
What does it mean @dcharkes ? Where can we find |
There's lots of things about this Finalizer API that I don't understand. If you could help me clarify some of these, I'll be grateful...
I'm sorry if these questions sound dumb, it's just that I'm not familiar with the whole Finalizer thing, and I'm trying to make sure that I don't leak memory when I'm working with native code cause I'm creating lots of them |
I don't like this design and I'm confused: do I need to change the .c code in order to call it from dart? what if the .c lib is a 3rd party library which I don't have source code ? |
You can write a C library that wraps the original library which attaches the finalizers. And yes, that is not ideal. Multiple packages are facing this issue. That's why we're investigating adding finalizers in Dart-land. (This issue, #35770, is tracking adding FFI-specific Dart finalizers. However, https://github.com/dart-lang/sdk/issues/45455 proposes to add finalizers to Dart in general.) Until one of those two issues has been resolved, your options are:
|
Yes.
I presume you meant Or
If you don't ever want to cancel it, you can ignore the return value of You should not disregard a
I presume you meant You can pass the
No problemo! P.S. These kind of questions are not really issues with Dart. So StackOverflow might be a better place for them than making this GitHub thread very long! |
This does not exist. This was a proposed API. That will likely never exist because of the pursuit of adding Dart non-FFI-specific finalizers: #45455. |
Given that C finalizers are available, and Dart finalizers are tracked in #45455, I'm going to lock this issue for now. If you have questions about how to use the finalizers in C, please post your questions on StackOverflow. If you want to chime in on the Dart finalizer design discussion, please post in #45455. If for whatever reason #45455 does not come to be, I'll unlock this discussion again. |
@dcharkes - are we ready to close this issue now? |
Yes! Available in https://api.dart.dev/stable/2.18.3/dart-ffi/NativeFinalizer-class.html. |
Update 2022:
Available in https://api.dart.dev/stable/2.18.3/dart-ffi/NativeFinalizer-class.html.
Update 2020-06-26:
Finalizers are now available in native code (not yet exposed in Dart):
Use dynamic linking (the
_DL
suffix) so that the symbols are available in Flutter. See documentation and samples on 7eac9f3 (C code, Dart code).See more documentation in native_api.h and dart_api_dl.h.
Caveat:
Dart_Handle
passing that object to native, that will keep it alive until that call (like a reachability fence). Or alternatively use the Dart calling convention to keep the object alive:==========================================================================================
This issue was originally intended to track exposing finalizers in Dart. (The above code exposes finalizers in C.)
A potential API would be.
However, exposing FFI-specific finalizers in Dart has been superseded by adding finalizers to Dart in general https://github.com/dart-lang/sdk/issues/45455.
The text was updated successfully, but these errors were encountered: