[ffi] Helpers for interacting with C data

[sjindel-google]

There are a number of helpful facilities which can be built on top of the FFI foundation and should probably be included in dart:ffi, for example:

• CString: wrapper for strings allocated in C
• CArray<T>: wrapper for arrays allocated in C, compatible with List, Iterable, TypedData, etc. types

Suggestions for others are welcome!

[Kleak]

Does external contribution welcome ?

[sjindel-google]

In general, yes, but at this stage the FFI interface is very unstable, so I think it would be premature to build a library on top of it. However, feature requests are still welcome since they will inform our API design.

[dcharkes]

package:ffi (https://github.com/dart-lang/ffi) contains helpers for null-terminated Utf8 and Utf16 Strings.

asTypedList extension members on Pointer expose native arrays of integers, floats, doubles, and pointers as TypedData.

We could expose an iterator on Struct .refs:

extension StructIterable<T extends Struct> on Pointer<T> {
  Iterable<T> asIterable(int length) => _StructIterable(this, length);
}

class _StructIterable<T extends Struct> extends IterableBase<T>
    implements Iterable<T> {
  final Iterator<T> iterator;

  _StructIterable(Pointer<T> pointer, int length)
      : iterator = _StructIterator(pointer, length);
}

class _StructIterator<T extends Struct> implements Iterator<T> {
  final Pointer<T> pointer;
  int i = -1;
  int length;

  _StructIterator(this.pointer, this.length);

  bool moveNext() {
    if (i + 1 >= length) {
      return false;
    }
    i++;
    return true;
  }

  T get current => pointer[i];
}

Since .refs are not writeable we can't expose it as a list. Maybe we should expose it as a UnmodifiableListView rather than an Iterable though.

Any thoughts @mkustermann @lrhn?

[mkustermann]

Tangential: We prefer to keep the dart:ffi core library as small (and stable) as possible, so we should probably move this issue to https://github.com/dart-lang/ffi

Small detail: The example above uses one iterator per iterable, which is not quite correct I believe, because it must be possible to iterate over an iterable multiple times, e.g.

final Iterable iterable = foo.asIterable(10);
for (final value in iterable) print(value);
for (final value in iterable) print(value);

In the example the second loop would re-use the same iterator as the first loop and probably run 0 times.

Otherwise I think it's reasonable. Though one has to be careful, as always, to manage the memory manually.

[dcharkes]

Yes, I was thinking package:ffi as well.

[dcharkes]

As discussed with @lrhn offline: The interface should be a List<T> and could use UnmodifiableListBase to throw on writes, that's the Dart-way of exposing random access, ordered, but unmodifiable collections.

[devxpy]

Hey, I was just wondering how StructPointer can be initialized.

With the earlier API, I just used:

var ptr = ffi.Pointer<Struct>.allocate();
ptr.store(struct);

But I can't find a way to do this with the new extensions interface.

Considering how commonplace struct * is, it'd be a pity if this wasn't supported.

[dcharkes]

@devxpy We currently do not support storing a whole struct by value. struct in ptr.store(struct) seems to indicate that would have a struct by value.

Assuming that is a Pointer to a struct, in order to allocate a pointer you use allocate from this package, just pin it to the latest dev version. (We'll release a new version when Dart 2.6 is released.)

In this package the signature would be

Pointer<Struct> struct; // from somewhere
final ptr = allocate<Pointer<Struct>>();
ptr.value = struct;

[devxpy]

Sorry for not filling in these details earlier.

struct is a struct by value, not a pointer to a struct.

---

To elaborate, this is what I wanted -

class FPDF_LIBRARY_CONFIG extends Struct {
    ...
}

FPDF_LIBRARY_CONFIG config = FPDF_LIBRARY_CONFIG();
Pointer<FPDF_LIBRARY_CONFIG> ptr = allocate<FPDF_LIBRARY_CONFIG>();
ptr.value = config;

But that results in the following error -

error: The setter 'value' isn't defined for the class 'Pointer<FPDF_LIBRARY_CONFIG>'. (undefined_setter at [pdfium] lib/src/functions.dart:34)

---

I want this because the underlying library expects FPDF_LIBRARY_CONFIG* to be passed as an argument to FPDF_InitLibraryWithConfig();

// fpdfview.h
FPDF_EXPORT void FPDF_CALLCONV FPDF_InitLibraryWithConfig(const FPDF_LIBRARY_CONFIG* config);

---

It would be sweet ffi could perform the following -

FPDF_LIBRARY_CONFIG config = FPDF_LIBRARY_CONFIG();
Pointer<FPDF_LIBRARY_CONFIG> ptr = allocate<FPDF_LIBRARY_CONFIG>([config]);

This could also make allocating Lists easy -

Pointer<Int32> ptr = allocate<Int32>([1, 2, 3, 4, 5]);

[dcharkes]

We do not support having C-values in the Dart heap (yet). So you want to allocate the C-memory first, and then write to the individual fields. However, nothing is stopping you from writing a constructor that does exactly that.

class Coordinate extends Struct {
  @Double()
  double x;

  @Double()
  double y;

  Pointer<Coordinate> next;

  factory Coordinate.allocate(double x, double y, Pointer<Coordinate> next) {
    return allocate<Coordinate>().ref
      ..x = x
      ..y = y
      ..next = next;
  }
}

See our samples.

For your example

Pointer<FPDF_LIBRARY_CONFIG> ptr = allocate<FPDF_LIBRARY_CONFIG>();
// ptr.field1 = value1;
// ptr.field2 = value2;
// ...
// (all of the above could be put in a FPDF_LIBRARY_CONFIG.allocate constructor.)
// call FPDF_InitLibraryWithConfig with ptr.

For allocating lists you could write your own helpers:

Pointer<Int32> allocateList(List<int> values) {
  final ptr = allocate<Int32>(count: values.length);
  for (int i = 0; i < values.length; i++) {
    ptr[i] = values[i];
  }
  return ptr;
}

[dcharkes]

The sample code got moved into this package https://github.com/dart-lang/ffi/blob/master/lib/src/utf8.dart

I see that the Utf16 variant is missing some methods, but they should be similar to the Utf8 ones.

P.S. feel free to post your questions on the mailing list dart-ffi@googlegroups.com or open separate GitHub issues.

[Sunbreak]

Any progress recently?

[dcharkes]

Any progress recently?

• Utf8 is in package:ffi
• Pointers have the asTypedList extension method for easy iteration in Dart.

Further String work is tracked in dart-lang/sdk#39787. We do not have any plans for any iteration support beyond asTypedList.

Closing this issue. Feel free to request more features or send pull requests.