ffigen is unable to locate system headers

[Michael-F-Bryan]

My understanding is that ffigen should use the same system include path as clang, but when I run ffigen on Linux it is unable to find several system header files (stdarg.h, stdbool.h, etc.) while clang and gcc are able to locate them and compile code perfectly fine.

This was originally reported as fzyzcjy/flutter_rust_bridge#108, but after some investigation we found the underlying bug was in ffigen. This may also be related to #345.

Steps to reproduce:

$ dart --version
Dart SDK version: 2.14.4 (stable) (Wed Oct 13 11:11:32 2021 +0200) on "linux_x64"
$ dart pub global list
ffigen 4.1.1
$ clang --version
clang version 12.0.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ mkdir /tmp/repro && cd "$_"

$ cat << EOF > header.h
#include <stdint.h>
#include <stdbool.h>

bool less_than(int32_t a, int32_t b);
EOF

$ clang header.h  # clang can typecheck the header just fine
$ echo $?
0

$ clang header.h -v  # the folders clang searches for #include <...>
...
#include <...> search starts here:
 /usr/local/include
 /usr/lib/clang/12.0.1/include
 /usr/include
End of search list.

$ cat << EOF > config.yml
name: LessThan
description: Check if one number is less than the other
output: "generated_bindings.dart"
headers:
  entry-points:
    - "header.h"
  include-directives:
    - "header.h"
EOF

$ dart pub global run ffigen --config config.yml
Running in Directory: '/tmp/repro'
Input Headers: [header.h]
[SEVERE] : Header header.h: Total errors/warnings: 1.
[SEVERE] :     header.h:2:10: fatal error: 'stdbool.h' file not found [Lexical or Preprocessor Issue]
Finished, Bindings generated in /tmp/repro/generated_bindings.dart

$ echo $?  # ffigen completed successfully, apparently 🤷
0

# We need to pass extra compiler options to find clang's include files
$ dart pub global run ffigen --config config.yml --compiler-opts '-I/usr/lib/clang/12.0.1/include/'
Running in Directory: '/tmp/repro'
Input Headers: [header.h]
Finished, Bindings generated in /tmp/repro/generated_bindings.dart

Interestingly, if I run ffigen from / (requires replacing the output field with its absolute path), the header files are located correctly.

$ cd /
$ grep output /tmp/repro/config.yml
output: "/tmp/repro/generated_bindings.dart"
$ dart pub global run ffigen --config /tmp/repro/config.yml
Running in Directory: '/'
Input Headers: []
Finished, Bindings generated in /tmp/repro/generated_bindings.dart

The generated bindings without extra compiler options:

// AUTO GENERATED FILE, DO NOT EDIT.
//
// Generated by `package:ffigen`.
import 'dart:ffi' as ffi;

/// Check if one number is less than the other
class LessThan {
  /// Holds the symbol lookup function.
  final ffi.Pointer<T> Function<T extends ffi.NativeType>(String symbolName)
      _lookup;

  /// The symbols are looked up in [dynamicLibrary].
  LessThan(ffi.DynamicLibrary dynamicLibrary) : _lookup = dynamicLibrary.lookup;

  /// The symbols are looked up with [lookup].
  LessThan.fromLookup(
      ffi.Pointer<T> Function<T extends ffi.NativeType>(String symbolName)
          lookup)
      : _lookup = lookup;

  int less_than(
    int a,
    int b,
  ) {
    return _less_than(
      a,
      b,
    );
  }

  late final _less_thanPtr =
      _lookup<ffi.NativeFunction<ffi.Int32 Function(ffi.Int32, ffi.Int32)>>(
          'less_than');
  late final _less_than = _less_thanPtr.asFunction<int Function(int, int)>();
}

The code generated *with* extra compiler options:

// AUTO GENERATED FILE, DO NOT EDIT.
//
// Generated by `package:ffigen`.
import 'dart:ffi' as ffi;

/// Check if one number is less than the other
class LessThan {
  /// Holds the symbol lookup function.
  final ffi.Pointer<T> Function<T extends ffi.NativeType>(String symbolName)
      _lookup;

  /// The symbols are looked up in [dynamicLibrary].
  LessThan(ffi.DynamicLibrary dynamicLibrary) : _lookup = dynamicLibrary.lookup;

  /// The symbols are looked up with [lookup].
  LessThan.fromLookup(
      ffi.Pointer<T> Function<T extends ffi.NativeType>(String symbolName)
          lookup)
      : _lookup = lookup;

  bool less_than(
    int a,
    int b,
  ) {
    return _less_than(
          a,
          b,
        ) !=
        0;
  }

  late final _less_thanPtr =
      _lookup<ffi.NativeFunction<ffi.Uint8 Function(ffi.Int32, ffi.Int32)>>(
          'less_than');
  late final _less_than = _less_thanPtr.asFunction<int Function(int, int)>();
}

As a side note, whenever this happens ffigen falls back to using int for any types it is unable to locate instead of erroring out (i.e. returning a non-zero exit code), meaning the only way for a build tool to detect these errors is by doing a hacky string.contains() on the output.

Instead of falling back to int, ffigen should omit a function entirely when it contains a type that can't be resolved. Unless the caller gets lucky and the function actually does return something that is compatible with ffi.Int32, any attempt to use the generated wrapper is going to result in broken code or UB.

[mannprerak2]

Hi @Michael-F-Bryan

I am not really not sure as to why ffigen is not able to detect the system headers, while clang can. Since ffigen utilises LLVM itself. I am guessing that /usr/lib/clang/12.0.1/include is not really part of standard C include path but something which clang uses internally.

You can just use compiler-opts to supply include path or you can also set the environment variable CPATH which also works.

About the int being generated due to missing headers, this happens because LLVM tries to correct the bindings incase of any errors and this behaviour can't be changed. We can either terminate or let LLVM do a best effort in fixing errors and generate code.

The best way to detect if the bindings are not correct as of now, would be to check for SEVERE errors emitted by ffigen. You can also set the log level for ffigen to SEVERE to only get those logs to be emitted.

We do currently use non zero exit code but thats only when ffigen is not able to generate anything. But I guess it would be a good to have a non zero exit code incase ffigen emitted any SEVERE errors.

[Michael-F-Bryan]

Thanks @mannprerak2 that makes sense.

I'm not exactly sure how to proceed here, because the required --compiler-opts will vary depending on the OS and distribution. For example, fzyzcjy/flutter_rust_bridge's CI has jobs that run on Ubuntu without any issues, while clang on Arch Linux must install headers in different places.

This feels a bit like dart-archive/ffigen#193 where ffigen needs to go looking for the correct paths to include, although you might be able to generalise this by asking clang for its system include path directly instead of checking hard-coded locations.

[sisou]

Since the env variable CPATH also works, I have added this to my .zshrc/.bashrc:

export CPATH="$(clang -v 2>&1 | grep "Selected GCC installation" | rev | cut -d' ' -f1 | rev)/include"

Now I don't have to run flutter_rust_bridge_codegen from / anymore.

[GZGavinZhao]

This is what I usually use on Linux to find missing stuffs:

1. If it's something like #include <atomic> without the .h, it's usually in /usr/include/c++/<version>. Through my experience, 11 seems to be the most common.
2. If it's something with the .h, try /usr/lib64/clang/<version>/include.
3. Last resort:

find /usr/include -type f -name "<missing-header>"
find /usr/lib64 -type f -name "<missing-header>"

For example, for #include <atomic> and #include <stdalign.h> would be:

find /usr/include -type f -name "atomic"
find /usr/lib64 -type f -name "stdalign.h"

[timleg002]

I had the same issue on Windows, and I fixed this by running ffigen in a x64 Native Tools Command Prompt for VS 2019 (2) command prompt. (It didn't work with VS 2022, probably something to do with installed VS addons/packages).

[dcharkes]

I had the same issue on Windows, and I fixed this by running ffigen in a x64 Native Tools Command Prompt for VS 2019 (2) command prompt. (It didn't work with VS 2022, probably something to do with installed VS addons/packages).

I've had similar issues developing https://github.com/dart-lang/native/tree/main/pkgs/c_compiler. The requirement there is that it's all Process.run invocations. Instead of using the command prompt, it invokes the vcvars.bat script that sets the correct paths:

native/pkgs/c_compiler/lib/src/cbuilder/run_cbuilder.dart

Lines 164 to 167 in 2de8813

	Future<void> runCl({required ToolInstance compiler}) async {
	final vcvars = (await _resolver.toolchainEnvironmentScript(compiler))!;
	final vcvarsArgs = _resolver.toolchainEnvironmentScriptArguments();
	final environment = await envFromBat(vcvars, arguments: vcvarsArgs ?? []);

Even when using clang, you need the system headers from a visual studio installation (clang doesn't have system headers for Windows).

When pkg:c_compiler gets more mature, FFIgen should depend on it to so solve this issue.

Until then using @timleg002's solution is a good workaround.