Don't run compiled binaries while cross-compiling

[whitequark]

Trying to run binaries while cross-compiling is an extremely annoying habit. To a degree it can be mitigated by manually shimming it, but for opam-cross-windows I need two shims, for opam-cross-ios I need four, and for opam-cross-android... uh... the amount is theoretically bounded, but practically the limit is too high to be manageable even in principle. Not to mention adding shims being dull and prone to produce errors that are extremely hard to debug, and in some cases just very hard to obtain; the immediate motivation for this was the fact that I have no damn clue how to run a regular binary on an iOS simulator (is it possible at all?) nor do I have any desire to figure that out.

So, I offer an alternative solution that does not need to run any binaries. It works with any sane C compiler, specifically any compiler that places a character array contiguously, verbatim, and in ASCII somewhere in the object file. OCaml doesn't work on non-ASCII systems and the design of contemporary linkers (that are dumb code combiners) combined with the fact that OCaml does not use linker LTO plugins mandate that these assumptions be true.

Specifically, what my solution does is it takes a compile-time constant, converts it to a string guarded by a prefix and a suffix, and assigns it to a global externally visible symbol. Then, it reads the object file and extracts the decimal between the guards. This is guaranteed by C99 to work for any compile-time constant, and FFI defines, sizeof and alignof count.

As a bonus, CC is not used anymore, but rather the C compiler is invoked through ocamlfind ocamlc, which respects the OCAMLFIND_TOOLCHAIN environment variable.

[whitequark]

Note: I've not yet ported the primitive detail extraction code as it does more work in C. I'd like to get feedback on existing changes though, and especially on how to deduplicate the C compiler interface code between configure/ and libffi-abigen/. (I've no idea how to do it in your build system.)

[whitequark]

Updated to encompass the C primitive queries as well. All tests pass on Linux.

[whitequark]

Updated again to add the $(HOSTOCAMLFIND) variable for almost completely seamless cross-compilation support. asneeded.config and libffi.config need to be provided explicitly, but I think that's OK.

[coveralls]

Coverage decreased (-0.03%) to 87.909% when pulling 1b608b7 on whitequark:dont-run-binaries into b40107d on ocamllabs:master.

[yallop]

Thanks! I'd like to improve support for cross-compilation, so I appreciate this. I'll aim to take a proper look in the next few days.

[whitequark]

I've now verified that I can cross-compile ctypes to Windows, Android and 32/64-bit iOS/iOS simulator using an opam file like this:

opam-version: "1.2"
maintainer: "whitequark@whitequark.org"
patches: ["patches/cross-compilation.patch" "patches/no-shlib.patch"]
substs: ["libffi.config"]
build: [
  [make "OCAMLFIND=ocamlfind -toolchain ios" "HOSTOCAMLFIND=ocamlfind"]
]
install: [
  [make "OCAMLFIND=ocamlfind -toolchain ios" "HOSTOCAMLFIND=ocamlfind"
        "install"]
]
remove: [
  ["ocamlfind" "-toolchain" "ios" "remove" "ctypes"]
]
depends: [
  "ocaml-ios"
  "libffi-sys-ios"
  "ocamlfind" {build}
]

[yallop]

Could you please make the configure step a bit quieter? I'd like to avoid displaying this sort of thing:

/tmp/ctypes_libffi_confige0bf36.c:23:7: error: ‘FFI_EABI’ undeclared here (not in a function)
   D9((FFI_EABI)),
       ^

which might alarm people compiling the library.

[yallop]

I think that nsplit could be implemented like this:

Str.(split (regexp_string sep)) str

(The behaviour is slightly different when the separator appears at the beginning or the end of the string, but that probably doesn't matter.)

[whitequark]

I think you'll agree with me that Str is badly designed and should be deprecated, if so, why use it if I already wrote code without it?

[yallop]

Writing the code is a sunk cost, but maintaining it isn't. I'd prefer to maintain a one-liner.

And yes, Str is badly designed. Its only real virtue is that it's in the standard library.

[whitequark]

Could you please make the configure step a bit quieter?

I initially did, but this caused issues when it failed on CI and I had no idea why. How about writing that to a temporary file in the Makefile and then displaying it if the build fails?

[yallop]

Yes, that sounds better.

[whitequark]

All fixed.

[coveralls]

Coverage decreased (-0.03%) to 87.909% when pulling cb8291d on whitequark:dont-run-binaries into b40107d on ocamllabs:master.

[yallop]

Thank you!