Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve detection of how to execute binaries on the host #10265

Merged
merged 4 commits into from
Dec 3, 2021

Conversation

andrewrk
Copy link
Member

@andrewrk andrewrk commented Dec 2, 2021

from zig-specific options to generally recognized zig build options that any project can take advantage of. See the updated usage text for more details.

Prerequisite to #10143 (cc @nuald)

Next I want to look at getExternalExecutor and consider moving it from CrossTarget to Target, or otherwise resolve those TODO comments. There is also this:

$ grep -RI canExecBinariesOf ../src/ ../lib/
../src/main.zig:        if (!builtin.target.canExecBinariesOf(target)) {
../lib/std/target.zig:    pub fn canExecBinariesOf(host_target: Target, binary_target: Target) bool {

I think this should probably be removed and zig test would unconditionally try to execute the test binary. You'd have to pass --test-no-exec to disable this. As long as the error message is nice and clear this should be OK. Reasoning for this is that we want to keep awareness of qemu, wine, etc. in the zig build system and out of the zig compiler itself.

Also the code that calls getExternalExecutor is duplicated in std/build.zig and src/test.zig and I want to resolve that.

End goal here is that in a build.zig you could declaratively use a RunStep, without putting any logic inside build.zig to disable it. The RunStep itself would potentially emit an error if the build graph required it to be run, and there were no executors available to run it.

from zig-specific options to generally recognized zig build options that
any project can take advantage of. See the updated usage text for more
details.
@andrewrk
Copy link
Member Author

andrewrk commented Dec 3, 2021

demo of 4f40d8e:

[nix-shell:~/dev/zig/build]$ uname -a
Linux ark 5.10.81 #1-NixOS SMP Sun Nov 21 12:46:37 UTC 2021 x86_64 GNU/Linux

[nix-shell:~/dev/zig/build]$ ./zig run ../test/standalone/hello_world/hello.zig -target aarch64-linux
warning: the host system (x86_64-linux.5.10.81...5.10.81-gnu.2.33) does not appear to be capable of executing binaries from the target (aarch64-linux.3.16...5.5.5-musl)
error: the following command failed to execve with 'InvalidExe':
/home/andy/.cache/zig/o/b61b3c9b78433477f6b5c343ce95631f/hello

[nix-shell:~/dev/zig/build]$ ./zig run ../test/standalone/hello_world/hello.zig -target x86_64-linux-gnu -lc
warning: the host system does not appear to be capable of executing binaries from the target because the host dynamic linker is located at '/nix/store/z56jcx3j1gfyk4sv7g8iaan0ssbdkhz1-glibc-2.33-56/lib/ld-linux-x86-64.so.2', while the target dynamic linker path is '/lib64/ld-linux-x86-64.so.2'. Consider using --dynamic-linker
error: the following command failed to execve with 'FileNotFound':
/home/andy/.cache/zig/o/f0f8bdc473cbcf7f00bd72f0450102ad/hello

[nix-shell:~/dev/zig/build]$ ./zig test ../test/behavior.zig -I ../test -target aarch64-linux 
warning: the host system (x86_64-linux.5.10.81...5.10.81-gnu.2.33) does not appear to be capable of executing binaries from the target (aarch64-linux.3.16...5.5.5-musl). Consider using --test-no-exec or --test-cmd
error: the following command failed with 'InvalidExe':
../test/zig-cache/o/b70989a95e10772fc12652c7f94a5cf5/test /home/andy/dev/zig/build/zig

[nix-shell:~/dev/zig/build]$ ./zig test ../test/behavior.zig -I ../test -target x86_64-linux-gnu -lc
warning: the host system does not appear to be capable of executing binaries from the target because the host dynamic linker is located at '/nix/store/z56jcx3j1gfyk4sv7g8iaan0ssbdkhz1-glibc-2.33-56/lib/ld-linux-x86-64.so.2', while the target dynamic linker path is '/lib64/ld-linux-x86-64.so.2'. Consider using --dynamic-linker, --test-no-exec, or --test-cmd
error: the following command failed with 'FileNotFound':
../test/zig-cache/o/795162652b84a90baa06fbb62db5eb68/test /home/andy/dev/zig/build/zig


Previously when using `zig run` or `zig test`, zig would try to guess
whether the host system was capable of running the target binaries. Now,
it will always try. If it fails, then Zig emits a helpful warning to
explain the probable cause.
@andrewrk
Copy link
Member Author

andrewrk commented Dec 3, 2021

I think the next step is going to be moving getExternalExecutor from CrossTarget to std.zig.system.Executor.detect and changing the fn signature from:

/// Note that even a `CrossTarget` which returns `false` for `isNative` could still be natively executed.
/// For example `-target arm-native` running on an aarch64 host.
pub fn getExternalExecutor(self: CrossTarget) Executor {

to

const NativeTargetInfo = std.zig.system.NativeTargetInfo;
pub fn detect(host: NativeTargetInfo, target: NativeTargetInfo) Executor {

This will make it correctly attempt to execute, e.g. 32-bit arm binaries on 64-bit arm hosts. It will also make it aware of dynamic linker incompatibilities.

`getExternalExecutor` is moved from `std.zig.CrossTarget` to
`std.zig.system.NativeTargetInfo.getExternalExecutor`.

The function also now communicates a bit more information about *why*
the host is unable to execute a binary. The CLI is updated to report
this information in a useful manner.

`getExternalExecutor` is also improved to detect such patterns as:
 * x86_64 is able to execute x86 binaries
 * aarch64 is able to execute arm binaries
 * etc.

Added qemu-hexagon support to `getExternalExecutor`.

`std.Target.canExecBinaries` of is removed; callers should use the more
powerful `getExternalExecutor` instead.

Now that `zig test` tries to run the resulting binary no matter what,
this commit has a follow-up change to the build system and docgen to
utilize the `getExternalExecutor` function and pass `--test-no-exec`
in some cases to avoid getting the error.

Additionally:

 * refactor: extract NativePaths and NativeTargetInfo into their own
   files named after the structs.
 * small improvement to langref to reduce the complexity of the `callconv`
   expression in a couple examples.
Observed on aarch64-macos when trying to execute an x86_64-macos
binary.
@andrewrk andrewrk changed the title zig build: promote qemu, wine, wasmtime, darling, and rosetta improve detection of how to execute binaries on the host Dec 3, 2021
@andrewrk andrewrk merged commit 1cac99c into master Dec 3, 2021
@andrewrk andrewrk deleted the zig-build-executors branch December 3, 2021 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant