macos mixed C/C++ and Zig builds fail

[prime31]

The last working zig version was 0.7.0 for mixed builds on macos. There have been various different errors since then but it seems the error has now stabilized so I figure I'll report it.

The test project is available here. Full steps to repro:

• build zig from master (tested every few days since 0.7.0 dropped)
• git clone --recursive https://github.com/prime31/zig-upaya/
• cd zig-upaya
• zig build empty

The resulting error seems to be a missing include path that has popped up after 0.7.0:

Compile C Objects [1/9] cimgui.cpp... error(compilation): clang failed with stderr: In file included from /Users/mikedesaro/zig-upaya/src/deps/sokol/compile_sokol.c:12:
/Users/mikedesaro/zig-upaya/src/deps/sokol/sokol/sokol_app.h:1088:17: fatal error: 'Metal/Metal.h' file not found

[mikdusan]

which macOS version?

[prime31]

Using the latest Big Sur on both x86 and ARM.

[mikdusan]

yes, these notable changes in how Apple wants software built since Catalina:

• system no longer ships with /usr/include
• SDK use is mandatory unless temporary workaround to force install of system headers from comandlinetools

since Big Sur:

• difficult path to reliable OS version detection on Big Sur for binaries built with SDK versions < 11.x
• dlopen-only access to system .dylib and framework binaries
• ARM, a new arch
• ARM also requires adhoc-signing of binaries which Apple's linker apparently does but embedded LLVM's linker does not (haven't verified this myself)

I'm impressed you've been able to get builds working (prior to this issue) using system libs, system frameworks, 3rd-party libs, etc with relatively minimal build.zig tweaking.

The exact problem to that compile error is that zig forces use of bundled macos /usr/include headers but that version is incompatible with my Xcode 12.3's SDK 11.1 (Big Sur). Once I manually remove zig bundled /usr/include, and replace with SDK 11.1 bundled /usr/include the file compiles.

The problem however, is we currently have no easy way with build.zig or zig command line to stop insertion of those bundled includes on the command line. Do note this only solves the compile issue. It will not solve linking issues. We would still be forced to use export ZIG_SYSTEM_LINKER_HACK=1 for at least 2 reasons on Big Sur.

Here's how I tweaked the command-line to make cimgui.cpp compile. [note I was lazy and forgot to insert c++ include dirs, but seems that .cpp doesn't need it]:

1. remove all -isystem command line options
2. add:

-isysroot `xcrun -show-sdk-path` \
-iwithsysroot /usr/include \
-iframeworkwithsysroot /System/Library/Frameworks \
-isystem /Users/mike/project/zig/work/main/lib/include \

and this produces the following search with cimgui.cpp compilation:

 /Users/mike/project/zig/work/upaya/src/deps/sokol
 /Users/mike/project/zig/work/upaya/src/deps/stb/src
 /Users/mike/project/zig/work/upaya/src/deps/imgui
 /Users/mike/project/zig/work/upaya/src/deps/imgui/cimgui
 /Applications/Xcode_12.3.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
 /Users/mike/project/zig/work/main/lib/include
 /Applications/Xcode_12.3.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)

[prime31]

Many thanks for the workaround.

The exact problem to that compile error is that zig forces use of bundled macos /usr/include headers but that version is incompatible with my Xcode 12.3's SDK 11.1 (Big Sur).

I am guessing this means there is no way to solve this from build.zig. Is there a fix planned when using the bundled headers? Perhaps an option to drop back to using the system headers like 0.7.0 did as a temporary hack is possible somehow?

[mikdusan]

Is there a fix planned when using the bundled headers? Perhaps an option to drop back to using the system headers like 0.7.0 did as a temporary hack is possible somehow?

nothing clear yet. One thing that should be possible right now, and I know it sounds like "but why?" ... but you can setup these 3rd party libs that build against frameworks, to build with Xcode's toolchain. Hopefully any @cImport you do doesn't deeply include any frameworks. So basically build .zig, and build any .c stub/interop with zig and stuff that needs a richer SDK have build.zig use xcode's toolchain.

This approach might work out... thinking about it we could have a kit that adds xcode toolchain support to build.zig in the long run. Needless to say, this kit would be for macos-hosted builds only, and nothing like the out-of-the-box convenience that zig's cross-compilation offers.

[prime31]

Gotcha. I think for the time being since 0.7.0 works like a champ on ARM and x86 I’ll stick with it for macOS builds and use master for Linux.

I like to have a good, workable version of zig to give to people that are interested in learning it and as long as we have 0.7.0 working so nicely all should be well ;)

[floooh]

FYI @prime31, these are the current macOS workarounds I have in sokol-zig to add the /usr/include and frameworks-directory:

https://github.com/floooh/sokol-zig/blob/ec0f1fe5452c560a1e489683cc726636ff0587bd/build.zig#L6-L12

In addition to this, the system-linker-hack must be used (export ZIG_SYSTEM_LINKER_HACK=1).

This is with macOS Big Sur 11.2 Beta, Xcode 12.2 and Zig 0.7.1

[prime31]

@floooh oddly, on latest non-beta Big Sur (11.1) with Xcode 12.3 that fails for me on x86 (didn't try ARM). I pulled your repo and grabbed zig master and it results in a linker error oddly enough.

export ZIG_SYSTEM_LINKER_HACK=1 && ../zig-master/build/zig build run-clear
warning(link): unexpected LD stderr: ld: library not found for -lSystem

error(link): ld: library not found for -lSystem

error: LDReportedFailure

That looks just like the norma LLD linker error but ZIG_SYSTEM_LINKER_HACK is definitely set (both in my .zshrc and directly here). Very odd.

[floooh]

Hmm export ZIG_SYSTEM_LINKER_HACK=1 && zig build still works fine here (with Xcode 12.2 and zig version 0.7.1+4128eea00). Now I'm afraid upgrading to Xcode 12.3 :/

[frarees]

Same error @prime31 reports. Using the linker hack in my .zprofile.

zig build -Drelease-safe
warning(link): unexpected LD stderr: ld: library not found for -lSystem

error(link): ld: library not found for -lSystem

error: LDReportedFailure

macOS: Big Sur 11.1 (20C69)
arch: x86_64 (2,7 GHz Quad-Core Intel Core i7)
zig: 0.8.0-dev.847+148c887ac (latest as of now, from here)
xcode: 12.3 (12C33)

Related: #7623, #7569. May get addressed in 0.8.0.

[mikdusan]

This is with macOS Big Sur 11.2 Beta, Xcode 12.2 and Zig 0.7.1

@floooh Zig 0.7.1 is binary download or that you built? If you built it, which Xcode/SDK used to create zig binary?

[floooh]

@mikdusan installed via homebrew with --HEAD, but not updated for a few weeks:

Built from source on 2020-12-23 at 12:07:52

zig version prints:

0.7.1+4128eea00

clang --version prints:

Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: x86_64-apple-darwin20.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

The macOS SDK is MacOSX11.0.sdk

[kubkon]

Unfortunately, if you link against any framework on macOS using the LLVM linker, you'll be greeted by a segfault, for which the only workaround currently is to use the system linker hack. In terms of the framework headers @prime31, @mikdusan is spot on: you now need to specify the sysroot manually with the contents of $(xcrun --show-sdk-path). Off the top of my head, perhaps it might be worth investigating how much effort it'd be to propagate the result of that call from system.zig to the Compilation.zig module. I reckon it should be doable and enabled only when compiling natively.

[kubkon]

@floooh oddly, on latest non-beta Big Sur (11.1) with Xcode 12.3 that fails for me on x86 (didn't try ARM). I pulled your repo and grabbed zig master and it results in a linker error oddly enough.

export ZIG_SYSTEM_LINKER_HACK=1 && ../zig-master/build/zig build run-clear
warning(link): unexpected LD stderr: ld: library not found for -lSystem

error(link): ld: library not found for -lSystem

error: LDReportedFailure

That looks just like the norma LLD linker error but ZIG_SYSTEM_LINKER_HACK is definitely set (both in my .zshrc and directly here). Very odd.

Hmm, that looks worrying. I can confirm what you are seeing is an actual system linker reported error and not LLD's. I'll need to investigate further but it looks like perhaps something changed with Xcode 12.3. Did you try with CLT instead?

[kubkon]

I can confirm this is a bug in master, and #7719 fixes it. @prime31 and @floooh, with this fix I was able to compile zig-upaya and sokol-zig on arm64 11.1 with Xcode 12.3 (using the system linker hack for linking). Also, I noticed that in both projects you explicitly concatenate SDK path with system include dirs on macOS: this is no longer required as we handle this automatically on 11.1 in the compiler. All you need now is to specify which frameworks you want to link with, and that should be it! :-)

[mikdusan]

@prime31 just going to drop the build.zig I was using when testing/incorporating the latest merged PRs and one pending PR:

const builtin = @import("builtin");
const std = @import("std");
const Builder = std.build.Builder;

// This works on host Big Sur 11.1 w/ Xcode 12.3, with zig also built with Xcode 12.3 .
//
// NOTE: when PR #7714 https://github.com/ziglang/zig/pull/7714 is merged, this will also
// work with zig continuous integration macos tarballs posted on Zig website. Until then,
// those tarballs are subject to issue #7569 https://github.com/ziglang/zig/issues/7569
// and Big Sur users will need to build their own zig from sources w/ SDK 11.1 .
//
// -- mikdusan

pub fn linkArtifact(b: *Builder, exe: *std.build.LibExeObjStep, target: std.build.Target, comptime prefix_path: []const u8) !void {
    exe.linkLibC();

    var c_flags = std.ArrayList([]const u8).init(b.allocator);
    try c_flags.append("-std=c99");

    switch (std.Target.current.os.tag) {
        .macos => {
            try c_flags.append("-ObjC");
            try c_flags.append("-fobjc-arc");

            // make verbose (note: contents will only be shown on failure as per `zig build`)
            try c_flags.append("-v");

            // LLD does not support **framework** text ABI (.tbd) files.
            // This forces zig to use system `ld` instead of embedded `lld`.
            //
            // NOTE: PR #7715 https://github.com/ziglang/zig/pull/7715 was merged
            // to support environ from Builder. Prior to merge, using `env_map` has no effect.
            try b.env_map.set("ZIG_SYSTEM_LINKER_HACK", "1");

            // NOTE: PR #7719 https://github.com/ziglang/zig/pull/7719 was merged
            // the following code is no longer required.
            if (false) {
                const sdk_dir = try std.zig.system.getSDKPath(b.allocator);
                const fwk_dir = try std.mem.concat(b.allocator, u8, &[_][]const u8 { sdk_dir, "/System/Library/Frameworks" });
                const inc_dir = try std.mem.concat(b.allocator, u8, &[_][]const u8 { sdk_dir, "/usr/include" });

                try c_flags.appendSlice(&[_][]const u8{ "-isystem", inc_dir });
                try c_flags.appendSlice(&[_][]const u8{ "-F", fwk_dir });

                exe.addFrameworkDir(fwk_dir);
            }

            exe.linkFramework("Foundation");
            exe.linkFramework("Cocoa");
            exe.linkFramework("Quartz");
            exe.linkFramework("QuartzCore");
            exe.linkFramework("Metal");
            exe.linkFramework("MetalKit");
            exe.linkFramework("OpenGL");
            exe.linkFramework("Audiotoolbox");
            exe.linkFramework("CoreAudio");
            exe.linkSystemLibrary("c++");
        },
        .linux => {
            exe.linkSystemLibrary("GL");
            exe.linkSystemLibrary("GLEW");
            exe.linkSystemLibrary("X11");
        },
        else => {},
    }

    exe.addIncludeDir(prefix_path ++ "src/deps/sokol");
    exe.addCSourceFile(prefix_path ++ "src/deps/sokol/compile_sokol.c", c_flags.items);
}