std.zig.system.NativeTargetInfo: use getconf for detecting glibc version on glibc systems #12567

BratishkaErik · 2022-08-22T00:33:30Z

Closes #6469 and closes #11137
Signed-off-by: Eric Joldasov bratishkaerik@getgoogleoff.me

BratishkaErik · 2022-08-22T00:36:27Z

lib/std/zig/system/NativeTargetInfo.zig

-                error.InvalidGnuLibCVersion,
-                => break :glibc_ver,
-            };
+        const getconf = std.ChildProcess.exec(.{ .allocator = allocator, .argv = &.{ "getconf", "GNU_LIBC_VERSION" } }) catch unreachable;


IIUC all glibc systems (even more, all POSIX systems https://pubs.opengroup.org/onlinepubs/000095399/utilities/getconf.html) has this program, and if not, it is a bug in systems itself.

BratishkaErik · 2022-08-22T00:37:49Z

lib/std/zig/system/NativeTargetInfo.zig

        }
+        const glibc_ver_from_stdout = getconf.stdout["glibc ".len..];


According to man:

_CS_GNU_LIBC_VERSION (GNU C library only; since glibc 2.3.2)
A string which identifies the GNU C library version on
this system (e.g., "glibc 2.3.4").

BratishkaErik · 2022-08-22T00:39:45Z

lib/std/zig/system/NativeTargetInfo.zig

@@ -641,167 +598,15 @@ pub fn abiAndDynamicLinkerFromFile(
    if (builtin.target.os.tag == .linux and result.target.isGnuLibC() and


But anyway this will be not executed on a non-glibc systems

BratishkaErik · 2022-08-22T00:48:31Z

@topolarity does this fixes #11137 (comment)
and #11137 (comment) for you?

BratishkaErik · 2022-08-23T17:57:22Z

@topolarity does this fixes #11137 (comment) and #11137 (comment) for you?

Oops, I forgot that zig doesn't provide glibc 2.35 yet :(

BratishkaErik · 2022-08-23T20:02:11Z

@topolarity does this fixes #11137 (comment) and #11137 (comment) for you?

Oops, I forgot that zig doesn't provide glibc 2.35 yet :(

error: zig does not yet provide glibc version 2.35, the max provided version is 2.34
error: unable to build glibc shared objects: InvalidTargetGLibCVersion
error: zig...
error: The following command exited with error code 1:
/home/bratishkaerik/zig/build/zig build-exe --stack 33554432 /home/bratishkaerik/zig/src/main.zig -lc /home/bratishkaerik/zig/build/zigcpp/libzigcpp.a /usr/lib/llvm/14/lib64/libclang-cpp.so.14 /usr/lib64/liblldMinGW.so /usr/lib64/liblldELF.so /usr/lib64/liblldCOFF.so /usr/lib64/liblldWasm.so /usr/lib64/liblldMachO.so /usr/lib64/liblldCommon.so /usr/lib/llvm/14/lib64/libLLVM-14.so /usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libstdc++.so -lunwind --cache-dir /home/bratishkaerik/zig/zig-cache --global-cache-dir /home/bratishkaerik/.cache/zig --name zig --pkg-begin build_options /home/bratishkaerik/zig/zig-cache/options/7X62S_IHrJHJyONEQXfH1Bt-Y-ZrGbNtZVv-QoWvD9uq09Hl8g4BY__8D6284ocN --pkg-end -I /usr/include -fno-build-id --enable-cache 
error: the following build command failed with exit code 1:
/home/bratishkaerik/zig/zig-cache/o/87897b425d0e88a403c948b1ad22e5dc/build /home/bratishkaerik/zig/build/zig /home/bratishkaerik/zig /home/bratishkaerik/zig/zig-cache /home/bratishkaerik/.cache/zig -p stage2 -Dstatic-llvm=false -Denable-llvm=true

BratishkaErik · 2022-08-23T20:05:55Z

My suggestion:

diff --git a/src/glibc.zig b/src/glibc.zig
index ceb60ff09..d714d4aa1 100644
--- a/src/glibc.zig
+++ b/src/glibc.zig
@@ -745,14 +745,7 @@ pub fn buildSharedObjects(comp: *Compilation) !void {
                     return error.InvalidTargetGLibCVersion;
                 },
             }
-        } else {
-            const latest_index = metadata.all_versions.len - 1;
-            // TODO Expose via compile error mechanism instead of log.
-            log.err("zig does not yet provide glibc version {}, the max provided version is {}", .{
-                target_version, metadata.all_versions[latest_index],
-            });
-            return error.InvalidTargetGLibCVersion;
-        };
+        } else metadata.all_versions.len - 1;
 
         {
             var map_contents = std.ArrayList(u8).init(arena);

topolarity · 2022-08-24T16:52:45Z

@topolarity does this fixes #11137 (comment) and #11137 (comment) for you?

~~Looks like if Zig dynamically links libc (e.g. -Dstatic-llvm=false), then this is still unresolved~~
Latest push resolved this

topolarity · 2022-08-24T16:57:41Z

@topolarity does this fixes #11137 (comment) and #11137 (comment) for you?

Oops, I forgot that zig doesn't provide glibc 2.35 yet :(

IIUC we should still be able to use the system glibc in this case, right?

BratishkaErik · 2022-08-24T17:02:41Z

@topolarity does this fixes #11137 (comment) and #11137 (comment) for you?

Looks like if Zig dynamically links libc (e.g. -Dstatic-llvm=false), then this is still unresolved

I force–pushed after this, can you try this version too? (and I'm using dynamic linking, too) ea7c4bc

BratishkaErik · 2022-08-24T17:07:06Z

IIUC we should still be able to use the system glibc in this case, right?

I'm hitting this #12567 (comment)

topolarity · 2022-08-24T17:17:52Z

I force–pushed recently, can you try this version too? (and I'm using dynamic linking, too) ea7cb4c

Yep, I was out of date! This appears to be resolved for me 👍

Re: the glibc version. Zig will currently fallback to using system-provided libc if you ask for system libraries (e.g. if you add -lz to the build-exe invocation, it will work). Perhaps it's worth performing the same fallback if Zig does not provide the requested version of libc but the system does?

BratishkaErik · 2022-08-27T17:19:43Z

I force–pushed recently, can you try this version too? (and I'm using dynamic linking, too) ea7cb4c

Yep, I was out of date! This appears to be resolved for me 👍

Re: the glibc version. Zig will currently fallback to using system-provided libc if you ask for system libraries (e.g. if you add -lz to the build-exe invocation, it will work). Perhaps it's worth performing the same fallback if Zig does not provide the requested version of libc but the system does?

Something like?:
[if glibc newer than supplied] —> (use system libc) —> [if system doesn't provide headers] —> (use latest supplied libc available)

topolarity

Tested on Debian (bookworm), NixOS (unstable), and Arch - all of which are affected by #6469 and #11137 and are fixed by this PR.

This change does mean linking libc will be broken on OS's with too-new libc (e.g. NixOS and Arch) until we have a fallback to link against system libc or update Zig-provided glibc, but this still seems like a step in the right direction even without that

P.S. Not sure what the CI failures are, but they seem unrelated?

lib/std/zig/system/NativeTargetInfo.zig

topolarity · 2022-08-29T18:43:52Z

Something like?:
[if glibc newer than supplied] —> (use system libc) —> [if system doesn't provide headers] —> (use latest supplied libc available)

Sounds OK to me - most of the logic should already be present in Compilation.zig. Might be as simple as updating canBuildLibC to account for the libc version

If you're up to tackling that, feel free to update this PR or start a new one - whichever you prefer 🙂

BratishkaErik · 2022-08-29T19:09:43Z

[if system doesn't provide headers] —> (use latest supplied libc available) ❌
01e3000

stage2: error if requested glibc version too high
Falling back to the max provided glibc version is insufficient as linking to shared objects compiled against the requested version will fail.

[if glibc newer than supplied] —> (use system libc) ❌
Unfortunately, not all users have required headers, our only option is updating glibc (IIRC from IRC, only core team member should do this)

topolarity · 2022-08-29T19:18:52Z

Ah, I misunderstood and thought you had meant use the Zig headers to link against the system-provided glibc.

Good catch, let's stay faithful to the requested glibc version

Previously, this code would fail to detect glibc version because it relied on libc.so.6 being a symlink which revealed the answer. On modern distros, this is no longer the case. This new strategy finds the path to libc.so.6 from /usr/bin/env, then inspects the .dynstr section of libc.so.6, looking for symbols that start with "GLIBC_2.". It then parses those as semantic versions and takes the maximum value as the system-native glibc version. closes #6469 see #11137 closes #12567

andrewrk · 2022-09-09T01:39:35Z

I created #12788 which supersedes this PR and does not depend on getconf.

…ion on glibc systems Signed-off-by: Eric Joldasov <bratishkaerik@getgoogleoff.me>

Previously, this code would fail to detect glibc version because it relied on libc.so.6 being a symlink which revealed the answer. On modern distros, this is no longer the case. This new strategy finds the path to libc.so.6 from /usr/bin/env, then inspects the .dynstr section of libc.so.6, looking for symbols that start with "GLIBC_2.". It then parses those as semantic versions and takes the maximum value as the system-native glibc version. closes #6469 see #11137 closes #12567

BratishkaErik commented Aug 22, 2022

View reviewed changes

topolarity approved these changes Aug 29, 2022

View reviewed changes

lib/std/zig/system/NativeTargetInfo.zig Show resolved Hide resolved

andrewrk mentioned this pull request Sep 9, 2022

std.zig.system.NativeTargetInfo: improve glibc version and dynamic linker detection #12788

Merged

std.zig.system.NativeTargetInfo: use getconf for detecting glibc vers…

5c0a46e

…ion on glibc systems Signed-off-by: Eric Joldasov <bratishkaerik@getgoogleoff.me>

andrewrk closed this in #12788 Sep 9, 2022

BratishkaErik mentioned this pull request Sep 9, 2022

zig cannot build against a native glibc greater than the latest cross-compilation-supported version of glibc #12797

Closed

BratishkaErik deleted the fix-glibc-version-detecting branch September 11, 2022 02:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

std.zig.system.NativeTargetInfo: use getconf for detecting glibc version on glibc systems #12567

std.zig.system.NativeTargetInfo: use getconf for detecting glibc version on glibc systems #12567

BratishkaErik commented Aug 22, 2022 •

edited by topolarity

Loading

BratishkaErik Aug 22, 2022

BratishkaErik Aug 22, 2022

BratishkaErik Aug 22, 2022 •

edited

Loading

BratishkaErik commented Aug 22, 2022 •

edited

Loading

BratishkaErik commented Aug 23, 2022

BratishkaErik commented Aug 23, 2022

BratishkaErik commented Aug 23, 2022

topolarity commented Aug 24, 2022 •

edited

Loading

topolarity commented Aug 24, 2022

BratishkaErik commented Aug 24, 2022 •

edited

Loading

BratishkaErik commented Aug 24, 2022

topolarity commented Aug 24, 2022

BratishkaErik commented Aug 27, 2022

topolarity left a comment

topolarity commented Aug 29, 2022

BratishkaErik commented Aug 29, 2022 •

edited

Loading

topolarity commented Aug 29, 2022

andrewrk commented Sep 9, 2022

		}
		const glibc_ver_from_stdout = getconf.stdout["glibc ".len..];

		@@ -641,167 +598,15 @@ pub fn abiAndDynamicLinkerFromFile(
		if (builtin.target.os.tag == .linux and result.target.isGnuLibC() and

std.zig.system.NativeTargetInfo: use getconf for detecting glibc version on glibc systems #12567

std.zig.system.NativeTargetInfo: use getconf for detecting glibc version on glibc systems #12567

Conversation

BratishkaErik commented Aug 22, 2022 • edited by topolarity Loading

BratishkaErik Aug 22, 2022

Choose a reason for hiding this comment

BratishkaErik Aug 22, 2022

Choose a reason for hiding this comment

BratishkaErik Aug 22, 2022 • edited Loading

Choose a reason for hiding this comment

BratishkaErik commented Aug 22, 2022 • edited Loading

BratishkaErik commented Aug 23, 2022

BratishkaErik commented Aug 23, 2022

BratishkaErik commented Aug 23, 2022

topolarity commented Aug 24, 2022 • edited Loading

topolarity commented Aug 24, 2022

BratishkaErik commented Aug 24, 2022 • edited Loading

BratishkaErik commented Aug 24, 2022

topolarity commented Aug 24, 2022

BratishkaErik commented Aug 27, 2022

topolarity left a comment

Choose a reason for hiding this comment

topolarity commented Aug 29, 2022

BratishkaErik commented Aug 29, 2022 • edited Loading

topolarity commented Aug 29, 2022

andrewrk commented Sep 9, 2022

BratishkaErik commented Aug 22, 2022 •

edited by topolarity

Loading

BratishkaErik Aug 22, 2022 •

edited

Loading

BratishkaErik commented Aug 22, 2022 •

edited

Loading

topolarity commented Aug 24, 2022 •

edited

Loading

BratishkaErik commented Aug 24, 2022 •

edited

Loading

BratishkaErik commented Aug 29, 2022 •

edited

Loading