Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmake: fix brotli lib order #13761

Closed
wants to merge 1 commit into from

Conversation

vszakats
Copy link
Member

@vszakats vszakats commented May 23, 2024

Fix the root cause that resulted in missing symbols when linking brotli statically with e.g. binutils ld (and any other "picky" linker, or "traditional" linker as CMake now calls them).

Also drop existing workaround that added brotli libs twice to the lib list.

x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.text$ProcessCommands[ProcessCommands]+0xbb5): undefined reference to `BrotliTransformDictionaryWord'
x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.text$SafeProcessCommands[SafeProcessCommands]+0xe8a): undefined reference to `BrotliTransformDictionaryWord'
x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.rdata$.refptr._kBrotliContextLookupTable[.refptr._kBrotliContextLookupTable]+0x0): undefined reference to `_kBrotliContextLookupTable'
x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.rdata$.refptr._kBrotliPrefixCodeRanges[.refptr._kBrotliPrefixCodeRanges]+0x0): undefined reference to `_kBrotliPrefixCodeRanges'
x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x21): undefined reference to `BrotliDefaultAllocFunc'
x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x2f): undefined reference to `BrotliDefaultFreeFunc'
x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x10e): undefined reference to `BrotliSharedDictionaryCreateInstance'
x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateCleanup[BrotliDecoderStateCleanup]+0xf4): undefined reference to `BrotliSharedDictionaryDestroyInstance'
collect2: error: ld returned 1 exit status

Breakage reproducible with curl-for-win config "win-gcc" and this curl-for-win patch:

--- a/curl.sh
+++ b/curl.sh
@@ -65,14 +65,6 @@ _VER="$1"
     fi
   fi

-  # Ugly hack. Everything breaks without this due to the accidental ordering
-  # of libs and objects, and offering no universal way to (re)insert libs at
-  # specific positions. Linker complains about a missing --end-group, then
-  # adds it automatically anyway.
-  if [ "${_LD}" = 'ld' ]; then
-    LDFLAGS+=' -Wl,--start-group'
-  fi
-
   if [ "${_OS}" = 'win' ]; then
     # Link lib dependencies in static mode. Implied by `-static` for curl,
     # but required for libcurl, which would link to shared libs by default.

(above hack still required for some non-brotli cases, e.g. libssh2 and zlib.)

Assisted-by: Kai Pastor
Ref: #10857 (comment)
Follow-up to 1e3319a #10857
Closes #13761


/cc @dg0yt

Fix the root cause that resulted in missing symbols when linking brotli
statically with binutils `ld`.

Also drop existing workaround that added brotli libs twice to the lib
list.

```
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.text$ProcessCommands[ProcessCommands]+0xbb5): undefined reference to `BrotliTransformDictionaryWord'
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.text$SafeProcessCommands[SafeProcessCommands]+0xe8a): undefined reference to `BrotliTransformDictionaryWord'
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.rdata$.refptr._kBrotliContextLookupTable[.refptr._kBrotliContextLookupTable]+0x0): undefined reference to `_kBrotliContextLookupTable'
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(decode.c.obj):decode.c:(.rdata$.refptr._kBrotliPrefixCodeRanges[.refptr._kBrotliPrefixCodeRanges]+0x0): undefined reference to `_kBrotliPrefixCodeRanges'
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x21): undefined reference to `BrotliDefaultAllocFunc'
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x2f): undefined reference to `BrotliDefaultFreeFunc'
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x10e): undefined reference to `BrotliSharedDictionaryCreateInstance'
/usr/local/Cellar/mingw-w64/11.0.1_1/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: .../curl/brotli/_x64-win-ucrt/usr/lib/libbrotlidec.a(state.c.obj):state.c:(.text$BrotliDecoderStateCleanup[BrotliDecoderStateCleanup]+0xf4): undefined reference to `BrotliSharedDictionaryDestroyInstance'
collect2: error: ld returned 1 exit status
```

Breakage reproducible with curl-for-win config `gcc' and this patch:
```diff
--- a/curl.sh
+++ b/curl.sh
@@ -65,14 +65,6 @@ _VER="$1"
     fi
   fi

-  # Ugly hack. Everything breaks without this due to the accidental ordering
-  # of libs and objects, and offering no universal way to (re)insert libs at
-  # specific positions. Linker complains about a missing --end-group, then
-  # adds it automatically anyway.
-  if [ "${_LD}" = 'ld' ]; then
-    LDFLAGS+=' -Wl,--start-group'
-  fi
-
   if [ "${_OS}" = 'win' ]; then
     # Link lib dependencies in static mode. Implied by `-static` for curl,
     # but required for libcurl, which would link to shared libs by default.
```
(curl-for-win hack still required for some non-brotli cases.)

Assisted-by: Kai Pastor
Ref: curl#10857 (comment)
Follow-up to 1e3319a curl#10857
Closes #xxxxx
Copy link
Contributor

@dg0yt dg0yt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

FTR CMake speaks of "Traditional linkers" and "Some linkers" when it comes to new de-duplication policies. "Traditional linker" is a good term IMO. It is not just "binutils ld".
https://cmake.org/cmake/help/latest/policy/CMP0156.html

@vszakats vszakats closed this in 7508e9e May 24, 2024
@vszakats vszakats deleted the cmake-fix-brotli-lib-order branch May 24, 2024 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

None yet

2 participants