[SR-3819] ICU often needs -ldl, even if linked statically

[weissi]

Previous ID	SR-3819
Radar	None
Original Reporter	@weissi
Type	Bug

Attachment: Download

Additional Detail from JIRA

Votes	6
Component/s	Compiler
Labels	Bug, Driver, Linux
Assignee	None
Priority	Medium

md5: 52de48fd6048234a1c72d6a42d29e78d

Issue Description:

If ICU isn't compiled with --disable-dyload it'll need -ldl to be linked. That's a problem in Swift when producing a static executable:

$ echo 'print("Hello World")' > test.swift && swiftc -static-executable test.swift
/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/libicuuc.a(putil.ao):function uprv_dl_open_55: error: undefined reference to 'dlopen'
/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/libicuuc.a(putil.ao):function uprv_dlsym_func_55: error: undefined reference to 'dlsym'
/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/libicuuc.a(putil.ao):function uprv_dl_close_55: error: undefined reference to 'dlclose'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

My Swift compiler is version

Swift version 3.1-dev (LLVM 5c165fb715, Clang e540ba0c30, Swift 3d3fdecbb4)
Target: x86_64-unknown-linux-gnu

which is master of 2017-02-01.

I already verified that it can be fixed the following way:

/tmp/jw-swift/usr/bin/swiftc -o main.o -c main.swift
"/usr/bin/ld.gold" "-z" "relro" "--hash-style=gnu" "--build-id" "-m" "elf_x86_64" "-static" "-o" "test" "/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/crt1.o" "/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/crti.o" "/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/crtbeginT.o" "-L/tmp/jw-swift/usr/lib/swift_static/linux" "-L/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0" "-L/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu" "-L/lib/x86_64-linux-gnu" "-L/lib/../lib64" "-L/usr/lib/x86_64-linux-gnu" "-L/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../.." "-L/usr/lib/llvm-3.8/bin/../lib" "-L/lib" "-L/usr/lib" "/tmp/jw-swift/usr/lib/swift_static/linux/x86_64/swift_begin.o" "main.o" "-ldl" "-lswiftCore" "-lswiftImageInspectionStatic" "--defsym=__import_pthread_self=pthread_self" "--defsym=__import_pthread_once=pthread_once" "--defsym=__import_pthread_key_create=pthread_key_create" "-lpthread" "-licui18n" "-licuuc" "-licudata" "-lswiftCore" "-lswiftSwiftOnoneSupport" "/tmp/jw-swift/usr/lib/swift_static/linux/x86_64/swift_end.o" "-lstdc++" "-lm" "--start-group" "-lgcc" "-lgcc_eh" "-lc" -ldl "--end-group" "/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/crtend.o" "/usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/crtn.o"

the only difference between this and the standard invocation is the -ldl inbetween the --start-group and --end-group linker flags.

I'd be happy to fix this but I'm unsure where exactly to do that. Any pointers?

Notes

$ icu-config --ldflags
 -L/usr/lib/x86_64-linux-gnu -licui18n -licuuc -licudata  
$ icu-config --ldflags-system
-ldl -lm   

• I'm running Ubuntu 16.04

• it's not enough to just pass -ldl to swiftc (or -Xlinker -ldl)

• the -ldl NEEDS to be in between ld.gold's --start-group and --end-group and I'm unsure how to get it there

• the addition of -ldl should obviously only happen if ICU is compiled without --disable-dyload.

[belkadan]

The link args come from utils/static-executable-args.lnk, which is copied into place by CMake. We could do some configure-time tests to figure out whether -ldl is necessary; do you think that would be sufficient? I'd rather not have the compiler call out to icu-config every time it needs to link a static executable.

[weissi]

@belkadan yes, I had seen that file and it would be enough to put -ldl in there but then it would even do that if your system's ICU is compiled with --disable-dyload.

But thinking about it, it probably improves the current situation because 1) it's unlikely that your ICU is compiled with --disable-dyload and 2) we can't make fully working static binaries anyway because glibc doesn't support that properly (and libFoundation.a also isn't a thing (yet)). I'll make a pull request...

[weissi]

@belkadan #7188

[belkadan]

I was thinking rather than update the file directly, we would change it to use CMake's configure_file feature, which allows us to insert substitutions. I'm fine with CMake calling icu-config.

[belkadan]

I suppose that's not great for binary distributions, though.

[weissi]

Sorry about that. But yes, as you say it shouldn't really be checked on the system that builds swiftc. And I'm guessing most users use the binary distribution...
Hmm, unsure what to do really.

[spevans]

For static binaries (using the -static-executable option) you should not link in libdl which is obviously a problem given how libicu is compiled on ubuntu.

To work around this I added a --libicu option to the swift build script which will compile up libicu using the correct configure options to produce static versions of the libraries that dont require libdl and put them in lib/swift_static/linux and shared versions in the lib/swift/linux. This means that if building a binary distribution you could provide versions of libicu suitable for static linking without worrying about how the distribution provided package has configured it.

My intention with utils/static-executable-args.lnk was that it could be overridden by a package maintainer or built from a script at build time if building on a different OS eg android or FreeBSD, the static file is just there as an initial version for ubuntu.

[weissi]

@spevans, I do agree that -ldl for static binaries is not quite right (but linking glibc statically is so too). What about the following: If Swift is compiled with --libicu we do not put -ldl in utils/static-executable-args.lnk but if it's not we do (as it's as you say an initial version for Ubuntu).

[spevans]

You are correct that if Swift is compiled with --libicu then -ldl should not be added. The easiest way to do this is probably to have a script like utils/gen-static-stdlib-link-args that does the same for static-executable-args.lnk (or expand utils/gen-static-stdlib-link-args to generate both). This is called from https://github.com/apple/swift/blob/master/lib/Driver/CMakeLists.txt#L24 and has knowledge of whether --libicu was used. This script can also be expanded in the future to handle other ELF platforms that may have different library requirements.

If the only reason to create static binaries using -ldl is to get the libicu statically linked into the executable then the -static-stdlib option should already do this on ubuntu as the static-stdlib-args.lnk file contains full paths to the ICU .a library files

The alternative is to not copy/create the static-executable-args.lnk file at all when --libicu has not been used. swiftc sees the lack of lib/swift_static/linux/static-executable-args.lnk as an indication that static linking isnt supported and gives the following error:

$ ~/swift/usr/bin/swiftc  -emit-executable -static-executable -o main.static main.swift                                                   
LLVM ERROR: -static-executable not supported on this platform

This is probably the correct action to take when libicu hasnt been built to be fully static.

[swift-ci]

Comment by jamlee (JIRA)

this problem is up to me.

[MaxDesiatov]

After #19860 has been merged, can this issue be closed now?

[spevans]

No, it requires the fix in #15183 which just needs an update for PR comments but otherwise works:

$ cat test.swift
print("Hello")
$ ~/swift-install/usr/bin/swiftc -static-executable -o test test.swift
$ ./test
Hello
$ ldd test
not a dynamic executable
$ ls -l test
-rwxrwxr-x 1 spse spse 36853040 Nov  4 21:50 test

[weissi]

@spevans cool, thanks! Mind checking nm test | grep ' dl' ? The bug was about us needed the static version of libdl (yes, that's a thing). I added that in #7188 and it's still there:

https://github.com/apple/swift/blob/master/utils/static-executable-args.lnk#L15

So if this is truly fixed, then we should remove -ldl from utils/static-executable-args.lnk again and then we can close this issue.

[spevans]

Ok, firstly libICU is now compiled without use of dlopen()

$ nm ~/swift-install/usr/lib/swift_static/linux/libicuucswift.a |grep ' dl'
$ nm ~/swift-install/usr/lib/swift_static/linux/libicui18nswift.a |grep ' dl'
$ nm ~/swift-install/usr/lib/swift_static/linux/libicudataswift.a |grep ' dl'
$ cat ~/swift-install/usr/lib/swift_static/linux/static-executable-args.lnk 
-static
-lswiftCore
-lswiftImageInspectionStatic
-lpthread
-latomic
-licui18nswift
-licuucswift
-licudataswift

$ ~/swift-install/usr/bin/swiftc -v -static-executable -o static_test ~/src/test/test.swift 
Swift version 4.2-dev (LLVM 49db994493, Clang 256dc06b21, Swift b5d0def2db)
Target: x86_64-unknown-linux-gnu
/home/spse/swift-install/usr/bin/swift -frontend -c -primary-file /home/spse/src/test/test.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -color-diagnostics -module-name static_test -o /tmp/test-fc87ff.o
/home/spse/swift-install/usr/bin/swift-autolink-extract /tmp/test-fc87ff.o -o /tmp/test-f85369.autolink
/usr/bin/clang++ -fuse-ld=gold -target x86_64-unknown-linux-gnu /tmp/test-fc87ff.o @/tmp/test-f85369.autolink -L /home/spse/swift-install/usr/lib/swift_static/linux @/home/spse/swift-install/usr/lib/swift_static/linux/static-executable-args.lnk --target=x86_64-unknown-linux-gnu -v -o static_test
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.3.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/bin/ld.gold" -z relro --hash-style=gnu -m elf_x86_64 -static -o static_test /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crt1.o /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/crtbeginT.o -L/home/spse/swift-install/usr/lib/swift_static/linux -L/usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../.. -L/usr/lib/llvm-6.0/bin/../lib -L/lib -L/usr/lib /tmp/test-fc87ff.o -lswiftSwiftOnoneSupport -lswiftCore -lswiftCore -lswiftImageInspectionStatic -lpthread -latomic -licui18nswift -licuucswift -licudataswift -lstdc++ -lm --start-group -lgcc -lgcc_eh -lc --end-group /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/crtend.o /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crtn.o

Note however that glibc still has its own internal dl* functions but none of the -ldl ones

$ nm static_test |grep ' dl'
0000000000844da0 t dlclose_doit
00000000023b97a0 b dl_close_state.10955
0000000000841790 t dlinfo_doit
00000000008082b0 W dl_iterate_phdr
00000000008418e0 t dlmopen_doit
0000000000844ca0 t dlopen_doit
0000000000842a80 t dl_open_worker
0000000000844de0 t dlsym_doit
0000000000844eb0 t dlvsym_doit

[weissi]

@spevans awesome, thanks! Yes, the glibc ones we can't fix without dropping glibc which is a bigger task not for today. Should we then just remove -ldl from utils/static-executable-args.lnk?