You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[SR-648] swift package manager should have the option to produce statically linked binaries #5432
SR-730 Flag to statically link Swift standard library
Issue Description:
When I do swift build for a simple hello world swift program it ldd tells me that it is linked against libstdc++.so.6, libswiftCore.so and other .so files. I would like to ask swift build to produce a single file that is not dynamically linked against any shared objects.
can you please provide your build output, and your "Hello world" program. Are you running it on Linux or Mac OS ?
I dont see that "libstdc" is included in my examples (not for exe or lib)
Discussion of this on the lists suggests this isn't yet possible, so we need to open a bug on Swift itself so that it gains the option to link statically on Linux.
I'm checking to see if this is working now (this issue hasn't been updated for a while now). If it works, great, but if not, it seems to me that we are out of time for Swift 3 for this.
Should a static executable option be added to swiftc or the swift-build-tool? I think an `-emit-static-executable' or '-emit-executable -static' option added to swiftc might be the best approach rather than the package manager so it can be used directly
I have a solution for this for Linux/ELF which uses a small stub to fix up the linker issues which I can turn into a PR
static_stub.c
#ifdefined(__LINUX__) && defined(__ELF__)
#error"This only works on Linux/ELF"// Needs testing on other ELF platforms
#else
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>
#define__USE_GNU
#include <link.h>
// This forces resolving of these weak symbols but keeps them hidden// externallyconstvoid *unused1__attribute__ ((unused, visibility("internal"))) = pthread_self;
constvoid *unused2__attribute__ ((unused, visibility("internal"))) = pthread_key_create;
constvoid *unused3__attribute__ ((unused, visibility("internal"))) = pthread_once;
// libdl emulation to load the protocol and metadata sections, these get fixed// using the --defsym linker commands belowexternvoid *__swift2_protocol_conformances_start;
externvoid *__swift2_type_metadata_start;
// Fake a handle if opening NULL (this binary) else abortvoid *
dlopen(constchar *filename, intflag)
{
if (filename != NULL) {
fprintf(stderr, "dlopen called with filename=%s\n", filename);
abort();
}
return (void *)1;
}
// Hardcoded to allow lookup of 2 known symbols, enough for nowvoid *
dlsym(void *handle, constchar *symbol)
{
if (handle != (void *)1) {
fprintf(stderr, "dlsym(): bad handle: %p\n", handle);
abort();
}
if (!strcmp(symbol, ".swift2_protocol_conformances_start")) {
return &__swift2_protocol_conformances_start;
} elseif (!strcmp(symbol, ".swift2_type_metadata_start")) {
return &__swift2_type_metadata_start;
} else {
returnNULL;
}
}
intdladdr(constvoid *addr, Dl_info *info)
{
// FIXME: Dummy implementation for now, returns `not found'fprintf(stderr, "dladdr() required for %p\n", addr);
// info declared as nonnull in dlfcn.hinfo->dli_fname = NULL;
info->dli_fbase = NULL;
info->dli_sname = NULL;
info->dli_saddr = NULL;
return0;
}
intdlclose(void *handle)
{
if (handle != (void *)1) {
fprintf(stderr, "dlclose(): bad handle: %p", handle);
}
return0;
}
// Hardcoded to return an empty structure, the caller of this only// cares about the filename anyway and can deal with it being NULLintdl_iterate_phdr(int (*callback) (structdl_phdr_info *info,
size_tsize, void *data), void *data)
{
// Dummy empty structure for dl_iterate_phdrstructdl_phdr_infophdr_info = { .dlpi_addr = 0 };
returncallback(&phdr_info, sizeof(structdl_phdr_info), data);
}
// These commands will get extracted by `swift-autolink-extract'// and passed to clang++
#defineAUTOLINK_ENTRY(x) asm(".asciz \""x"\"")
asm(".section .swift1_autolink_entries");
AUTOLINK_ENTRY("-static");
AUTOLINK_ENTRY("-Xlinker --defsym=__swift2_protocol_conformances_start=.swift2_protocol_conformances_start");
AUTOLINK_ENTRY("-Xlinker --defsym=__swift2_type_metadata_start=.swift2_type_metadata_start");
AUTOLINK_ENTRY("-licui18n");
AUTOLINK_ENTRY("-licuuc");
AUTOLINK_ENTRY("-licudata");
AUTOLINK_ENTRY("-lpthread");
#endif// linux && ELF
$ clang -O -Wall -Werror -c static_stub.c -o static_stub.o
$ swiftc -static-stdlib -emit-executable -o main main.swift static_stub.o
$ file main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=41e9e4c815e695b8369a03239db50e06d4f00085, not stripped
$ ldd main
not a dynamic executable
$ ./main
Hello there
This method takes advantage of the auto link section so the static_stub can provide its own linker commands
$ swift-autolink-extract static_stub.o
-static
-Xlinker --defsym=__swift2_protocol_conformances_start=.swift2_protocol_conformances_start
-Xlinker --defsym=__swift2_type_metadata_start=.swift2_type_metadata_start
-licui18n
-licuuc
-licudata
-lpthread
I have a slightly different solution that patches swift/stdlib which reduces the size of the libdl emulation code and makes it simpler to understand which might be a better approach if its decided to add the option to swiftc rather than swiftpm
I would be happy to help review/mentor this project, for any future readers of this bug.
Just to follow up on your note @ddunbar, I wanted to ask, is there anything that I myself could do that would help move this long? Based on Simon's and Ankit's comment above, it sounds as if the next steps are at least as much procedural as technical:
1. Apple/core team agrees to include a clone or fork of ICU, and build it as part of building Swift.
2. Similarly, Apple/core team agrees to include a clone or fork of a stripped-down version of libcurl, which contains only what is needed to support Foundation's networking requirements on Linux.
So as regards item 2, there's probably some technical work to figure out how to build the stripped down libcurl on Ubuntu that would meet Foundation's needs. With a bit of help, I could probably figure that out, and would be glad to do so.
But as regards the procedural work, who is the right person to explain and plan for these changes in how Swift is built? I doubt it's me, since it seems like a core team issue. Or is this a feature that realistically is at a stage where it first needs to be "pitched" for, because many folks won't think that the benefit (easily deployable executables for language users) is worth the cost (a more complicated language build system)?
@ddunbar We actually do have fully static executables working on macOS with SwiftPM, just not on Linux
we... don't as of Swift 5
It looks to me that static linking doesn't work on Linux nor macOS any longer. According to SR-9946 "Swift on macOS isn't going to support static linking at all", and static linking is disabled for Linux in SPM.
Would it make sense to disable the command line `--static-swift-stdlib` option in the `swift build` command for the time being? Statically linking binaries doesn't work right now on Linux. On macOS, it also doesn't make sense to have that option, since statically linking binaries is not possible anymore (if I understand correctly).
I think it's confusing for Linux users that calling `swift build` with the `--static-swift-stdlib` option doesn't actually produce the desired result and does not produce a warning like on macOS. If disabling that option is not favourable, I would propose that we at least add a warning.
As a side note, what would it take to get statically linked binaries working on Linux? I would be interested in working on this issue if someone could mentor me.
I agree it would be best to disable the option if it does not work.
That said, being able to distribute stand-alone CLI tools would still be pretty handy on Linux.
Hi, just wanted to mention that I've just been able to produce a binary with statically linked stdlibs with the latest 5.3 nightly docker image.
docker run --rm -v $(pwd):/src -w /src swiftlang/swift:nightly-5.3-bionic swift build --enable-test-discovery --product MyProduct -c release -Xswiftc -static-stdlib
In other words: Using -Xswiftc -static-stdlib instead of --static-swift-stdlib provides the wanted functionality. Many thanks to @drexin for making this possible.
I think -static-stdlib is still broken for some uses of FoundationNetworking especially methods that are in Foundation but call into FoundationNetworking eg String(contentsOf..) where the url is an http one, although it should be noted this is a swift-corelibs-foundation issue, not a swiftpm bug.
Activity
swift-ci commentedon Feb 1, 2016
Comment by Kostiantyn Koval (JIRA)
can you please provide your build output, and your "Hello world" program. Are you running it on Linux or Mac OS ?
I dont see that "libstdc" is included in my examples (not for exe or lib)
Example:
swift-ci commentedon Feb 1, 2016
Comment by Kostiantyn Koval (JIRA)
Also I've verified that generated `Simple.a` static library don't have any dynamic loaded dependencies
swift-ci commentedon Feb 1, 2016
Comment by Jay Buffington (JIRA)
This is in Ubuntu 15.10
swift-ci commentedon Feb 3, 2016
Comment by Kostiantyn Koval (JIRA)
Here is the output I get for the MacOS. It includes 3 dynamic libraries.
Also the package manager doesn't control this behaviour. I think it's more related to the LLBD (the build system).
mxcl commentedon Feb 19, 2016
Discussion of this on the lists suggests this isn't yet possible, so we need to open a bug on Swift itself so that it gains the option to link statically on Linux.
ddunbar commentedon Jun 9, 2016
This should be unblocked, it would be great to get this in the Swift 3.0 timeframe.
abertelrud commentedon Aug 3, 2016
I'm checking to see if this is working now (this issue hasn't been updated for a while now). If it works, great, but if not, it seems to me that we are out of time for Swift 3 for this.
abertelrud commentedon Aug 4, 2016
When I try `swift build -Xswiftc -static-stdlib` on Ubuntu 14.0.4, I get this:
abertelrud commentedon Aug 4, 2016
It isn't that clear to me where this comes from. SwiftPM doesn't seem to be passing it down.
abertelrud commentedon Aug 4, 2016
On macOS it works, so this might be a toolchain incompatibility with Ubuntu 14.0.4. I will try on Ubuntu 15.10 and file a JIRA if it's incompatible.
abertelrud commentedon Aug 4, 2016
I can reproduce this in a small test case, and I have filed SR-2280.
abertelrud commentedon Aug 4, 2016
I am removing the swift-3.0 label from this, since it is blocked by SR-2280.
spevans commentedon Sep 12, 2016
Should a static executable option be added to swiftc or the swift-build-tool? I think an `-emit-static-executable' or '-emit-executable -static' option added to swiftc might be the best approach rather than the package manager so it can be used directly
I have a solution for this for Linux/ELF which uses a small stub to fix up the linker issues which I can turn into a PR
static_stub.c
I have a slightly different solution that patches swift/stdlib which reduces the size of the libdl emulation code and makes it simpler to understand which might be a better approach if its decided to add the option to swiftc rather than swiftpm
spevans commentedon Sep 14, 2016
Proposed fix https:// github.com/swiftlang/swift/pull/4754Updated fix that doesnt use any libdl hacks but instead builds its own version of libicu:
swiftlang/swift#5394
ddunbar commentedon Nov 30, 2016
Is this done now?
15 remaining items
aciidgh commentedon Sep 24, 2018
@spevans We should probably get a thread going on forums for including ICU in the toolchain process.
Also, I remember there was some talk of separating URLSession from Foundation to isolate the libcurl dependency. /cc @parkera @weissi
parkera commentedon Sep 24, 2018
see: https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002
swift-ci commentedon Oct 19, 2018
Comment by Alexis Gallagher (JIRA)
Just to follow up on your note @ddunbar, I wanted to ask, is there anything that I myself could do that would help move this long? Based on Simon's and Ankit's comment above, it sounds as if the next steps are at least as much procedural as technical:
1. Apple/core team agrees to include a clone or fork of ICU, and build it as part of building Swift.
2. Similarly, Apple/core team agrees to include a clone or fork of a stripped-down version of libcurl, which contains only what is needed to support Foundation's networking requirements on Linux.
So as regards item 2, there's probably some technical work to figure out how to build the stripped down libcurl on Ubuntu that would meet Foundation's needs. With a bit of help, I could probably figure that out, and would be glad to do so.
But as regards the procedural work, who is the right person to explain and plan for these changes in how Swift is built? I doubt it's me, since it seems like a core team issue. Or is this a feature that realistically is at a stage where it first needs to be "pitched" for, because many folks won't think that the benefit (easily deployable executables for language users) is worth the cost (a more complicated language build system)?
krzyzanowskim commentedon May 6, 2019
we... don't as of Swift 5
It looks to me that static linking doesn't work on Linux nor macOS any longer. According to SR-9946 "Swift on macOS isn't going to support static linking at all", and static linking is disabled for Linux in SPM.
It's painful especially on Linux though.
swift-ci commentedon Jun 18, 2019
Comment by Gilles Magalhaes (JIRA)
Would it make sense to disable the command line `--static-swift-stdlib` option in the `swift build` command for the time being? Statically linking binaries doesn't work right now on Linux. On macOS, it also doesn't make sense to have that option, since statically linking binaries is not possible anymore (if I understand correctly).
I think it's confusing for Linux users that calling `swift build` with the `--static-swift-stdlib` option doesn't actually produce the desired result and does not produce a warning like on macOS. If disabling that option is not favourable, I would propose that we at least add a warning.
As a side note, what would it take to get statically linked binaries working on Linux? I would be interested in working on this issue if someone could mentor me.
swift-ci commentedon Oct 27, 2019
Comment by Mickaël Rémond (JIRA)
I agree it would be best to disable the option if it does not work.
That said, being able to distribute stand-alone CLI tools would still be pretty handy on Linux.
fabianfett commentedon Sep 11, 2020
Hi, just wanted to mention that I've just been able to produce a binary with statically linked stdlibs with the latest 5.3 nightly docker image.
docker run --rm -v $(pwd):/src -w /src swiftlang/swift:nightly-5.3-bionic swift build --enable-test-discovery --product MyProduct -c release -Xswiftc -static-stdlibIn other words: Using
-Xswiftc -static-stdlibinstead of--static-swift-stdlibprovides the wanted functionality. Many thanks to @drexin for making this possible.swiftlang/swift#33678
keith commentedon Sep 11, 2020
I've submitted #2921 which enables --static-swift-stdlib with SwiftPM
norio-nomura commentedon Nov 13, 2020
I tried
swift build --configuration release --static-swift-stdlibto buildsourcekittenonswift-5.3.1-RELEASE, but it failed with a link error like:full build log
After investigation, the link error was avoided with the following changes:
Add
apt-get update && apt-get install -y libxml2-devUse
-Xswiftc -static-stdlibinstead of--static-swift-stdlibspevans commentedon Nov 13, 2020
I think
-static-stdlibis still broken for some uses ofFoundationNetworkingespecially methods that are inFoundationbut call intoFoundationNetworkingegString(contentsOf..)where the url is anhttpone, although it should be noted this is aswift-corelibs-foundationissue, not aswiftpmbug.norio-nomura commentedon Nov 17, 2020
Since #3048 has been merged,
--static-swift-stdliboption now works properly onswift-DEVELOPMENT-SNAPSHOT-2020-11-16-a.