[SR-730] Flag to statically link Swift standard library

[drewcrawford]

Previous ID	SR-730
Radar	None
Original Reporter	@drewcrawford
Type	Bug
Status	Closed
Resolution	Done

Attachment: Download

Environment

swift-DEVELOPMENT-SNAPSHOT-2016-02-08-a

Additional Detail from JIRA

Votes	6
Component/s	Compiler
Labels	Bug, Driver
Assignee	@drewcrawford
Priority	Medium

md5: 750294aa83c3f055bad270719f53d070

relates to:

• SR-648 swift package manager should have the option to produce statically linked binaries
• SR-2280 swiftc -static-stdlib option fails on Ubuntu 14.04 & 15.10
• SR-727 (emscripten support) Compile Swift code with statically linked stdlib
• SR-9821 Swift Static Standard Library is being treated as private api

Issue Description:

If I build an executable with swiftc:

$ swiftc test.swift

Then I find out the Swift standard library was dynamically linked:

$ otool -L test
@rpath/libswiftCore.dylib (compatibility version 0.0.0, current version 0.0.0)

However, the binary distributions also have a static version that I didn't use:

/Library/Developer/Toolchains/swift-latest.xctoolchain//usr/lib/swift_static/macosx/libswiftCore.a

I want to pass e.g. -static-stdlib to swiftc to compile a program that is statically linked with the Swift standard library.

[drewcrawford]

I attempted to go diving to find the correct flags, but have not worked it out just yet.

swiftc -o foo foo.swift /usr/local/lib/swift_static/linux/libswiftCore.a /usr/local/lib/swift_static/linux/libswiftGlibc.a -licuuc -licui18n -lbsd

"works", but still generates a dynamically-linked executable.

Diving deeper, on Linux x64, somebody is passing -lswiftCore to clang++:

/usr/bin/clang++ /tmp/foo-4c5e3f.o /usr/local/lib/swift_static/linux/libswiftCore.a /usr/local/lib/swift_static/linux/libswiftGlibc.a -licuuc -licui18n -lbsd -L /usr/local/lib/swift/linux --target=x86_64-unknown-linux-gnu -Xlinker -rpath -Xlinker /usr/local/lib/swift/linux -lswiftCore @/tmp/libswiftCore-0309d2.autolink -Xlinker -T /usr/local/lib/swift/linux/x86_64/swift.ld -o foo

But I can't figure out who did that and where I can disable it.

[drewcrawford]

On OSX, the flags I'm looking for are

/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/lib/swift_static/macosx/libswiftFoundation.a /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/lib/swift_static/macosx/libswiftObjectiveC.a /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/lib/swift_static/macosx/libswiftCore.a -lc++ -framework Foundation

Which works ✅

It seems that OSX uses ld for linking, which (for reasons unknown to me) do not try to dynamically link libswiftCore.

However, Linux uses clang++ as its linker, and attempts to link libswiftCore.so no matter what I do. So I still can't figure out how to do this on Linux.

[drewcrawford]

cc: @belkadan you're the culprit here 🙂

In ToolChains.cpp appears the code

// Always add the stdlib
1175      Arguments.push_back("-lswiftCore");

We should not "always" add -lswiftCore because then the stdlib cannot be statically linked.

[belkadan]

Um. We should certainly not require people to explicitly specify -lswiftCore normally, though. What do you recommend?

[drewcrawford]

Of course. What I would recommend would be one or both of:

1. -no-stdlib, which would not use -lswiftCore and allow me to cobble together the behavior I want
2. -static-stdlib, which would just go ahead and link the standard library/runtime statically

For background, the standard library breaks frequently (e.g.,each time the last few snapshots) right now on Linux. I currently have to coordinate binary releases around language snapshots (user needs to run same snapshot as binary was built with) and I don't have good visibility into when those come out to plan my releases around it.

One or both of these would let me (with a little effort) create static binaries, so that I'm not chasing the wind every few days.

[drewcrawford]

Just a friendly bump on this. I don't see that a solution has been blessed by core.

cc: @mxcl also expressed an interest in this bug.

[drewcrawford]

Friendly bump.

At the moment I have to recompile all binaries with each new snapshot; that's quite an effort.

[ephemer]

Just so I understand- you were able to build a binary on OS X compiled from swift sources that is entirely statically linked? This would be big news for emscripten support.

Is it really just commenting out Arguments.push_back("- lswiftCore") and running build-script with --extra-swift-flags --build-static-stdlib=1? Would you might putting together a short rundown on what changes you had to make to support this?

[drewcrawford]

Sure, let me tell The Story So Far™.

You can actually build a statically-linked binary on OSX right now. This has always worked, because Xcode does it by default for OSX executables. What you must do is simply

swiftc -L path/to/swift_static -lc++ -framework Foundation -Xlinker -force_load_swift_libs

I discovered this by simply looking through Xcode build logs.

On Linux, however, it doesn't work, for starters, because Linux linker doesn't have -force_load_swift_libs. I believe the correct incantation is

swiftc -licuuc -licu18n -lbsd /path/to/swift_static/a.a /path/to/swift_static/b.a /path/to/swift_static/c.a ...

However because swiftc forcibly injects -lswiftCore on Linux this results in a library that is both statically and dynamically linked. Which defeats the purpose.

I was hoping to hear back from @belkadan or perhaps even @mxcl regarding how to proceed on the -lswiftCore issue (we need an option to disable it I guess?), but I don't yet know what is acceptable on that question.

[mxcl]

We should have this feature and then open a bug for the swift Linux driver to fix it.

I want to propose this goes into some other file than Package.swift.

My rationale is that the Package.swift describes modules and products, but not the installation or system linkage of those products.

The enduser should have complete control over the final binary composition of their products, and not be at the mercy of their dependencies.

[drewcrawford]

This bug is the swift driver bug.

I'm not especially concerned with how this gets exposed from SwiftPM other than it should, but to me that is kind of a side issue. Right now we can't even statically link a Swift binary with a makefile.

[mxcl]

My bad, got mixed up since there is another bug open on SwiftPM for this.

[drewcrawford]

So I took a pass at this, you can see the WIP at https://github.com/drewcrawford/swift/tree/static-stdlib

That tree introduces a new flag, -static-stdlib, that in theory statically links the stdlib. This is Linux-only, I haven't yet gotten to OSX.

The immediate problem is that something odd is happening with the binaries that get built. I try to build this simple hello world program:

print("Hello world")

When linked dynamically against libswiftCore.so, it works fine. However, when linked statically against libswiftCore.a, the hello world program prints this mysterious thing:

String(_core: Swift._StringCore(_baseAddress: Swift.OpaquePointer(_rawValue: (Opaque Value)), _countAndFlags: Swift.UInt(_value: (Opaque Value)), _owner: Swift.Optional<Swift.AnyObject>.none))

It looks like the internal representation of a string, but I'm not sure why this happens. Can someone ID this print and figure out why I am printing it?

Here's a verbose build, and these invocations can reproduce the problem without any stuff in my tree (which is merely driver stuff):

'/swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swiftc' -v  -static-stdlib test2.swift
Swift version 3.0-dev (LLVM 699a786c15, Clang 77080f2c03, Swift 21c9372d82)
Target: x86_64-unknown-linux-gnu
/swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift -frontend -c -primary-file test2.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -color-diagnostics -module-name test2 -o /tmp/test2-9cbb9a.o
/swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-autolink-extract /tmp/test2-9cbb9a.o -o /tmp/test2-49baa1.autolink
/usr/bin/clang++ -ldl -lpthread -lbsd -licui18n -licuuc /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift/linux/x86_64/swift_begin.o /tmp/test2-9cbb9a.o -L /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift_static/linux --target=x86_64-unknown-linux-gnu -Xlinker -rpath -Xlinker /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift_static/linux -lswiftCore @/tmp/test2-49baa1.autolink /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift/linux/x86_64/swift_end.o -o test2

[ephemer]

Great stuff for working on this!

Off the top of my head, this is the same thing that was happening before either
a) the swift.ld linker script, or
b) the runtime protocol conformance mapping
... was incorporated into the Android port of Swift. I forget which, my recollection is leaning me towards the runtime. I think it was something to do with the type introspection.

[ephemer]

I'm not at my computer so I can't edit my comment above.. Try declaring an enum or conforming a custom Struct or Class to a protocol, if the program crashes on startup it's the runtime.

The swift.ld linker script was changed (by an apple employee from memory) to an inline assembly version as far as I understand, I lost track of that somewhere along the way. I understand the script basically performed a recursive loop to find the "end-of-chain" memory space for things like printing values (instead of just printing info about the Struct wrapper, as you're seeing here). So this contradicts what I said above, I now actually think it's more likely to be swift.ld, assuming it really is one of the two issues. In any case it's worth checking whether the linker script (in whatever current form it has) is being invoked in your statically linked branch

[drewcrawford]

Protocol conformance and enums work fine. Maybe it's the linker script?

Unfortunately, I am a complete "project internals" noob. My attempt to understand what you are talking about was to grep the tree for swift.ld. It returned one item in a changelog, but no code or script.

Could you talk a little bit about where I could find this thing and how I could determine if it causes the problem?

[swift-ci]

Comment by Wojtek Czekalski (JIRA)

@drewcrawford swift.ld has been discarded.

I've been trying to get rid of dylibs from the statically linked binary on OS X.

By running the command Drew posted with -v I was able to reduce it so that it doesn't link dylibs.

swift -frontend -c -primary-file main.swift -target x86_64-apple-macosx10.9 -color-diagnostics -module-name main -o /var/folders/.../main-4193f8.o

ld  /var/folders/.../main-4193f8.o -force_load /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_macosx.a -force_load_swift_libs -L /path/to/toolchain/usr/lib/swift_static/macosx -arch x86_64 -macosx_version_min 10.9.0 -o main -static

results in:
ld: library not found for -lobjc for architecture x86_64

The part I don't understand is where -lobjc flag is actually set.

Any suggestions are appreciated!

[ephemer]

wczekalski (JIRA User) thanks for the link. So the two points that came to my mind were actually one in the same, the runtime reading the protocol conformance table. From what I understand @hpux735 has fixed this avoid using the linker script altogether, and instead writes the conformances into the final binaries (nice work!).

Maybe @hpux735 could comment here quickly, because I'm pretty sure we saw this same issue in early versions of the armv7 port too? If not, a 'no idea' would suffice 🙂 Thanks

[hpux735]

Hi.

Yep, I've seen this a time or two. 🙂 When you link your static lib, you have to make sure that all the objects are preceeded by swift_begin.o and followed by swift_end.o. These add the metadata markers into the resulting binary so that Runtime can find the type metadata.

When the conformances are missing, the overrides of 'var description: String' can't be located, and the generic description is used. It's a pretty reliable troubleshooting tool. 🙂

[ephemer]

@hpux735 thanks! Edit: Any idea why protocol conformances and enums would work, but

[hpux735]

that part, I don't know. IIRC, if you do .description in print() you can force it to work.

To check whether the conformance markers are in there, use objdump:

wdillon@tegra-ubuntu:~$ objdump -t example |grep \.swift._
0000a008 g       .swift2_protocol_conformances  00000000              .protected .swift2_protocol_conformances_start
0000a010 g       .swift2_type_metadata  00000000              .protected .swift2_type_metadata_start
0000a008 g       .swift3_assocty    00000000              .protected .swift3_assocty_section
0000a008 g       .swift3_fieldmd    00000000              .protected .swift3_fieldmd_section
0000a008 g       .swift3_reflstr    00000000              .protected .swift3_reflstr_section
0000a008 g       .swift3_typeref    00000000              .protected .swift3_typeref_section
0000a010 g       .swift2_protocol_conformances  00000000              .protected .swift2_protocol_conformances_end
0000a018 g       .swift2_type_metadata  00000000              .protected .swift2_type_metadata_end

(meta: Why does "preformatted" never work??)

You want to see a start and end for both conformances and metadata.

[drewcrawford]

Thanks for the leads everyone)

/usr/bin/clang++ -ldl -lpthread -lbsd -licui18n -licuuc /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift/linux/x86_64/swift_begin.o /tmp/test2-9cbb9a.o -L /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift_static/linux --target=x86_64-unknown-linux-gnu -Xlinker -rpath -Xlinker /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift_static/linux -lswiftCore @/tmp/test2-49baa1.autolink /swift-dev/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift/linux/x86_64/swift_end.o -o test2

Can someone who understands the swift_begin behavior eyeball that and tell me if it looks wrong and what might be more correct?

I wanted to address one other clue from @hpux735:

IIRC, if you do .description in print() you can force it to work.

This is actually not true in my case: print("hello world"). I don't know if that makes a difference in the diagnosis.

[hpux735]

Hmm, yah, they do seem to be in the right places. Also, your behavior with print is different than what I was seeing. To put the metadata issue to rest, would you mind pasting the output of your objdump?

[drewcrawford]

Sure!

objdump -t test2 | grep \.swift._
00000000006215f0 l    d  .swift1_autolink_entries   0000000000000000              .swift1_autolink_entries
0000000000886a78 l    d  .swift2_protocol_conformances  0000000000000000              .swift2_protocol_conformances
00000000008899e0 l    d  .swift2_type_metadata  0000000000000000              .swift2_type_metadata
0000000000886a80 l     O .swift2_protocol_conformances  0000000000002f60              l_protocol_conformances
00000000008899e8 l     O .swift2_type_metadata  0000000000000288              l_type_metadata_table
0000000000889c70 l     O .swift2_type_metadata  0000000000000010              l_type_metadata_table
0000000000889c80 g       .swift2_type_metadata  0000000000000000              _edata
00000000008899e0 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_start
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_assocty_section
00000000008899e0 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_end
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_fieldmd_section
0000000000886a78 g     O .swift2_protocol_conformances  0000000000000000              .hidden __TMC_END__
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_reflstr_section
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_typeref_section
0000000000889c80 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_end
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_start

[drewcrawford]

For comparison, here is the objdump for a dynamically-linked (working) program:

objdump -t test2 | grep \.swift._
0000000000400f30 l    d  .swift1_autolink_entries   0000000000000000              .swift1_autolink_entries
0000000000601378 l    d  .swift2_protocol_conformances  0000000000000000              .swift2_protocol_conformances
0000000000601380 l    d  .swift2_type_metadata  0000000000000000              .swift2_type_metadata
0000000000601388 g       .swift2_type_metadata  0000000000000000              _edata
0000000000601380 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_start
0000000000601378 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_assocty_section
0000000000601380 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_end
0000000000601378 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_fieldmd_section
0000000000601378 g     O .swift2_protocol_conformances  0000000000000000              .hidden __TMC_END__
0000000000601378 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_reflstr_section
0000000000601378 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_typeref_section
0000000000601388 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_end
0000000000601378 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_start

[hpux735]

Very strange. There is a clue, but I'm not sure how to interpret it... I notice that the order of the sections is wrong in both examples. On my system, they come up in order (start before end), but the address of them indicates that they're ordered correctly. This could be a red herring, but I think it's strange none the less.

[drewcrawford]

wczekalski (JIRA User) Can you post the exact test program you're using?

[drewcrawford]

@hpux735 Objdump from a program built with swift-DEVELOPMENT-SNAPSHOT-2016-03-01-a (e.g. definitely not something I broke):

objdump -t test | grep \.swift._
0000000000400f08 l    d  .swift1_autolink_entries   0000000000000000              .swift1_autolink_entries
0000000000601350 l    d  .swift2_protocol_conformances  0000000000000000              .swift2_protocol_conformances
0000000000601358 l    d  .swift2_type_metadata  0000000000000000              .swift2_type_metadata
0000000000601360 g       .swift2_type_metadata  0000000000000000              _edata
0000000000601358 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_start
0000000000601350 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_assocty_section
0000000000601358 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_end
0000000000601350 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_fieldmd_section
0000000000601350 g     O .swift2_protocol_conformances  0000000000000000              .hidden __TMC_END__
0000000000601350 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_reflstr_section
0000000000601350 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_typeref_section
0000000000601360 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_end
0000000000601350 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_start

Let's see if we can put this minor discrepancy to rest. I wonder if maybe this discrepancy is due to some difference in objdump? Here is the actual binary in question: test. Can you objdump and compare the output and see if your output is inverted as well? Can you also confirm the swift version where you're seeing the non-inverted behavior?

This is probably a dead end, but it's the only lead I have.

[drewcrawford]

Update: I found some mystery libraries libsection_magic_begin.a and libsection_magic_end.a which "seem" similar to swift_begin.o and swift_end.o, perhaps for the static case. However, no futzing around with those files seem to have any effect on the print behavior 🙁

[hpux735]

libsection_magic_begin.a and libsection_magic_end.a should only exist during compilation of the standard library. They are identical (or should be) to the swift_begin.o and swift_end.o files. We had to make those weird object files to deal with CMake's agressive reordering of .o's during linking the stdlib.

[drewcrawford]

libsection_magic_begin.a etc. are actually distributed with Swift; if you download the snapshot tarball from swift.org they are in there.

Regardless, I will ignore them if they aren't relevant, as slimming the install is not a goal of this bug :-)

Unfortunately I am out of leads here.

[hpux735]

Interesting, it's not distributed in the Linux buildbot tarballs.

I guess one thing to note is it sounds like you're using MacOS, and I don't have (perhaps ironically) as much experience with open source swift in MacOS.

[drewcrawford]

It definitely is in the Linux buildbot tarballs. 100%. Go download https://swift.org/builds/development/ubuntu1510/swift-DEVELOPMENT-SNAPSHOT-2016-03-01-a/swift-DEVELOPMENT-SNAPSHOT-2016-03-01-a-ubuntu15.10.tar.gz and check. It is in /usr/lib/swift_static/linux.

I actually run both Linux/OSX pretty heavily, my immediate interest is trying to statically link on Linux, as I have already sketched out a working solution for OSX (although there may be more than meets the eye, wczekalski (JIRA User) suggests the OSX solution doesn't work for him.)

[swift-ci]

Comment by Wojtek Czekalski (JIRA)

@drewcrawford The OS X solution does work, however it also links to dylibs.

The program I am trying to compile is simple print("hello world"). When I compile it using the command, it does link statically to libraries but it also links dynamically to the following (otool -L output):

    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1256.14.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
    /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1256.1.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

Well, I'm no expert in the build process (I got engaged in this because would love to dive deeper into it) but it looks like things are indeed linked dynamically. Please correct me if I'm wrong.

[hpux735]

Weird. I didn't know about swift_static. Well, good luck!

[drewcrawford]

cezarywojcik (JIRA User) You are correct but I think that's expected behavior, or at least outside the scope of the present bug. If you aren't linking libswiftCore.dylib you have dynamically linked the Swift standard library.

Your otool output shows you are dynamically linking CoreFoundation, Foundation, libobjc and libSystem, all of which ship with OSX and should be present on any OSX install, and your binary should be portable to any El Cap system (certainly report if this isn't true). We should probably support the static linking of corelibs Foundation, but I don't think corelibs is even supported dynamically on OSX yet. And IMO static Foundation should be its own bug.

[drewcrawford]

Sadly I am totally out of ideas on why print is printing String(_core: Swift._StringCore(_baseAddress: Swift.OpaquePointer(_rawValue: (Opaque Value)), _countAndFlags: Swift.UInt(_value: (Opaque Value)), _owner: Swift.Optional<Swift.AnyObject>.none)) instead of a string. I am now desperate enough to ping the folks who wrote print to see if they have any idea why this might occur: @gribozavr @dabrahams

[gribozavr]

Likely print() is not working because conditional casts to protocols don't work because protocol conformance tables can't be found.

[swift-ci]

Comment by Wojtek Czekalski (JIRA)

@drewcrawford You are correct. Thanks for clarifying. Looks like I misunderstood the issue (which is btw clearly stated by the title of it). Yeah, it's definitely out of scope.

[drewcrawford]

I'm still at a dead-end here. For folks following along at home:

• swift-dev post, no replies

• I am pretty confident, based on playing around with old snapshots, that linux static linking has never worked. Asking around suggests I'm the first one to try it, and the old snapshots fail similarly to the current failure (although there are minor differences in behavior with older snapshots).

• I've been not-very-subtly begging folks who "might know" off-list, and I've gathered some great suggestions but all of them are now dead ends.

• I dunno how to go any further.

🙁

[drewcrawford]

@hpux735 Here's an oddball idea. I am pretty sure the section of size 2f60 above is the one we want. This is because it's the largest (plausibly the stdlib) and there's a table of that size in libswiftCore.a.

I notice, however, that this entry appears definitely outside the start/end markers (this is besides the observation of my markers being reversed). Could an entry appearing outside those markers be related?

[drewcrawford]

Actually, I think I'm just misreading it. When I pipe the output to sort, start/end are in order and the large table is between the markers again. I think objdump is just unsorted, which creates the confusion.

00000000006215f0 l    d  .swift1_autolink_entries   0000000000000000              .swift1_autolink_entries
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_start
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_assocty_section
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_fieldmd_section
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_reflstr_section
0000000000886a78 g       .swift2_protocol_conformances  0000000000000000              .protected .swift3_typeref_section
0000000000886a78 g     O .swift2_protocol_conformances  0000000000000000              .hidden __TMC_END__
0000000000886a78 l    d  .swift2_protocol_conformances  0000000000000000              .swift2_protocol_conformances
0000000000886a80 l     O .swift2_protocol_conformances  0000000000002f60              l_protocol_conformances
00000000008899e0 g       .swift2_protocol_conformances  0000000000000000              .protected .swift2_protocol_conformances_end
00000000008899e0 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_start
00000000008899e0 l    d  .swift2_type_metadata  0000000000000000              .swift2_type_metadata
00000000008899e8 l     O .swift2_type_metadata  0000000000000288              l_type_metadata_table
0000000000889c70 l     O .swift2_type_metadata  0000000000000010              l_type_metadata_table
0000000000889c80 g       .swift2_type_metadata  0000000000000000              .protected .swift2_type_metadata_end
0000000000889c80 g       .swift2_type_metadata  0000000000000000              _edata

[hpux735]

That confused me several times. I had to ask someone about it. The output is in file order, not address order.

[drewcrawford]

Okay, I found the problem! Open source FTW. But I need help figuring out the right solution. Pinging @gparker42 tinysun (JIRA User), who wrote the code in question.

In ProtocolConformances.cpp we look up the protocol conformance table with dlsym. However, we aren't finding the section because dlsym looks in the dynamic symbol table, not the global symbol table, and in a statically linked executable the section is inside the latter.

Here are some suggestions for solving this, which one do we pick?

• Link with -rdynamic to promote all symbols to the dynamic symbol table. This feels heavyhanded to me.

• We could use --dynamic-list link option to export this symbol particularly? That would require us to emit a random file as part of the build process I think?

• Maybe there is a way to grab this symbol out of the global table without dlsym? I'm not sure how, I'm not much of an ELF expert.

Appreciate any direction.

[drewcrawford]

Discussions offline suggest door #2. It's time I self-assigned this issue, I think :-)

[drewcrawford]

I've sent up the first PR (of several that will be required) on this, 1806. Review appreciated.

Will likely start on the next PR tomorrow.

[drewcrawford]

Second PR is up 1817. This should resolve this bug for the Darwin target family. Review appreciated.

[drewcrawford]

PR wars episode 3: return of the ELF.

We now have Linux support, completing (AFAIK) the trilogy of PRs on this topic, and resolving the underlying bug.

It's now a question of getting the changes reviewed.

[ddunbar]

Pinged PRs, would be nice to see these land so SwiftPM static lib support is unblocked.

[ddunbar]

All the PRs above have landed, @drewcrawford is this done?

[drewcrawford]

Done! Closing

[ddunbar]

Awesome, thanks!!

[swift-ci]

Comment by Юсипов Тимур (JIRA)

@drewcrawford, I'm stil a bit confused. Can I statically link libswift*.dylib to my iOS application?

I'm trying to optimize my app's start time and think this could introduce some benefit

[swift-ci]

Comment by Юсипов Тимур (JIRA)

@drewcrawford, I accidentally forgot to mention you in the above comment

[swift-ci]

Comment by Юсипов Тимур (JIRA)

Ups =)
I meant @ddunbar

Is it possible to link libswift*.dylib to an iOS application?