[SR-6004] Static libraries don't automatically load extensions

[swift-ci]

Previous ID	SR-6004
Radar	rdar://problem/34686622
Original Reporter	soffes (JIRA User)
Type	Bug
Status	Resolved
Resolution	Done

Attachment: Download

Environment

Xcode 9.0 (9A235)

Additional Detail from JIRA

Votes	7
Component/s	Compiler
Labels	Bug
Assignee	None
Priority	Medium

md5: a88822ddd333e2719ad59655283f788c

is duplicated by:

• SR-10507 Dynamic cast to existential fails when conformance is added in extension in static library

Issue Description:

I'm not sure if this is the intended behavior, but a static library doesn't load extensions unless something else in the same file is explicitly referenced. I realize adding `all_load` to the target it is linked into solves this. I'd love to avoid that if possible.

This is more of a question of is this the indeed behavior and if so is doing `all_load` the intended use?

I've attached a sample project reproducing this. Here's an explanation:

1. Create a static library A containing a protocol and some types that conform to it

2. Create another static library B that creates some another protocol and adds extensions to types in static library A

3. In another target that links both A & B, you can observe types from A that conform to a protocol from B not doing so at runtime.

4. If you explicitly reference something in a file that adds conformation, it will load the extension with the conformance and work as expected.

The attached sample project may make this more clear. Start in ViewController.swift and check out the comments.

[belkadan]

Heh, we have -ObjC to solve this for categories, but that's not the same thing. @jckarter, @gparker42, thoughts?

[gparker42]

You can use -force_load to get load-everything semantics for a single archive, which is better than -all_load.

-ObjC might just work. The linker will pull in the ObjC category representing the extension, and that might be enough to pull in the rest.

[belkadan]

Not all Swift extensions have corresponding categories, and the protocol conformance info probably isn't attached. I suppose we could start vending dummy categories…

[gparker42]

If I recall correctly there is some symbol attribute that the compiler can add which tells the linker to pull in that definition unconditionally. Let me look around.

[keith]

There definitely is, at some point I was actually trying to debug an issue with something like that, you can see when they're marked with that attribute using `nm -m`. Example:

% nm -m ./Library.o
...
00000000000002b8 (__DATA,__const) non-external [no dead strip] __TMfV7Library3Foo
...

[gparker42]

I talked to the linker guys and there is not. Symbols can be marked no-dead-strip, but the lazy-loading semantics of static archives do not honor that. We may need to get linker enhancements if we need a fix for Swift there.

The usual solution is to avoid static archives. Instead, use ld -r to combine your .o files into one big .o file, then dead-strip that. That will remove some of the unused code that you get from static archive linking while also honoring no-dead-strip symbols.

[belkadan]

@swift-ci create

[swift-ci]

Comment by Michael Bachand (JIRA)

We are experiencing this issue at Airbnb with Buck-generated Xcode projects that use Swift static libraries. We plan to use @objc (paired with the -ObjC "Other Linker Flag") to work around it.

[swift-ci]

Comment by Michael Bachand (JIRA)

Here is a repro in a Buck-generated project: airbnb/BuckSample#38

[keith]

@belkadan @gparker42 any idea on how we can move forward to a fix for this one? I'd be willing to take a stab at some implementation for this if possible

[AnnaZaks]

soffes (JIRA User), Could you verify if the problem is fixed in the latest Xcode betas and if so move the JIRA to "Closed"?

Thanks!
Anna

[keith]

Did something change in Swift that should have fixed this?

[brentleyjones]

In Swift 5.1, per the Xcode release notes, this is now fixed for us. We use -ObjC though, so that might be it.

[AnnaZaks]

There was a linker side change to address this.

[keith]

Ah interesting, I see this in the release notes "Static libraries are now always force-loaded in their entirety during linking, fixing runtime errors related to demangling failures. (47598583)"

On the surface this sounds pretty scary! Since we're intentionally not force loading all of our libraries, only the ones that experience this problem. But I can try to see if it fixed our case as well