Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-11362] Include bitcode in Xcode toolchain libraries #53763

Open
dmcyk opened this issue Aug 23, 2019 · 12 comments
Open

[SR-11362] Include bitcode in Xcode toolchain libraries #53763

dmcyk opened this issue Aug 23, 2019 · 12 comments

Comments

@dmcyk
Copy link
Contributor

@dmcyk dmcyk commented Aug 23, 2019

Previous ID SR-11362
Radar rdar://problem/54636425
Original Reporter @dmcyk
Type Improvement
Additional Detail from JIRA
Votes 0
Component/s
Labels Improvement, Runtime
Assignee None
Priority Medium

md5: 728f539cd14772a5a86b33c3d613fa52

Issue Description:

Env:
macOS 10.14.6
Xcode 11 beta 6

Xcode toolchain recently added some new Swift 5.0 compatibility libraries (noticed in Xcode 11 beta 6 as well as open source toolchain):

  • libswiftCompatibility50.a

  • libswiftCompatibilityDynamicReplacements.a

Those libraries are included for all platforms, not only macOS, what is their goal? Is it to provide compatibility for e.g. dynamic replacements on older OSes?

This is problematic for testing bitcode on older macOS releases than 10.15 because Swift compiler apparently tries to always link those libraries, even when Swift 5.1 features are not used.
Running `swiftc -embed-bitcode empty.swift` with empty Swift source files fails to include bitcode because the listed libraries do not have it.

ld: warning: all bitcode will be dropped because '/Applications/Xcode_11.0_b6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftCompatibility50.a(ProtocolConformance.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Applications/Xcode_11.0_b6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftCompatibility50.a(Overrides.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Applications/Xcode_11.0_b6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftCompatibilityDynamicReplacements.a(DynamicReplaceable.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.

@belkadan
Copy link
Contributor

@belkadan belkadan commented Aug 23, 2019

cc @jckarter, aschwaighofer@apple.com (JIRA User), @mikeash. Oops.

@mikeash
Copy link
Contributor

@mikeash mikeash commented Aug 23, 2019

To answer the questions: libswiftCompatibilityDynamicReplacements.a does indeed exist to support dynamic replacements on older OSes. It should be inert if you're not using dynamic replacements (other than this serious problem you're pointing out...) and I think dead code stripping will mean the code doesn't even get included in the final binary in that case. libswiftCompatibility50.a contains a couple of important fixes for runtime bugs and uses the compatibility override mechanism in the runtime (https://github.com/apple/swift/blob/master/stdlib/public/runtime/CompatibilityOverride.cpp) to get those fixes into the 5.0 runtime.

Also: yes, oops.

@jckarter
Copy link
Member

@jckarter jckarter commented Aug 23, 2019

The libraries provide backward compatibility for new runtime features and also backport fixes for known runtime bugs to older OSes. They ought to be built with bitcode for platforms that support bitcode. Since bitcode isn't used for anything on macOS, it looks like we don't build bitcode slices for anything in the toolchain for macOS, not only these archives (though by their nature, archives might be the first thing that show there's a problem). `libarclite` for instance also does not include bitcode for macOS, so you'd have the same problem building bitcode when targeting macOS <10.10.

@jckarter
Copy link
Member

@jckarter jckarter commented Aug 23, 2019

You can disable linking these libraries by passing -runtime-compatibility-version none to the compiler.

@aschwaighofer
Copy link
Member

@aschwaighofer aschwaighofer commented Aug 23, 2019

And -Xfrontend -disable-autolinking-runtime-compatibility-dynamic-replacements to disable linking libswiftCompatibilityDynamicReplacements.a.

@belkadan
Copy link
Contributor

@belkadan belkadan commented Aug 23, 2019

-runtime-compatibility-version should cover both libraries.

@dmcyk
Copy link
Contributor Author

@dmcyk dmcyk commented Aug 23, 2019

Thanks for the explanation, I appreciate quick feedback!
Indeed iphoneos and others do include bitcode. Guess it makes sense that those libraries do not have bitcode, though for testing and experimentation purposes it'd be convenient if they did. Adding bitcode there is something that could be considered?

With -runtime-compatibility-version only I'm still getting missing bitcode warning for the libswiftCompatibilityDynamicReplacements.a, adding also the flag aschwaighofer@apple.com (JIRA User) mentioned solved it.

@swift-ci
Copy link
Collaborator

@swift-ci swift-ci commented Jun 8, 2021

Comment by Alex LIU (JIRA)

Hi guys, I think we are currently facing the similar situation.

We have a framework project which was written in objc but we are trying to write new features with Swift. We also configured correctly so that we can call objc function from Swift and vice versa but the same problem happens when we build it with `xcodebuild build` command. (due to CI/CD requirement we have to use command line to build a framework)

ld: warning: all bitcode will be dropped because '/tmp/DerivedData/xxxxxxxx.build/Objects-normal/x86_64/NewSwiftTestFeature.o' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility50.a(Overrides.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility51.a(Overrides.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibilityDynamicReplacements.a(DynamicReplaceable.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility50.a(ProtocolConformance.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility51.a(ProtocolConformance.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.

The issue is for this x86_64 arch we do need bit code enabled in this build however when we check it `otool` the framework doesn't have bitcode enabled. I guess due to the warning above at least. How should I make bitcode enabled or how can I use "-runtime-compatibility-version" argument along with `xcodebuild` tool? Please help, thanks.

@mikeash
Copy link
Contributor

@mikeash mikeash commented Jun 8, 2021

You can add the flags to Other Swift Flags in your Xcode project, and then xcodebuild will pick them up. Or you can specify them in the command like, xcodebuild OTHER_SWIFT_FLAGS='-runtime-compatibility-version none'.

@swift-ci
Copy link
Collaborator

@swift-ci swift-ci commented Jun 8, 2021

Comment by Alex LIU (JIRA)

Thank you so much for your quick reply, @mikeash

It worked since I don't see those warnings about swift Lib compatibility anymore except one...

I have a file called `NewSwiftTestFeature` which is the only Swift file I added into the entire objc project, however i'm still getting this erorr which will make my output binary with disabled bitcode. Any configuration I should change in order to make bitcode enabled?

ld: warning: all bitcode will be dropped because '/tmp/DerivedData/xxxx-DerivedData/Build/Intermediates.noindex/xxxx.build/Release-iphonesimulator/xxxx.build/Objects-normal/arm64/NewSwiftTestFeature.o' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.

Here's my xcodebuild command:

xcodebuild build BITCODE_GENERATION_MODE=bitcode \    
-configuration "$CONFIGURATION" \    
-derivedDataPath "${DERIVEDDATA_PATH}" \    
-scheme "$SCHEME" \    
-sdk iphonesimulator \    
-destination "generic/platform=iOS Simulator" \    
BUILD_LIBRARY_FOR_DISTRIBUTION=YES \    
SKIP_INSTALL=NO \    
PRODUCT_NAME="xxx" \    
EXECUTABLE_PREFIX=""  \    
OTHER_SWIFT_FLAGS='-runtime-compatibility-version none'

@cachemeifyoucan
Copy link
Contributor

@cachemeifyoucan cachemeifyoucan commented Jun 9, 2021

Here are few things about bitcode:

  • Bitcode is not support on macOS and simulators* (more about simulator below)

  • Built without bitcode only turns into an error during archive build option (and only emitted during that build action to save build time and disk space)

  • When building during development, either for device or for simulator, linker will only do a quick sanity check to surface any build configuration issues about bitcode that might turn into an error during archive time. That is the warning you see here.

In your situation, you can safely ignore those warnings. The compatibility libraries are built with bitcode for iphoneos/watchOS/tvOS so you won't hit that problem during archive build for submission. On the other hand, feel free to file a bug report to feedbackassistant.apple.com or using the feedback app for suggestion to improve with spurious warnings.

@swift-ci
Copy link
Collaborator

@swift-ci swift-ci commented Dec 10, 2021

Comment by Sarah Koop (JIRA)

We are also experiencing this issue when building with xcframeworks on an M1 Mac. Bitcode is enabled for the modules within the SDK that we are testing with. We are not able to replicate this issue on an Intel Mac.

By default, Xcode sets ENABLE_BITCODE to YES. We are able to consistently reproduce this issue, if we toggle this setting to NO and back to YES.

Steps to reproduce:

  1. Clone the braintree_ios repo and build the Demo app
  2. Toggle Demo Xcode project ENABLE_BITCODE Build Setting to NO
  3. Build the project
  4. Toggle back to YES
  5. Build the project
  6. See warnings similar to the ones posted above

When we ran the sample app on a device, these warnings/errors went away. We were also able to archive successfully.
There appears to be an Apple Developer Forum Issue] about this also.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants