`swift test` linker issue on macOS

[dan-zheng]

Reproducer (macOS-only):

$ swift test
[84/84] Linking TensorFlowPackageTests
2019-08-26 16:25:01.853 xctest[37643:14366877] The bundle “TensorFlowPackageTests.xctest” couldn’t be loaded. Try reinstalling the bundle.
2019-08-26 16:25:01.853 xctest[37643:14366877] (dlopen(/Users/danielzheng/swift-apis/.build/x86_64-apple-macosx/debug/TensorFlowPackageTests.xctest/Contents/MacOS/TensorFlowPackageTests, 265): Symbol not found: _$s11AllKeyPathss0B12PathIterablePTl
  Referenced from: /Users/danielzheng/swift-apis/.build/x86_64-apple-macosx/debug/TensorFlowPackageTests.xctest/Contents/MacOS/TensorFlowPackageTests
  Expected in: /usr/lib/swift/libswiftCore.dylib
 in /Users/danielzheng/swift-apis/.build/x86_64-apple-macosx/debug/TensorFlowPackageTests.xctest/Contents/MacOS/TensorFlowPackageTests)

The key issue is /usr/lib/swift/libswiftCore.dylib. After ABI stability, the linker attempts to link the OS-vendored standard library instead of the one in the toolchain.

This issue is related to swiftlang/swift#25328. We need to revert the change that introduced linking /usr/lib/swift/libswiftCore.dylib.

[dan-zheng]

Some progress: I saw dlopen in the error message above, so I tried otool -l to print the rpaths of the opened library.

$ otool -l /Users/danielzheng/swift-apis/.build/x86_64-apple-macosx/debug/TensorFlowPackageTests.xctest/Contents/MacOS/TensorFlowPackageTests
Load command 31
          cmd LC_RPATH
      cmdsize 32
         path /usr/lib/swift (offset 12)
Load command 32
          cmd LC_RPATH
      cmdsize 120
         path /Library/Developer/Toolchains/swift-tensorflow-DEVELOPMENT-2019-08-25-a.xctoolchain/usr/lib/swift/macosx (offset 12)

Maybe the issue is that /usr/lib/swift appears before <TOOLCHAIN>/usr/lib/swift/macosx>.

[eaplatanios]

@dan-zheng have you tried manually changing the rpath in TensorFlowPackageTests using install_name_tool? I remember also seeing this error but haven't gotten a chance to look into it yet.

[dan-zheng]

The following didn't work for me:

$ install_name_tool -delete_rpath /usr/lib/swift /Users/danielzheng/swift-apis/.build/x86_64-apple-macosx/debug/TensorFlowPackageTests.xctest/Contents/MacOS/TensorFlowPackageTests

otool -l still shows /usr/lib/swift as the first rpath.

Maybe we should try another rpath modification tool. patchelf didn't work:

$ patchelf --print-rpath /Users/danielzheng/swift-apis/.build/x86_64-apple-macosx/debug/TensorFlowPackageTests.xctest/Contents/MacOS/TensorFlowPackageTests
patchelf: not an ELF executable

[dan-zheng]

I also grepped for /usr/lib/swift in the apple/swift and apple/swift-package-manager repositories but didn't find relevant code fo setting rpaths. (There's code for setting rpaths for stdlib/private libraries as seen in swiftlang/swift#25328 but I'm not sure that's relevant.)

I wasn't thorough, perhaps I overlooked something. 🙂

[rxwei]

Fixed (TF-797).