SR-6084: libdispatch.so missing from snapshot builds

[dgrove-oss]

Adjust install path for libdispatch in CMakelists.txt

[dgrove-oss]

Seems to fix https://bugs.swift.org/browse/SR-6084 for me (but I am not a buildbot ;) )

@jrose-apple @compnerd

[jrose-apple]

@shahmishal, how would you suggest testing that this fixes installable toolchains on Linux? (which currently don't get libdispatch copied into them).

[shahmishal]

apple/swift#12383

@swift-ci test

[jrose-apple]

Well, that still didn't test the installable package part, but it didn't make any of the existing parts worse. Do you think we should run that preset explicitly, or just merge this?

[dgrove-oss]

I'd suggest just to merge it and check the next nightly snapshot tar ball to be sure it is fixed.

I manually ran something closely derived from the linux-buildbot config and verified that libdispatch.so was in the expected plan in the produced tarball. So I really do expect this to fix the problem, but since I can't exactly reproduce the build machine environment, I could have missed something.

[jrose-apple]

All right, makes sense.

[jrose-apple]

Mishal, does it make sense to kick off a middle-of-the-day package build so we can see if it worked? Or should we just wait until tomorrow/Monday?

[shahmishal]

@dgrove-oss @compnerd We are still seeing test failure:

https://bugs.swift.org/browse/SR-6123

******************** TEST 'swift-package-tests :: swift-package-init-lib.md' FAILED ********************
Script:
--
rm -rf /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir
mkdir -p /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin/swift package --package-path /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project init --type library
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin/swift build --package-path /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project 2>&1 | tee /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.build-log
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin/swift test --package-path /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project 2>&1 | tee /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.test-log
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/llvm-linux-x86_64/bin/FileCheck --check-prefix CHECK-BUILD-LOG --input-file /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.build-log /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift-integration-tests/swift-package-init-lib.md
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/llvm-linux-x86_64/bin/FileCheck --check-prefix CHECK-TEST-LOG --input-file /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.test-log /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift-integration-tests/swift-package-init-lib.md
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/llvm-linux-x86_64/bin/FileCheck --check-prefix CHECK-NO-WARNINGS-OR-ERRORS --input-file /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.build-log /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift-integration-tests/swift-package-init-lib.md
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/llvm-linux-x86_64/bin/FileCheck --check-prefix CHECK-NO-WARNINGS-OR-ERRORS --input-file /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.test-log /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift-integration-tests/swift-package-init-lib.md
--
Exit Code: 1

Command Output (stdout):
--
$ "rm" "-rf" "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir"
$ "mkdir" "-p" "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project"
$ "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin/swift" "package" "--package-path" "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project" "init" "--type" "library"
# command output:
Creating library package: Project
Creating Package.swift
Creating README.md
Creating .gitignore
Creating Sources/
Creating Sources/Project/Project.swift
Creating Tests/
Creating Tests/LinuxMain.swift
Creating Tests/ProjectTests/
Creating Tests/ProjectTests/ProjectTests.swift

$ "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin/swift" "build" "--package-path" "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project"
$ "tee" "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.build-log"
# command output:
Compile Swift Module 'Project' (1 sources)

$ "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin/swift" "test" "--package-path" "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project"
note: command had no output on stdout or stderr
error: command failed with exit status: 1
$ "tee" "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.test-log"
# command output:
Compile Swift Module 'ProjectTests' (1 sources)
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "CoreFoundation.h"
         ^
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/lib/swift/CoreFoundation/CoreFoundation.h:77:10: note: in file included from /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/lib/swift/CoreFoundation/CoreFoundation.h:77:
#include <CoreFoundation/CFPropertyList.h>
         ^
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/lib/swift/CoreFoundation/CFPropertyList.h:17:10: note: in file included from /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/lib/swift/CoreFoundation/CFPropertyList.h:17:
#include <CoreFoundation/CFStream.h>
         ^
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/lib/swift/CoreFoundation/CFStream.h:20:10: error: 'dispatch/dispatch.h' file not found
#include <dispatch/dispatch.h>
         ^
<unknown>:0: error: could not build C module 'CoreFoundation'
error: terminated(1): /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin/swift-build-tool -f /home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/none-swift_package_sandbox_linux-x86_64/tests/Output/swift-package-init-lib.md.tmp.dir/Project/.build/debug.yaml test


--

********************

[compnerd]

@shahmishal I think that is a different issue. Looks like the Foundation module is failing to be built due to the headers missing. Can we get a verbose trace of that build? I suspect that the path being used for building the module is incorrect or that headers are being copied in a weird way. This would be easier to understand if the Foundation build system was also done similar to the rest of swift/libdispatch so that everything could be tied together with dependency tracking.

[shahmishal]

@compnerd Thanks I will keep bug open https://bugs.swift.org/browse/SR-6123.

What about this error?

https://ci.swift.org/job/swift-protobuf-linux-ubuntu-16_10/14/console

+ swift build -c release
latest_toolchain/swift-DEVELOPMENT-SNAPSHOT-2017-10-15-a-ubuntu16.10/usr/bin/swift-build: error while loading shared libraries: libdispatch.so: cannot open shared object file: No such file or directory

[compnerd]

@shahmishal , that error is pretty interesting. It implies that the DT_RUNPATH (or possibly DT_RPATH) is incorrect, or the embedded values are not correct. Would it be possible to get the output of readelf -Wa of the swift-build binary?

[jrose-apple]

Why would it mean that, rather than simply that libdispatch.so isn't being installed?

[compnerd]

@jrose-apple oh it could absolutely be that too. However, cmake will build binaries with a rpath for testing purposes and then adjust them at install time. So, that was the first thing which came to mind. The readelf output would show where it was looking for the libraries.