build: migrate swift to CMake 3.15.1 #2524

CMake Error at Tools/plutil/cmake_install.cmake:41 (file):
  file INSTALL cannot find
  "/Users/norio/github/swift-dev/swift-corelibs-foundation/Tools/plutil/plutil".
Call Stack (most recent call first):
  Tools/cmake_install.cmake:42 (include)
  cmake_install.cmake:52 (include)


FAILED: CMakeFiles/install.util 
cd /Users/norio/github/swift-dev/build/buildbot_linux/foundation-linux-x86_64 && /Users/norio/github/swift-dev/build/cmake-linux-x86_64/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.
swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting
swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

I guess that is not reproduced with the test triggered by #2524 (comment)

compnerd · 2019-10-24T15:24:34Z

Ugh, seems that PROGRAMS uses the FILES path, not the TARGETS path, which is probably why it fails. Should be trivial to fix... changing PROGRAMS to TARGETS.

compnerd · 2019-10-24T15:51:03Z

@swift-ci please test

spevans · 2019-10-24T19:36:19Z

@swift-ci please test

compnerd · 2019-10-25T00:51:37Z

@swift-ci please test

norio-nomura · 2019-10-25T03:13:08Z

building toolchain package with updated commit fails with following error:

CMake Error at cmake_install.cmake:93 (file):
  file INSTALL cannot find
  "/Users/norio/github/swift-dev/build/buildbot_linux/foundation-linux-x86_64/plutil".


FAILED: CMakeFiles/install.util 
cd /Users/norio/github/swift-dev/build/buildbot_linux/foundation-linux-x86_64 && /Users/norio/github/swift-dev/build/cmake-linux-x86_64/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.
swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting
swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

plutil seems to be created at /Users/norio/github/swift-dev/build/buildbot_linux/foundation-linux-x86_64/Tools/plutil/plutil.

compnerd · 2019-10-25T03:25:58Z

That is the correct location; I wonder why its looking for plutil in the wrong location. Did you do a clean build @norio-nomura?

norio-nomura · 2019-10-25T03:29:27Z

Did you do a clean build @norio-nomura?

No.
I'll try again with removing foundation-linux-x86_64.

norio-nomura · 2019-10-25T04:28:47Z

I'll try again with removing foundation-linux-x86_64.

I got same result.
regenerated build/buildbot_linux/foundation-linux-x86_64/cmake_install.cmake has entry:

file(INSTALL DESTINATION "/usr/bin" TYPE PROGRAM FILES "/Users/norio/github/swift-dev/build/buildbot_linux/foundation-linux-x86_64/plutil")

but, plutil is placed at:

…/build/buildbot_linux/foundation-linux-x86_64/Tools/plutil/plutil

instead of

…/build/buildbot_linux/foundation-linux-x86_64/plutil

that cmake_install.cmake expects.

compnerd · 2019-10-25T17:38:30Z

Something seems cached there, this is what is getting generated for me:

if("x${CMAKE_INSTALL_COMPONENT}x" STREQUAL "xUnspecifiedx" OR NOT CMAKE_INSTALL_COMPONENT)
  file(INSTALL DESTINATION "${CMAKE_INSTALL_PREFIX}/bin" TYPE EXECUTABLE FILES "S:/b/foundation/Tools/plutil/plutil.exe")
  if(EXISTS "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}/bin/plutil.exe" AND
     NOT IS_SYMLINK "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}/bin/plutil.exe")
    if(CMAKE_INSTALL_DO_STRIP)
      execute_process(COMMAND "S:/b/toolchain/bin/strip.exe" "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}/bin/plutil.exe")
    endif()
  endif()
endif()

norio-nomura · 2019-10-27T00:07:42Z

building toolchain package with updated commit fails with following error:

+ /Users/norio/github/swift-dev/indexstore-db/Utilities/build-script-helper.py build --verbose --package-path /Users/norio/github/swift-dev/indexstore-db --build-path /Users/norio/github/swift-dev/build/buildbot_linux/indexstoredb-linux-x86_64 --configuration release --toolchain /Users/norio/github/swift-dev/build/swift-nightly-install --ninja-bin /Users/norio/github/swift-dev/build/buildbot_linux/ninja-build/ninja
/Users/norio/github/swift-dev/build/swift-nightly-install/usr/bin/swift build --package-path /Users/norio/github/swift-dev/indexstore-db --build-path /Users/norio/github/swift-dev/build/buildbot_linux/indexstoredb-linux-x86_64 --configuration release --verbose -Xcxx -I -Xcxx /Users/norio/github/swift-dev/build/swift-nightly-install/usr/lib/swift -Xcxx -I -Xcxx /Users/norio/github/swift-dev/build/swift-nightly-install/usr/lib/swift/Block
lsb_release -r
/Users/norio/github/swift-dev/build/swift-nightly-install/usr/bin/swiftc --driver-mode=swift -L /Users/norio/github/swift-dev/build/swift-nightly-install/usr/lib/swift/pm/4_2 -lPackageDescription -swift-version 4.2 -I /Users/norio/github/swift-dev/build/swift-nightly-install/usr/lib/swift/pm/4_2 -sdk / -package-description-version 4.2 /Users/norio/github/swift-dev/indexstore-db/Package.swift -fileno 7
/Users/norio/github/swift-dev/indexstore-db: error: manifest parse error(s):
/Users/norio/github/swift-dev/indexstore-db/Package.swift:3:8: error: missing required module 'Foundation'
import PackageDescription
       ^
Traceback (most recent call last):
  File "/Users/norio/github/swift-dev/indexstore-db/Utilities/build-script-helper.py", line 94, in <module>
    main()
  File "/Users/norio/github/swift-dev/indexstore-db/Utilities/build-script-helper.py", line 83, in main
    swiftpm('build', swift_exec, swiftpm_args, env)
  File "/Users/norio/github/swift-dev/indexstore-db/Utilities/build-script-helper.py", line 15, in swiftpm
    subprocess.check_call(cmd, env=env)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/norio/github/swift-dev/build/swift-nightly-install/usr/bin/swift', 'build', '--package-path', '/Users/norio/github/swift-dev/indexstore-db', '--build-path', '/Users/norio/github/swift-dev/build/buildbot_linux/indexstoredb-linux-x86_64', '--configuration', 'release', '--verbose', '-Xcxx', '-I', '-Xcxx', '/Users/norio/github/swift-dev/build/swift-nightly-install/usr/lib/swift', '-Xcxx', '-I', '-Xcxx', '/Users/norio/github/swift-dev/build/swift-nightly-install/usr/lib/swift/Block']' returned non-zero exit status 1
swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting
swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

It seems to fail while building indexstore-db with packaging toolchain at build/swift-nightly-install.
It looks that this PR changes locations where Foundation related files are installed.

$ docker-swiftnightly-run find /usr/lib/ -name "*Foundation*"|sort>nightly20191024.txt
$ (cd build/swift-nightly-install; find . -name "*Foundation*"|sed "s/^\.//"|sort)>pr-2524.txt
$ diff -u nightly20191024.txt pr-2524.txt
--- nightly20191024.txt	2019-10-27 09:03:51.000000000 +0900
+++ pr-2524.txt	2019-10-27 09:03:57.000000000 +0900
@@ -7,17 +7,17 @@
 /usr/lib/swift/CoreFoundation/CoreFoundation.h
 /usr/lib/swift/CoreFoundation/ForFoundationOnly.h
 /usr/lib/swift/CoreFoundation/ForSwiftFoundationOnly.h
-/usr/lib/swift/linux/libFoundation.so
-/usr/lib/swift/linux/libFoundationNetworking.so
-/usr/lib/swift/linux/libFoundationXML.so
-/usr/lib/swift/linux/x86_64/Foundation.swiftdoc
-/usr/lib/swift/linux/x86_64/Foundation.swiftmodule
-/usr/lib/swift/linux/x86_64/FoundationNetworking.swiftdoc
-/usr/lib/swift/linux/x86_64/FoundationNetworking.swiftmodule
-/usr/lib/swift/linux/x86_64/FoundationXML.swiftdoc
-/usr/lib/swift/linux/x86_64/FoundationXML.swiftmodule
 /usr/lib/swift/shims/CoreFoundationOverlayShims.h
 /usr/lib/swift/shims/CoreFoundationShims.h
 /usr/lib/swift/shims/FoundationOverlayShims.h
 /usr/lib/swift/shims/FoundationShimSupport.h
 /usr/lib/swift/shims/FoundationShims.h
+/usr/lib/x86_64-linux-gnu/libFoundation.so
+/usr/lib/x86_64-linux-gnu/libFoundationNetworking.so
+/usr/lib/x86_64-linux-gnu/libFoundationXML.so
+/usr/lib/x86_64-linux-gnu/swift/linux/x86_64/Foundation.swiftdoc
+/usr/lib/x86_64-linux-gnu/swift/linux/x86_64/Foundation.swiftmodule
+/usr/lib/x86_64-linux-gnu/swift/linux/x86_64/FoundationNetworking.swiftdoc
+/usr/lib/x86_64-linux-gnu/swift/linux/x86_64/FoundationNetworking.swiftmodule
+/usr/lib/x86_64-linux-gnu/swift/linux/x86_64/FoundationXML.swiftdoc
+/usr/lib/x86_64-linux-gnu/swift/linux/x86_64/FoundationXML.swiftmodule

compnerd · 2019-10-27T00:39:17Z

It seems that CMAKE_INSTALL_LIBDIR is different now? Is it being specified explicitly? It should be lib by default. which would give the same path as previously.

compnerd · 2019-10-29T22:13:04Z

@swift-ci please test

compnerd · 2019-10-30T00:15:01Z

@swift-ci please test

compnerd · 2019-10-30T01:47:30Z

@swift-ci please test

compnerd · 2019-10-30T20:44:00Z

@swift-ci please test

compnerd · 2019-10-30T23:00:13Z

@swift-ci please test

compnerd · 2019-10-31T01:10:29Z

@swift-ci please test Linux platform

compnerd · 2019-11-04T22:26:29Z

@swift-ci please test

compnerd · 2019-11-05T00:47:22Z

@swift-ci please test

compnerd · 2019-11-05T01:45:31Z

@swift-ci please test Linux platform

compnerd · 2019-11-05T03:46:41Z

Please test with following PR:
apple/swift-corelibs-xctest#287

@swift-ci please test Linux platform

shahmishal · 2019-11-05T08:37:17Z

Please test with following PR:
apple/swift-corelibs-xctest#287

@swift-ci please test Linux platform

compnerd · 2019-11-05T15:45:39Z

Please test with following PR:
apple/swift-corelibs-xctest#287

@swift-ci please test Linux platform

compnerd · 2019-11-05T16:29:04Z

@swift-ci please test macOS platform

This migrates Foundation to CMake 3.15.1 and uses the new export targets from libdispatch to enable simpler builds with better dependency tracking.

Ensure that we export the Foundation libraries, plutil, and uuid for users of Foundation against the build tree.

compnerd · 2019-11-05T18:02:41Z

@swift-ci please test

compnerd force-pushed the cmake-3.15.1 branch 2 times, most recently from e6c66cb to 678690e Compare September 28, 2019 00:11

compnerd force-pushed the cmake-3.15.1 branch from 678690e to e4cadc7 Compare September 30, 2019 17:17

compnerd mentioned this pull request Oct 15, 2019

build: add CMake based build jpsim/Yams#211

Merged

compnerd force-pushed the cmake-3.15.1 branch from e4cadc7 to 9fdb9eb Compare October 24, 2019 15:50

compnerd force-pushed the cmake-3.15.1 branch from 9fdb9eb to a3360e4 Compare October 24, 2019 21:06

compnerd force-pushed the cmake-3.15.1 branch from a3360e4 to 1864067 Compare October 26, 2019 19:47

compnerd force-pushed the cmake-3.15.1 branch from 1864067 to 62e1d61 Compare October 30, 2019 00:14

compnerd force-pushed the cmake-3.15.1 branch 2 times, most recently from 25c849b to 99ddef3 Compare October 30, 2019 20:43

compnerd force-pushed the cmake-3.15.1 branch from 99ddef3 to 2c6d20f Compare October 30, 2019 22:57

compnerd force-pushed the cmake-3.15.1 branch from 2c6d20f to 8787c66 Compare November 4, 2019 21:54

compnerd force-pushed the cmake-3.15.1 branch from 8787c66 to e98ba47 Compare November 5, 2019 00:44

compnerd added 2 commits November 5, 2019 09:55

build: migrate swift to CMake 3.15.1

5025ecf

This migrates Foundation to CMake 3.15.1 and uses the new export targets from libdispatch to enable simpler builds with better dependency tracking.

build: add export targets for the appropriate components

388e84a

Ensure that we export the Foundation libraries, plutil, and uuid for users of Foundation against the build tree.

compnerd force-pushed the cmake-3.15.1 branch from e98ba47 to 388e84a Compare November 5, 2019 17:55

compnerd mentioned this pull request Nov 5, 2019

build: port to new Swift support #2304

Closed

compnerd merged commit cb3925a into apple:master Nov 5, 2019

compnerd deleted the cmake-3.15.1 branch November 5, 2019 21:05

spevans mentioned this pull request Nov 8, 2019

[SR-9251] Cannot run code that imports Foundation on Linux with latest toolchain: error while loading shared libraries: libicuucswift.so.61: cannot open shared object file: No such file or directory #3591

Closed

build: migrate swift to CMake 3.15.1 #2524

build: migrate swift to CMake 3.15.1 #2524

Conversation

compnerd commented Sep 28, 2019

compnerd commented Sep 29, 2019

compnerd commented Sep 29, 2019

compnerd commented Sep 30, 2019

compnerd commented Sep 30, 2019

compnerd commented Sep 30, 2019

compnerd commented Oct 1, 2019

compnerd commented Oct 1, 2019

compnerd commented Oct 3, 2019

compnerd commented Oct 14, 2019

compnerd commented Oct 14, 2019

spevans commented Oct 14, 2019

spevans commented Oct 14, 2019

compnerd commented Oct 14, 2019

compnerd commented Oct 23, 2019

norio-nomura commented Oct 24, 2019

compnerd commented Oct 24, 2019

compnerd commented Oct 24, 2019

spevans commented Oct 24, 2019

compnerd commented Oct 25, 2019

norio-nomura commented Oct 25, 2019

compnerd commented Oct 25, 2019

norio-nomura commented Oct 25, 2019

norio-nomura commented Oct 25, 2019

compnerd commented Oct 25, 2019

norio-nomura commented Oct 27, 2019

compnerd commented Oct 27, 2019

compnerd commented Oct 29, 2019

compnerd commented Oct 30, 2019

compnerd commented Oct 30, 2019

compnerd commented Oct 30, 2019

compnerd commented Oct 30, 2019

compnerd commented Oct 31, 2019

compnerd commented Nov 4, 2019

compnerd commented Nov 5, 2019

compnerd commented Nov 5, 2019

compnerd commented Nov 5, 2019

shahmishal commented Nov 5, 2019

compnerd commented Nov 5, 2019

compnerd commented Nov 5, 2019

compnerd commented Nov 5, 2019