Revert "Merge pull request #40595 from hamishknight/straw-bales"

[aschwaighofer]

This reverts commit a67a043, reversing
changes made to 9965df7.

This commit or the earlier commit this commit is based on (#40531) broke the
incremental bot.

[aschwaighofer]

@swift-ci Please test

[aschwaighofer]

preset=buildbot_incremental,tools=RA,stdlib=RA
@swift-ci Please test with preset macOS

[aschwaighofer]

incremental builder: https://ci.swift.org/job/oss-swift-incremental-RA-macos/14501/

[aschwaighofer]

20:58:28 -- Configuring done
20:58:28 CMake Error at stdlib/cmake/modules/AddSwiftStdlib.cmake:923 (add_library):
20:58:31   No SOURCES given to target: swift_MatchingEngine-watchos-armv7k
20:58:31 Call Stack (most recent call first):
20:58:31   stdlib/cmake/modules/AddSwiftStdlib.cmake:2045 (add_swift_target_library_single)
20:58:31   stdlib/public/MatchingEngine/CMakeLists.txt:25 (add_swift_target_library)

[milseman]

Reverting this is likely to cause significantly more breakage overall, so let’s see if we can’t fix the CI. Is it just not getting the sources, or is there a compilation error?

[milseman]

The incremental builder needs to run update checkout, that is, it needs a fullish build once to get the new sources

[rxwei]

cc @drewmaxwell-apple

[aschwaighofer]

I don't know why the failure is. But nearly all bots are red right now.

[aschwaighofer]

There seem to be different issues. I have seen the issue mentioned above and here is a different one:

https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04/18158/console

10:56:23 <unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
10:56:23 
/home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:4:40: error: cannot find 'libswiftParseRegexLiteral' in scope
10:56:23   Parser_registerRegexLiteralParsingFn(libswiftParseRegexLiteral)
10:56:23                                        ^~~~~~~~~~~~~~~~~~~~~~~~~
10:56:23 /home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:5:39: error: cannot find 'libswiftLexRegexLiteral' in scope
10:56:23   Parser_registerRegexLiteralLexingFn(libswiftLexRegexLiteral)
10:56:23                                       ^~~~~~~~~~~~~~~~~~~~~~~

[aschwaighofer]

Here is a -- probably incomplete -- list of bots currently failing with either of these two errors:

https://ci.swift.org/job/oss-swift-incremental-RA-macos/14503/
https://ci.swift.org/job/oss-swift-incremental-RA-macos-apple-silicon/1969
https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04/18158
https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04-long-test/9401/
https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-18_04/13485
https://ci.swift.org/job/oss-swift-pr-test/19434
https://ci.swift.org/job/oss-swift-incremental-ASAN-RA-macos/6148
https://ci.swift.org/job/oss-swift_tools-RA_stdlib-RDA_long-test/5536
https://ci.swift.org/job/oss-swift_tools-RA_stdlib-RDA_test-simulator/4172
https://ci.swift.org/job/oss-swift_tools-R_stdlib-RD_test-simulator/6761
https://ci.swift.org/job/oss-swift-test-stdlib-with-toolchain/746
https://ci.swift.org/job/swift-main-source-compat-suite/6355
https://ci.swift.org/view/Dashboard/job/swift-main-source-compat-suite-debug/4393

[rxwei]

There seem to be different issues. I have seen the issue mentioned above and here is a different one:

https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04/18158/console

10:56:23 <unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
10:56:23 
/home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:4:40: error: cannot find 'libswiftParseRegexLiteral' in scope
10:56:23   Parser_registerRegexLiteralParsingFn(libswiftParseRegexLiteral)
10:56:23                                        ^~~~~~~~~~~~~~~~~~~~~~~~~
10:56:23 /home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:5:39: error: cannot find 'libswiftLexRegexLiteral' in scope
10:56:23   Parser_registerRegexLiteralLexingFn(libswiftLexRegexLiteral)
10:56:23                                       ^~~~~~~~~~~~~~~~~~~~~~~

This one has found the sources and has SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING=TRUE, but did not try to build either _MatchingEngine or _StringProcessing.

10:47:37 -- Using Experimental String Processing library for _MatchingEngine (/home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift-experimental-string-processing).
10:47:37 -- Using Experimental String Processing library for _StringProcessing (/home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift-experimental-string-processing).

I wonder if it just needs a clean build.

[aschwaighofer]

The build bot should have been cleaned in between 2 days. That at least would be my understanding of doing things sensibly (i.e the bots clean the workspace overnight).

The issue did not seem to resolve itself since yesterday, though. I can't tell why.

That said I tried a clean build locally with update-checkout --clone-with-ssh --scheme main and this build did not reproduce the issue and was fine.
Neither would it appear we had issue when testing the PR, right?

It could be that the bots don't use update-checkout and the bot needs to be configured to checkout whatever repo we are missing (but then how could your PR have passed?). Or we indeed just need a clean workspace build and that has not happened overnight for whatever reason.

I will wait until tomorrow to see what happens tonight.

[aschwaighofer]

According to the following run the bot macOS-31 was cleaned today the last time: https://ci.swift.org/job/clean-workspace-macOS-bk/102923/console

That also seems to be the bot that the incremental macOS bot runs on: https://ci.swift.org/job/oss-swift-incremental-RA-macos/14503/.

I am going to try to trigger a build now by committing an NFC change. This build should be on a clean workspace if it runs on macOS-31.

[aschwaighofer]

Okay, this run https://ci.swift.org/job/oss-swift-incremental-RA-macos/14504 is indeed running on macOS-31 now. So it should be running a clean build because of https://ci.swift.org/job/clean-workspace-macOS-bk/102923.

[rxwei]

Mishal informed us that new repos need to be manually added to the CI and that @drewmaxwell-apple should be able to help add it. Drew, could you help us add it when you get a chance?

[aschwaighofer]

Okay. Sounds like we have found a way to not break the bots: first we configure the various CI configurations to check out the necessary repo. Once this is complete we can re-commit this change.