Integrate experimental string processing modules and enable end-to-end regex.

[rxwei]

• Checkout apple/swift-experimental-string-processing using a tag.
• Build _MatchingEngine as part of libswift (ExperimentalRegex) using sources from the package.
• Parse regex literals using the parser from _MatchingEngine.
• Build both _MatchingEngine and _StringProcessing as part of core libs using sources from the package.
• Use Regex<DynamicCaptures> as the default regex type until we finalize 'Regex<Captures>' -> 'Regex<Match>' swift-experimental-string-processing#68.

---

Rationale for out-of-tree development:

At this point we are interested in faster development iterations, and a Swift package makes it significantly quicker for us to come up with a prototype. We integrate into the compiler based on tags, which means that our daily development will stay at high speed on the main branch of apple/swift-experimental-string-processing without breaking Swift CI. Long term maybe we can use a bot to clone the code into the Swift repo, so that Swift no longer depends on the package.

Testing is done first in our repo via XCTests. Then we create a PR to apple/swift to update the tag and run CI. There’s also parser, type checker, SILGen and validation tests on the compiler side for this feature. Those tests link against the modules built using sources from the string processing package, and will reflect any failures from the integration PR.

[rxwei]

@swift-ci please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - 71c0778d4da23f34812809aaaaa367a4bcd97fa1

[swift-ci]

Build failed
Swift Test Linux Platform
Git Sha - 71c0778d4da23f34812809aaaaa367a4bcd97fa1

[milseman]

LGTM

@hamishknight and I can take over Regex.swift after this gets merged.

[rxwei]

@swift-ci please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - d2bed9579a953f19130f9f9d1d5ad2e16c3ff05e

[rxwei]

@swift-ci please test macOS

[rxwei]

@milseman It might better to just drop Regex.swift altogether (though not in this PR) and define all compiler interfaces in the package. This way we can get everything mock-tested within the package, and prevent any internal symbol conflicts with Regex.swift (since it's not included in the package build).

[rxwei]

@swift-ci please test windows platform

[swift-ci]

Build failed
Swift Test Linux Platform
Git Sha - d2bed9579a953f19130f9f9d1d5ad2e16c3ff05e

[rxwei]

@swift-ci please clean test Linux

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - d2bed9579a953f19130f9f9d1d5ad2e16c3ff05e

[rxwei]

@swift-ci please clean test macos

[rxwei]

@swift-ci please clean test windows

[rxwei]

@swift-ci please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - bb386798dcb74926c65e09ce0533af60727dd950

[rxwei]

@swift-ci please test

[compnerd]

@swift-ci please test Windows platform

[compnerd]

@swift-ci please test macOS platform

[compnerd]

@swift-ci please test Linux platform

[rxwei]

@swift-ci please test

[rxwei]

@compnerd suggested that we add cmake support to the string processing package itself so that we can avoid globbing in this repo. We plan to do this as a follow-up patch.

[porglezomp]

I have some questions about being able to disable this feature in builds.

[rxwei]

I've added conditional import of ExperimentalRegex in libswift based on SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING. Also, I verified that when I delete the string processing repo and set --swift-enable-experimental-string-processing to false, it builds successfully without producing string processing related targets.

[rxwei]

@swift-ci please test

[rxwei]

@swift-ci please test

[rxwei]

@swift-ci please test

[rxwei]

This has been merged along with #40595.
300cbab

[aschwaighofer]

This looks like it broke the incremental builder: https://ci.swift.org/job/oss-swift-incremental-RA-macos/14501/

20:58:28 -- Configuring done
20:58:28 CMake Error at stdlib/cmake/modules/AddSwiftStdlib.cmake:923 (add_library):
20:58:31   No SOURCES given to target: swift_MatchingEngine-watchos-armv7k
20:58:31 Call Stack (most recent call first):
20:58:31   stdlib/cmake/modules/AddSwiftStdlib.cmake:2045 (add_swift_target_library_single)
20:58:31   stdlib/public/MatchingEngine/CMakeLists.txt:25 (add_swift_target_library)