Branch: master
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
43 lines (27 sloc) 3.97 KB

Sourcekitd Stress Tester

The Sourcekitd stress tester is a utility for running a range of sourcekitd requests at various locations in a set of Swift source files in order to find reproducible crashes in sourcekitd. It currently tests the CursorInfo, RangeInfo, SemanticRefactoring, CodeComplete, EditorOpen and EditorClose sourcekitd requests, which support a range of editor features like syntax highlighting, code completion, jump-to-definition, live issues, quick help, and refactoring.


You can build SourceKitStressTester using Swift's build-script, using the command line, or using Xcode. In the latter two cases, you'll use the script in the parent directory. Please see the README file adjacent to it for full instructions.


However you build, you will end up with two executables: sk-stress-test and sk-swiftc-wrapper. These will be available in the .build/debug directory if building on the command line or via the Swift repo's build-script, and under Products/Debug in the Xcode project's DerivedData directory if building there. They are also available in the usr/bin directory of recent trunk and swift 5.0 development toolchains from, if you're just interested in running them, rather than building them locally.


The sk-stress-test executable is the sourcekitd stress tester itself. It takes as input a swift source file to run over, along with the set of driver arguments you would pass to swiftc to compile those files. Here is a simple example invocation:

$ echo 'print("hello")' > /tmp/test.swift
$ .build/debug/sk-stress-test /tmp/test.swift swiftc /tmp/test.swift


The sk-swiftc-wrapper executable allows the stress tester to be easily run on an existing project. It serves as a drop-in replacement for swiftc during a build. When invoked, it simply invokes swiftc proper with the same arguments. If the swiftc invocation fails, sk-swiftc-wrapper will exit with the same exit code. If it succeeds, it additionally invokes sk-stress-test for each Swift file that appears in its arguments, followed by the arguments themselves. By default it then exits with whatever exit code was returned by sk-stress-test, meaning a stress testing failure will cause the build to fail. Specify the SK_STRESS_SILENT environment variable to have the wrapper return the same exit code as the swiftc invocation, regardless of any stress tester failures. Here is an example invocation:

$ echo 'print("hello")' > /tmp/test.swift
$ .build/debug/sk-swiftc-wrapper /tmp/test.swift

By default sk-swiftc-wrapper invokes the swiftc from the toolchain specified by the TOOLCHAINS environment variable, or the default toolchain of the currently xcode-selected Xcode. You can override this behaviour by setting the SK_STRESS_SWIFTC environment variable. Similarly, it looks for sk-stress-test adjacent to its own launch path, but you can override this by setting the SK_STRESS_TEST environment variable.

Stress testing sourcekitd with an existing Xcode project

Since sk-swiftc-wrapper works as a drop in replacement for swiftc, you can run the stress tester over an existing Xcode project by setting sk-swiftc-wrapper as the Swift compiler to use and then building. This is done by adding a user-defined build setting, SWIFT_EXEC, to the project or target you would like to stress test, with the path to sk-swiftc-wrapper as its value. When you next build, stress testing failures will then manifest in the build output as compilation failures.

Stress testing sourcekitd with a Swift package manager project

For Swift package manager projects, you can stress test sourcekitd by setting the SWIFT_EXEC environment variable to point to sk-swiftc-wrapper and building as normal. For projects without any custom flags, this is as simple as running:

SWIFT_EXEC=/path/to/sk-swiftc-wrapper swift build

in the directory with the package manifest file.