Build fails with `--static-swift-stdlib` (Linux)

[gohanlon]

My project's server target uses swift build --static-swift-stdlib to statically link the Swift runtime. (Note: Below I'm using the long form of -Xswiftc -static-stdlib because swift test doesn't support the short-form option.) My server app is setup similarly to Vapor's default Dockerfile template.

Usually my Linux builds use the release configuration, and so xctest-dynamic-overlay builds as expected. But, I'm tracing down a hard-to-reproduce SIGSEGV (which I don't think has anything to do with xctest-dynamic-overlay) and want to put a debug build onto a staging environment.

I was able to locally produce a build error matching that of my project using the xctest-dynamic-overlay test suite:

error: emit-module command failed with exit code 1 (use -v to see invocation)[1/6] Emitting module XCTestDynamicOverlay
/Users/gohanlon/src/gohanlon/xctest-dynamic-overlay/Sources/XCTestDynamicOverlay/XCTFail.swift:70:28: error: no such module 'XCTest'
    @_exported import func XCTest.XCTFail

I'm not at all sure sure how or even whether this is fixable. I just started reading xctest-dynamic-overlay today, and what's immediately clear is that something I've been enjoying taking for granted is actually no straightforward feat. At a minimum, maybe this NB: could be amended?

For now, rather than convincing the linker to build, I plan to switch to using the quite-heavy official "slim" Swift image and abandon statically linking the Swift runtime. (So long as I can still produce my SIGSEGV!) Eventually, it'd be great to be able to create images containing debug builds that as similar as possible to my production images.

Methodology

I suspect that the test-linux task is currently used? It's using swift:5.6.2-focal as the run image, but that image doesn't have make:

% docker run -it swift:5.6.2-focal bash
root@3a2e9e48a354:/# pwd
/
root@3a2e9e48a354:/# find | grep make
./usr/share/mime/application/vnd.framemaker.xml
./usr/share/mime/application/x-pagemaker.xml
./usr/share/mime/application/x-font-framemaker.xml
./usr/share/mime/text/x-cmake.xml
./usr/share/mime/text/x-makefile.xml
./usr/include/c++/v1/__algorithm/make_heap.h
./usr/bin/makeconv

So, I kludged the Makefile to:

• apt-get install make,
• output swift test directly so build errors will be shown, and
• add test-linux-static-swift-stdlib and test-static-swift-stdlib.

diff --git a/Makefile b/Makefile
index 4021b67..9ddceba 100644
--- a/Makefile
+++ b/Makefile
@@ -6,9 +6,16 @@ EXPECTED_STRING = This is expected to fail!
 EXPECTED = \033[31m\"$(EXPECTED_STRING)\"\033[0m
 
 test:
-	@swift test 2>&1 | grep '$(EXPECTED_STRING)' > /dev/null \
-		&& (echo "$(PASS) $(XCT_FAIL) was called with $(EXPECTED)" && exit) \
-		|| (echo "$(FAIL) expected $(XCT_FAIL) to be called with $(EXPECTED)" >&2 && exit 1)
+	@swift test
+	# @swift test 2>&1 | grep '$(EXPECTED_STRING)' > /dev/null \
+	# 	&& (echo "$(PASS) $(XCT_FAIL) was called with $(EXPECTED)" && exit) \
+	# 	|| (echo "$(FAIL) expected $(XCT_FAIL) to be called with $(EXPECTED)" >&2 && exit 1)
+
+test-static-swift-stdlib:
+	@swift test -Xswiftc -static-stdlib
+	# @swift test -Xswiftc -static-stdlib 2>&1 | grep '$(EXPECTED_STRING)' > /dev/null \
+	# 	&& (echo "$(PASS) $(XCT_FAIL) was called with $(EXPECTED)" && exit) \
+	# 	|| (echo "$(FAIL) expected $(XCT_FAIL) to be called with $(EXPECTED)" >&2 && exit 1)
 
 test-linux:
 	@docker run \
@@ -16,7 +23,15 @@ test-linux:
 		-v "$(PWD):$(PWD)" \
 		-w "$(PWD)" \
 		swift:5.6.2-focal \
-		bash -c "make test"
+		bash -c "apt-get update && apt-get install make && make test"
+
+test-linux-static-swift-stdlib:
+	@docker run \
+		--rm \
+		-v "$(PWD):$(PWD)" \
+		-w "$(PWD)" \
+		swift:5.6.2-focal \
+		bash -c "apt-get update && apt-get install make && make test-static-swift-stdlib"
 
 format:
 	@swift format \

I verified that the test-linux output is correct:

% make test-linux                 
[...snip apt-install'ing make...]
[1/1] Planning build
Compiling plugin Swift-DocC...
Compiling plugin Swift-DocC Preview...
Building for debugging...
Build complete! (4.90s)
Test Suite 'All tests' started at 2022-07-07 23:18:50.194
Test Suite 'debug.xctest' started at 2022-07-07 23:18:50.195
Test Suite 'XCTestDynamicOverlayTests' started at 2022-07-07 23:18:50.195
Test Case 'XCTestDynamicOverlayTests.testXCTFail' started at 2022-07-07 23:18:50.195
/Users/gohanlon/src/gohanlon/xctest-dynamic-overlay/Tests/XCTestDynamicOverlayTests/TestHelpers.swift:4: error: XCTestDynamicOverlayTests.testXCTFail : failed - This is expected to fail!
Test Case 'XCTestDynamicOverlayTests.testXCTFail' failed (0.001 seconds)
Test Suite 'XCTestDynamicOverlayTests' failed at 2022-07-07 23:18:50.197
	 Executed 1 test, with 1 failure (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'debug.xctest' failed at 2022-07-07 23:18:50.197
	 Executed 1 test, with 1 failure (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'All tests' failed at 2022-07-07 23:18:50.197
	 Executed 1 test, with 1 failure (0 unexpected) in 0.001 (0.001) seconds

And, here's the test-linux-static-swift-stdlib failure:

% make test-linux-static-swift-stdlib
[...snip apt-install'ing make...]
[1/1] Planning build
Compiling plugin Swift-DocC...
Compiling plugin Swift-DocC Preview...
Building for debugging...
error: emit-module command failed with exit code 1 (use -v to see invocation)[1/6] Emitting module XCTestDynamicOverlay
/Users/gohanlon/src/gohanlon/xctest-dynamic-overlay/Sources/XCTestDynamicOverlay/XCTFail.swift:70:28: error: no such module 'XCTest'
    @_exported import func XCTest.XCTFail
                           ^

[gohanlon]

Works great, thank you!