[test] Fix testExamplePackageDealer #9134

sebsto · 2025-09-12T19:14:45Z

The test testExamplePackageDealer was failing on Amazon Linux and not in Swift.org's CI
See #9126

Motivation:

The reason why the test is working on Swift.org's CI is because it silently ignores outbound connection issues. The test fails to clone the project on github.com and silently stops.

When the clone operation succeed, the test fails. This is the case on macOS and Amazon Linux 2023, probably others OSes too.

Modifications:

I observed multiple problems in this test.

the test was not marked async and the withTemporaryDirectory was not await. The test completed execution before the git clone operation finished.
The git clone operation requires an environment variable HOME to be defined
On Swift 6.1 and later, the second build recompiles the plugins. There are some lines Compiling plugin that are valid and should not fail the test.

Here is an example of a valid output. It contains Compiling plugin

➜  example-package-dealer git:(main) ✗ swift build > output.txt 2>&1
➜  example-package-dealer git:(main) ✗ cat output.txt 
[0/1] Planning build
[1/1] Compiling plugin GenerateManual
[2/2] Compiling plugin GenerateDoccReference
Building for debugging...
[2/5] Write swift-version-1FA6DEBB9BBC2F77.txt
Build complete! (1.01s)
➜  example-package-dealer git:(main) ✗ swift --version 
Apple Swift version 6.1.2 (swift-6.1.2-RELEASE)
Target: arm64-apple-macosx15.0

Result:

The test passes on macOS and Amazon Linux 2023

✗ swift test --filter testExamplePackageDealer
Building for debugging...
[9/9] Linking SwiftPMPackageTests
Build complete! (12.99s)
Test Suite 'Selected tests' started at 2025-09-12 21:04:15.723.
Test Suite 'SwiftPMPackageTests.xctest' started at 2025-09-12 21:04:15.727.
Test Suite 'SwiftPMPackageTests.xctest' passed at 2025-09-12 21:04:15.727.
         Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'Selected tests' passed at 2025-09-12 21:04:15.727.
         Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.004) seconds
􀟈  Test run started.
􀄵  Testing Library Version: 6.1.2 (d6b70f9ef9eb207)
􀄵  Target Platform: arm64-apple-macosx
􀟈  Suite BasicTests started.
􀟈  Test testExamplePackageDealer() started.
􁁛  Test testExamplePackageDealer() passed after 17.278 seconds.
􁁛  Suite BasicTests passed after 17.279 seconds.
􁁛  Test run with 1 test passed after 17.279 seconds.

plemarquand · 2025-09-12T19:27:15Z

@swift-ci test

sebsto · 2025-09-14T17:30:28Z

I removed the $HOME addition to git clone, looks like this is only required on macOS and causes the test to fail on Linux. This clone operation is protected by a withKnownIssue anyway.

jakepetroules · 2025-09-15T05:48:52Z

@swift-ci test

sebsto · 2025-09-15T07:17:05Z

thank you @jakepetroules for having approved. The test was still failing on the CI because the git clone operation failed to create the directory. This went unnoticed so far because the test finished executing before the end of the git clone.

I made two changes

8b552d8 : make sure the test runs on macOS (there is no reasons to skip it on that platform)
8b552d8 : skip the test completely when the git clone fails. This primarily happen on Swift.org's CI due to restricted network access.

dschaefer2 · 2025-09-15T16:37:34Z

@bkhouri does this make sense?

bkhouri

I question the purpose and intension of this test and whether we have coverage already via other SwiftPM tests.

If the intent is to ensure the project builds, this should be handled by a https://github.com/apple/example-package-dealer PR build.

bkhouri · 2025-09-15T17:17:32Z

Tests/IntegrationTests/BasicTests.swift

+
+                #if os(macOS)
+                    // On macOS, we add the HOME variable to avoid git errors.                
+                    try sh("git\(ProcessInfo.exeSuffix)", "clone", "https://github.com/apple/example-package-dealer", packagePath, env: ["HOME": tempDir.pathString])


question: what git error occur on macOS? can we also add the same for all platform to ensure the test is uniform?

􀄵 Command failed with exit code: terminated(code: 1) - command: git clone https://github.com/apple/example-package-dealer /var/folders/14/nwpsn4b504gfp02_mrbyd2jr0000gr/T/TemporaryDirectory.Wew2EB/dealer stdout: stderr: Cloning into '/var/folders/14/nwpsn4b504gfp02_mrbyd2jr0000gr/T/TemporaryDirectory.Wew2EB/dealer'... Could not get home directory: $HOME is not defined

This is the only test I found that uses the git command via a shell.
The other tests using git are using SPM's GitRepositoryProvider struct from

swift-package-manager/Sources/SourceControl/GitRepository.swift

Line 86 in a9e72ad

public struct GitRepositoryProvider: RepositoryProvider, Cancellable {

bkhouri · 2025-09-15T17:20:52Z

Tests/IntegrationTests/BasicTests.swift

+
+            // Do not run the test when the git clone operation failed
+            if !FileManager.default.fileExists(atPath: packagePath.pathString) {
+                return


I'm not fond of this as it gives the impression the tests passes, when it wasn't executed at all. We should include a TODO to use the Test cancellation feature once it's available.

https://forums.swift.org/t/pitch-test-cancellation/81847/18

In the meantime, can we record a warning Issue.record("....", severity: .warning) as introduced by https://forums.swift.org/t/accepted-st-0013-test-issue-severity/81385/2 (though our self-hosted pipelines doesn't currently use a version of the toolchain that has these features)

I added this code to record the issue, only when compiling with Swift 6.3 or more recent

#if compiler(>=6.3) Issue.record("Can't clone the repository \(repoToClone), abording the test.", severity: .warning) #endif

Tests/IntegrationTests/BasicTests.swift

sebsto · 2025-09-16T09:38:52Z

I question the purpose and intention of this test and whether we have coverage already via other SwiftPM tests.

@bkhouri I let you judge if this test is useful or not. I don't have the overall view of all test. I was just trying to fix that one that fails on our CI when building the Swift toolchain. I was surprised it passes on Swift.org's CI

If you deem this is not necessary anymore, happy to delete it entirely :-)

… works on Swift 6.3)

plemarquand · 2025-09-16T14:37:40Z

@swift-ci test

bkhouri · 2025-09-16T18:20:17Z

@sebsto: We can delete this test once https://github.com/swiftlang/example-package-dealer/ does not have any PR builds.

@shahmishal : where should I file an issue for this as Issues are not enabled in the example-package-dealer repository.

bkhouri · 2025-09-16T18:20:31Z

@swift-ci test windows

sebsto · 2025-09-18T16:05:16Z

@bkhouri It seems the errors we see on Windows are not related to this change

[2025-09-16T20:36:06.656Z]   stderr: "TSCBasic/Path.swift:969: Assertion failed\nCurrent stack trace:\n0    swiftCore.dll                      0x00007ffcc03d91d0 swift_reportError + 94\n1    swiftCore.dll                      0x00007ffcc0476300 swift_stdlib_reportFatalErrorInFile + 132\n2    swiftCore.dll                      0x00007ffcc00e83d0 StaticString.withUTF8Buffer<A>(_:) + 1060\n3    swiftCore.dll                      0x00007ffcc00e7580 _assertionFailure(_:_:file:line:flags:) + 326\n4    swift-test.exe                     0x00007ff74febfb30 AbsolutePath.relative(to:) + 3703\n5    swift-test.exe                     0x00007ff74c0f2deb <unavailable> + 994795\n6    swift-test.exe                     0x00007ff74d062415 <unavailable> + 17179669\n7    swift-test.exe                     0x00007ff74d06244d <unavailable> + 17179725\n8    swift-test.exe                     0x00007ff74c021ab5 <unavailable> + 137909\n9    swift-test.exe                     0x00007ff74d062206 <unavailable> + 17179142\n10   swift-test.exe                     0x00007ff74c26681a <unavailable> + 2517018\n11   swift-test.exe                     0x00007ff74c263ab6 <unavailable> + 2505398\n12   swift-test.exe                     0x00007ff74c26e2ef <unavailable> + 2548463\n13   swift_Concurrency.dll              0x00007ffcd1dae5d0 AsyncMapSequence.transform.getter + 20830\n14   swift_Concurrency.dll              0x00007ffcd1db41a0 swift_job_run + 114\n15   dispatch.dll                       0x00007ffcd1c1a8c0 dispatch_assert_queue_barrier + 1008\n16   dispatch.dll                       0x00007ffcd1c1a8c0 dispatch_assert_queue_barrier + 632\n17   dispatch.dll                       0x00007ffcd1c22d20 dispatch_prohibit_transition_to_multithreaded + 8323\n18   ucrtbase.dll                       0x00007ffcd0952630 o_exp + 90\n19   KERNEL32.DLL                       0x00007ffcd2a27ab0 BaseThreadInitThunk + 20\n20   ntdll.dll                          0x00007ffce526a8a0 RtlUserThreadStart + 33\n"

sebsto · 2025-09-19T06:30:03Z

@bkhouri any blocker left before merging this ?

shahmishal · 2025-09-19T18:08:05Z

This is now merged: swiftlang/example-package-dealer#21

bkhouri · 2025-09-19T21:50:26Z

@bkhouri any blocker left before merging this ?

No blockers from my perspective.

bkhouri · 2025-09-19T21:53:44Z

@bkhouri It seems the errors we see on Windows are not related to this change

[2025-09-16T20:36:06.656Z]   stderr: "TSCBasic/Path.swift:969: Assertion failed\nCurrent stack trace:\n0    swiftCore.dll                      0x00007ffcc03d91d0 swift_reportError + 94\n1    swiftCore.dll                      0x00007ffcc0476300 swift_stdlib_reportFatalErrorInFile + 132\n2    swiftCore.dll                      0x00007ffcc00e83d0 StaticString.withUTF8Buffer<A>(_:) + 1060\n3    swiftCore.dll                      0x00007ffcc00e7580 _assertionFailure(_:_:file:line:flags:) + 326\n4    swift-test.exe                     0x00007ff74febfb30 AbsolutePath.relative(to:) + 3703\n5    swift-test.exe                     0x00007ff74c0f2deb <unavailable> + 994795\n6    swift-test.exe                     0x00007ff74d062415 <unavailable> + 17179669\n7    swift-test.exe                     0x00007ff74d06244d <unavailable> + 17179725\n8    swift-test.exe                     0x00007ff74c021ab5 <unavailable> + 137909\n9    swift-test.exe                     0x00007ff74d062206 <unavailable> + 17179142\n10   swift-test.exe                     0x00007ff74c26681a <unavailable> + 2517018\n11   swift-test.exe                     0x00007ff74c263ab6 <unavailable> + 2505398\n12   swift-test.exe                     0x00007ff74c26e2ef <unavailable> + 2548463\n13   swift_Concurrency.dll              0x00007ffcd1dae5d0 AsyncMapSequence.transform.getter + 20830\n14   swift_Concurrency.dll              0x00007ffcd1db41a0 swift_job_run + 114\n15   dispatch.dll                       0x00007ffcd1c1a8c0 dispatch_assert_queue_barrier + 1008\n16   dispatch.dll                       0x00007ffcd1c1a8c0 dispatch_assert_queue_barrier + 632\n17   dispatch.dll                       0x00007ffcd1c22d20 dispatch_prohibit_transition_to_multithreaded + 8323\n18   ucrtbase.dll                       0x00007ffcd0952630 o_exp + 90\n19   KERNEL32.DLL                       0x00007ffcd2a27ab0 BaseThreadInitThunk + 20\n20   ntdll.dll                          0x00007ffce526a8a0 RtlUserThreadStart + 33\n"

This is a known issue: #8602

fix testExamplePackageDealer

5580c85

sebsto requested review from jakepetroules, dschaefer2, shawnhyam, bripeticca, plemarquand, owenv, bkhouri, cmcgee1024 and daveyc123 as code owners September 12, 2025 19:14

sebsto mentioned this pull request Sep 12, 2025

[Amazon Linux 2023] testExamplePackageDealer fails #9126

Open

remove HOME in glit clone - this is macOS only

4fb3eca

jakepetroules approved these changes Sep 15, 2025

View reviewed changes

sebsto added 2 commits September 15, 2025 09:13

make sur ethe test runs on macOS

8b552d8

when git clone fails, skip the test completly

3f804ee

bkhouri reviewed Sep 15, 2025

View reviewed changes

Tests/IntegrationTests/BasicTests.swift Outdated Show resolved Hide resolved

Tests/IntegrationTests/BasicTests.swift Outdated Show resolved Hide resolved

sebsto added 2 commits September 16, 2025 14:28

Use IssueRecord(_:severity) to report the fact we skip the test (only…

d3ea1a6

… works on Swift 6.3)

add a TODO to use Test Cancellation when it will become available

41b99f5

bkhouri approved these changes Sep 16, 2025

View reviewed changes

shahmishal merged commit 9fc88d2 into swiftlang:main Sep 19, 2025
6 checks passed

[test] Fix testExamplePackageDealer #9134

[test] Fix testExamplePackageDealer #9134

Uh oh!

Conversation

sebsto commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation:

Modifications:

Result:

Uh oh!

plemarquand commented Sep 12, 2025

Uh oh!

sebsto commented Sep 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jakepetroules commented Sep 15, 2025

Uh oh!

sebsto commented Sep 15, 2025

Uh oh!

dschaefer2 commented Sep 15, 2025

Uh oh!

bkhouri left a comment

Choose a reason for hiding this comment

Uh oh!

bkhouri Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

sebsto Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

sebsto Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

bkhouri Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

sebsto Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sebsto commented Sep 16, 2025

Uh oh!

plemarquand commented Sep 16, 2025

Uh oh!

bkhouri commented Sep 16, 2025

Uh oh!

bkhouri commented Sep 16, 2025

Uh oh!

sebsto commented Sep 18, 2025

Uh oh!

sebsto commented Sep 19, 2025

Uh oh!

shahmishal commented Sep 19, 2025

Uh oh!

bkhouri commented Sep 19, 2025

Uh oh!

bkhouri commented Sep 19, 2025

Uh oh!

Uh oh!

Uh oh!

sebsto commented Sep 12, 2025 •

edited

Loading

sebsto commented Sep 14, 2025 •

edited

Loading

sebsto Sep 16, 2025 •

edited

Loading