WaitCommandTests times out in Docker-based E2E test infrastructure #14993
Description
Build information
Build: https://github.com/microsoft/aspire/actions/runs/23963979582
Build error leg or test failing: Aspire.Cli.EndToEnd.Tests.WaitCommandTests.CreateStartWaitAndStopAspireProject
Artifact: logs-Cli.EndToEnd-ubuntu-latest
Fill in the error message template
Error Message
Fill the error message using step by step known issues guidance.
{
"ErrorMessage": "",
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": false
}Error details
Error details (160 lines)
Error Message: Hex1b.Automation.Hex1bAutomationException : Step 49 of 49 failed — WaitUntilText("AppHost started successfully.")
Timed out after 00:03:00 waiting for: text "AppHost started successfully." to appear
at WaitCommandTests.cs:52
Completed steps (48 of 49):
[1] WaitUntilText("# ") — 254ms ✓ (CliE2EAutomatorHelpers.cs:24)
[2] Wait(500ms) — 501ms ✓ (CliE2EAutomatorHelpers.cs:26)
[3] Type("CMDCOUNT=0; PROMPT_COMMAND=...") — 1ms ✓ (CliE2EAutomatorHelpers.cs:30)
[4] Key(Enter) — 0ms ✓ (CliE2EAutomatorHelpers.cs:31)
[5] WaitUntil("success prompt [1 OK] $") — 253ms ✓ (Hex1bAutomatorTestHelpers.cs:25)
[6] Type("umask 000") — 1ms ✓ (CliE2EAutomatorHelpers.cs:35)
[7] Key(Enter) — 3ms ✓ (CliE2EAutomatorHelpers.cs:36)
[8] WaitUntil("success prompt [2 OK] $") — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:25)
[9] Type("export ASPIRE_PLAYGROUND=tr...") — 1ms ✓ (CliE2EAutomatorHelpers.cs:40)
[10] Key(Enter) — 0ms ✓ (CliE2EAutomatorHelpers.cs:41)
[11] WaitUntil("success prompt [3 OK] $") — 252ms ✓ (Hex1bAutomatorTestHelpers.cs:25)
[12] Type("cd /workspace/695c70d1-2a1d...") — 0ms ✓ (CliE2EAutomatorHelpers.cs:46)
[13] Key(Enter) — 0ms ✓ (CliE2EAutomatorHelpers.cs:47)
[14] WaitUntil("success prompt [4 OK] $") — 283ms ✓ (Hex1bAutomatorTestHelpers.cs:25)
[15] Type("trap 'cp -r ~/.aspire/logs ...") — 1ms ✓ (CliE2EAutomatorHelpers.cs:51)
[16] Key(Enter) — 0ms ✓ (CliE2EAutomatorHelpers.cs:52)
[17] WaitUntil("success prompt [5 OK] $") — 254ms ✓ (Hex1bAutomatorTestHelpers.cs:25)
[18] Type("/opt/aspire-scripts/get-asp...") — 0ms ✓ (CliE2EAutomatorHelpers.cs:77)
[19] Key(Enter) — 0ms ✓ (CliE2EAutomatorHelpers.cs:78)
[20] WaitUntil("success prompt [6 OK] $ (fail-fast on error)") — 7478ms ✓ (Hex1bAutomatorTestHelpers.cs:73)
[21] Type("export PATH=~/.aspire/bin:$...") — 0ms ✓ (CliE2EAutomatorHelpers.cs:80)
[22] Key(Enter) — 0ms ✓ (CliE2EAutomatorHelpers.cs:81)
[23] WaitUntil("success prompt [7 OK] $") — 293ms ✓ (Hex1bAutomatorTestHelpers.cs:25)
[24] Type("aspire new") — 2ms ✓ (Hex1bAutomatorTestHelpers.cs:174)
[25] Key(Enter) — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:175)
[26] WaitUntil("template selection list (> Starter App)") — 309ms ✓ (Hex1bAutomatorTestHelpers.cs:176)
[27] Key(Enter) — 1ms ✓ (Hex1bAutomatorTestHelpers.cs:185)
[28] WaitUntil("project name prompt") — 305ms ✓ (Hex1bAutomatorTestHelpers.cs:255)
[29] Type("AspireWaitApp") — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:259)
[30] Key(Enter) — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:260)
[31] WaitUntil("output path prompt") — 287ms ✓ (Hex1bAutomatorTestHelpers.cs:263)
[32] Key(Enter) — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:267)
[33] WaitUntil("URLs prompt") — 1743ms ✓ (Hex1bAutomatorTestHelpers.cs:270)
[34] Key(Enter) — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:274)
[35] WaitUntil("Redis cache prompt") — 391ms ✓ (Hex1bAutomatorTestHelpers.cs:279)
[36] Key(Enter) — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:289)
[37] WaitUntil("test project prompt") — 726ms ✓ (Hex1bAutomatorTestHelpers.cs:295)
[38] Key(Enter) — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:299)
[39] WaitUntil("agent init prompt or success prompt [8 OK] $") — 0:15.295 ✓ (Hex1bAutomatorTestHelpers.cs:123)
[40] Wait(500ms) — 501ms ✓ (Hex1bAutomatorTestHelpers.cs:136)
[41] Type("n") — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:141)
[42] Key(Enter) — 0ms ✓ (Hex1bAutomatorTestHelpers.cs:142)
[43] WaitUntil("success prompt [8 OK] $ after agent init") — 254ms ✓ (Hex1bAutomatorTestHelpers.cs:145)
[44] Type("cd AspireWaitApp/AspireWait...") — 0ms ✓ (WaitCommandTests.cs:45)
[45] Key(Enter) — 0ms ✓ (WaitCommandTests.cs:46)
[46] WaitUntil("success prompt [9 OK] $") — 252ms ✓ (Hex1bAutomatorTestHelpers.cs:25)
[47] Type("aspire start") — 2ms ✓ (WaitCommandTests.cs:50)
[48] Key(Enter) — 0ms ✓ (WaitCommandTests.cs:51)
[49] WaitUntilText("AppHost started successfully.") — FAILED after 3:00.148
Total elapsed: 3:29.804
Terminal snapshot at failure (160x48, cursor at 10,41, normal screen):
bash-5.2# CMDCOUNT=0; PROMPT_COMMAND='s=$?;((CMDCOUNT++));PS1="[$CMDCOUNT $([ $s -eq 0 ] && echo OK || echo ERR:$s)] \$ "'
[1 OK] $ umask 000
[2 OK] $ export ASPIRE_PLAYGROUND=true TERM=xterm DOTNET_CLI_TELEMETRY_OPTOUT=true DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true DOTNET_GENERATE_ASPNET_CERTIFICATE=fal
se
[3 OK] $ cd /workspace/695c70d1-2a1d-42d8-92e5-08de9b4346e4
[4 OK] $ trap 'cp -r ~/.aspire/logs /workspace/695c70d1-2a1d-42d8-92e5-08de9b4346e4/.aspire-logs 2>/dev/null; cp -r ~/.aspire/packages /workspace/695c70d1-2a1d-
42d8-92e5-08de9b4346e4/.aspire-packages 2>/dev/null' EXIT
[5 OK] $ /opt/aspire-scripts/get-aspire-cli.sh
Downloading aspire-cli-linux-x64.tar.gz from 'ga/daily'
######################################################################################################################################################### 100.0%
Downloading aspire-cli-linux-x64.tar.gz.sha512 from 'ga/daily'
######################################################################################################################################################### 100.0%
Aspire CLI successfully installed to: /root/.aspire/bin/aspire
Path /root/.aspire/bin already exists in $PATH, skipping addition
To use the Aspire CLI in new terminal sessions, restart your terminal or run:
source /root/.bashrc
[6 OK] $ export PATH=~/.aspire/bin:$PATH
[7 OK] $ aspire new
Select a template: Starter App (ASP.NET Core/Blazor)
Enter the project name (695c70d1-2a1d-42d8-92e5-08de9b4346e4): AspireWaitApp
Enter the output path (./AspireWaitApp): ./AspireWaitApp
✔ Using Redis Cache for caching.
\ud83d\udce6 Using project templates version: 13.2.1
⚠ Developer certificates may not be fully trusted (trust exit code was: PartiallyFailedToTrustTheCertificate).
✔ Project created successfully in /workspace/695c70d1-2a1d-42d8-92e5-08de9b4346e4/AspireWaitApp.
Would you like to configure AI agent environments for this project? [y/n] (y): n
[8 OK] $ cd AspireWaitApp/AspireWaitApp.AppHost
[9 OK] $ aspire start
AppHost: AspireWaitApp.AppHost.csproj
Dashboard: https://localhost:17260/login?t=4530f9d7b3b3bce21f631ad23dbf78f2
Logs: /root/.aspire/logs/cli_20260403T222931462_detach-child_a33797943b4f4b4cacdb23a51426b4ab.log
PID: 661
✔ Apphost started successfully.
[10 OK] $
---- Hex1b.Automation.WaitUntilTimeoutException : WaitUntil timed out after 00:03:00 waiting for: text "AppHost started successfully." to appear
at Hex1bTerminalAutomator.cs:137
Terminal (160x48, cursor at 10,41, normal screen):
bash-5.2# CMDCOUNT=0; PROMPT_COMMAND='s=$?;((CMDCOUNT++));PS1="[$CMDCOUNT $([ $s -eq 0 ] && echo OK || echo ERR:$s)] \$ "'
[1 OK] $ umask 000
[2 OK] $ export ASPIRE_PLAYGROUND=true TERM=xterm DOTNET_CLI_TELEMETRY_OPTOUT=true DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true DOTNET_GENERATE_ASPNET_CERTIFICATE=fal
se
[3 OK] $ cd /workspace/695c70d1-2a1d-42d8-92e5-08de9b4346e4
[4 OK] $ trap 'cp -r ~/.aspire/logs /workspace/695c70d1-2a1d-42d8-92e5-08de9b4346e4/.aspire-logs 2>/dev/null; cp -r ~/.aspire/packages /workspace/695c70d1-2a1d-
42d8-92e5-08de9b4346e4/.aspire-packages 2>/dev/null' EXIT
[5 OK] $ /opt/aspire-scripts/get-aspire-cli.sh
Downloading aspire-cli-linux-x64.tar.gz from 'ga/daily'
######################################################################################################################################################### 100.0%
Downloading aspire-cli-linux-x64.tar.gz.sha512 from 'ga/daily'
######################################################################################################################################################### 100.0%
Aspire CLI successfully installed to: /root/.aspire/bin/aspire
Path /root/.aspire/bin already exists in $PATH, skipping addition
To use the Aspire CLI in new terminal sessions, restart your terminal or run:
source /root/.bashrc
[6 OK] $ export PATH=~/.aspire/bin:$PATH
[7 OK] $ aspire new
Select a template: Starter App (ASP.NET Core/Blazor)
Enter the project name (695c70d1-2a1d-42d8-92e5-08de9b4346e4): AspireWaitApp
Enter the output path (./AspireWaitApp): ./AspireWaitApp
✔ Using Redis Cache for caching.
\ud83d\udce6 Using project templates version: 13.2.1
⚠ Developer certificates may not be fully trusted (trust exit code was: PartiallyFailedToTrustTheCertificate).
✔ Project created successfully in /workspace/695c70d1-2a1d-42d8-92e5-08de9b4346e4/AspireWaitApp.
Would you like to configure AI agent environments for this project? [y/n] (y): n
[8 OK] $ cd AspireWaitApp/AspireWaitApp.AppHost
[9 OK] $ aspire start
AppHost: AspireWaitApp.AppHost.csproj
Dashboard: https://localhost:17260/login?t=4530f9d7b3b3bce21f631ad23dbf78f2
Logs: /root/.aspire/logs/cli_20260403T222931462_detach-child_a33797943b4f4b4cacdb23a51426b4ab.log
PID: 661
✔ Apphost started successfully.
[10 OK] $
Stack Trace:
at Hex1b.Automation.Hex1bTerminalAutomator.RunAndRecordAsync(Hex1bTerminalInputSequence sequence, String description, CancellationToken ct, String callerFilePath, Int32 callerLineNumber) in /home/runner/work/hex1b/hex1b/src/Hex1b/Automation/Hex1bTerminalAutomator.cs:line 665
at Hex1b.Automation.Hex1bTerminalAutomator.WaitUntilTextAsync(String text, Nullable`1 timeout, String callerFilePath, Int32 callerLineNumber) in /home/runner/work/hex1b/hex1b/src/Hex1b/Automation/Hex1bTerminalAutomator.cs:line 140
at Aspire.Cli.EndToEnd.Tests.WaitCommandTests.CreateStartWaitAndStopAspireProject() in /_/tests/Aspire.Cli.EndToEnd.Tests/WaitCommandTests.cs:line 52
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
at Hex1b.Automation.WaitUntilStep.ExecuteAsync(Hex1bTerminal terminal, Hex1bTerminalInputSequenceOptions options, CancellationToken ct) in /home/runner/work/hex1b/hex1b/src/Hex1b/Automation/WaitUntilStep.cs:line 39
at Hex1b.Automation.Hex1bTerminalInputSequence.ApplyAsync(Hex1bTerminal terminal, CancellationToken ct) in /home/runner/work/hex1b/hex1b/src/Hex1b/Automation/Hex1bTerminalInputSequence.cs:line 41
at Hex1b.Automation.Hex1bTerminalAutomator.RunAndRecordAsync(Hex1bTerminalInputSequence sequence, String description, CancellationToken ct, String callerFilePath, Int32 callerLineNumber) in /home/runner/work/hex1b/hex1b/src/Hex1b/Automation/Hex1bTerminalAutomator.cs:line 652Standard Output
Temporary workspace created at: /tmp/Aspire.Cli.Tests/TemporaryWorkspaces/695c70d1-2a1d-42d8-92e5-08de9b4346e4
Creating Docker test terminal:
Test name: CreateStartWaitAndStopAspireProject
Install mode: GaRelease
Variant: DotNet
Dockerfile: /home/runner/work/aspire/aspire/tests/Shared/Docker/Dockerfile.e2e
Workspace: /tmp/Aspire.Cli.Tests/TemporaryWorkspaces/695c70d1-2a1d-42d8-92e5-08de9b4346e4
Docker socket: True
Dimensions: 160x48
Recording: /home/runner/work/aspire/aspire/testresults/recordings/CreateStartWaitAndStopAspireProject.cast
Description
The WaitCommandTests.WaitCommand_WaitsForResourceToBeUp E2E test consistently times out when running inside Docker containers (the new Docker-based E2E test infrastructure introduced in #14974).
Root Cause
The test runs aspire run --detach to start a starter template AppHost, then uses aspire wait webfrontend --status up --timeout 300 to wait for the webfrontend resource to reach running state. In Docker-in-Docker environments (the Docker container mounts the host Docker socket), the nested container startup (redis, apiservice, webfrontend) is significantly slower than on bare metal.
Even with a 300-second timeout, the resources don't reach 'up' status. Other tests that use aspire run (StartStopTests, PsCommandTests, LogsCommandTests) pass because they don't wait for individual resource readiness.
Impact
This test is quarantined as it cannot reliably pass in the Docker-based E2E test infrastructure. The aspire wait command itself works correctly; the issue is purely infrastructure-related (Docker-in-Docker performance).
Possible Solutions
- Skip this test in Docker-based runs and only run it on bare metal
- Optimize Docker-in-Docker container startup performance
- Use a simpler AppHost with fewer resources for the wait test