Skip to content

agentHost: harden integration test timeouts - #329099

Merged
roblourens merged 2 commits into
mainfrom
roblou/agents/failed-tests-investigation-461453
Aug 5, 2026
Merged

agentHost: harden integration test timeouts#329099
roblourens merged 2 commits into
mainfrom
roblou/agents/failed-tests-investigation-461453

Conversation

@roblourens

Copy link
Copy Markdown
Member

Summary

  • use extended protocol-operation and graceful-shutdown budgets on CI workers
  • preserve fast timeout feedback for local macOS and Linux runs
  • apply CI-aware waits to the protocol and Copilot customization suites affected by Azure build 461453

Why

The failed Linux worker was abnormally slow: its filesystem diagnostics took 4m14s, compared with 35s on a successful comparison worker. Fixed 5–10 second waits caused otherwise responsive tests to time out, and the 5-second shutdown budget could force-kill the Agent Host before persistence completed, cascading into later failures.

This change uses the existing normal/extended timeout policy for CI, as was already done for Windows and coverage runs, while retaining the shorter local budgets.

Validation

  • npm run compile
  • npm run hygiene
  • affected Agent Host integration cluster: 6 consecutive CI-mode passes (42 passing, 2 pending)
  • affected cluster with local timeout selection: 42 passing, 2 pending

The broader Agent Host integration run passed the affected tests but later encountered a separate Copilot idle-release assertion on current main.

(Written by Copilot)

Use extended operation, shutdown, and watcher budgets on CI while preserving fast local timeouts. This prevents constrained workers from force-killing the test server and cascading one slowdown across later suites. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 5, 2026 03:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens Agent Host integration tests against slow CI workers while retaining shorter local timeouts.

Changes:

  • Adds CI-aware protocol and shutdown timeout budgets.
  • Extends affected protocol and Copilot customization test waits.
  • Preserves existing local macOS/Linux timeout behavior.
Show a summary per file
File Description
serverIntegrationTestHelpers.ts Centralizes CI-aware operation, shutdown, and test timeouts.
copilotCustomizations.integrationTest.ts Extends customization-suite CI budgets.
turnExecution.integrationTest.ts Applies CI-aware turn test timeouts.
sessionLifecycle.integrationTest.ts Applies CI-aware lifecycle test timeouts.
sessionDiffs.integrationTest.ts Extends setup, test, and notification waits on CI.

Review details

  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/vs/platform/agentHost/test/node/serverIntegrationTestHelpers.ts Outdated
Keep default protocol waits short, scope extended waits to the operations that were slow in Azure, and give their Mocha tests enough outer budget for request-specific timeout errors to surface. (Written by Copilot)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@roblourens
roblourens marked this pull request as ready for review August 5, 2026 17:39
@roblourens
roblourens enabled auto-merge (squash) August 5, 2026 17:39
@roblourens
roblourens merged commit a15228d into main Aug 5, 2026
29 checks passed
@roblourens
roblourens deleted the roblou/agents/failed-tests-investigation-461453 branch August 5, 2026 18:12
@vs-code-engineering vs-code-engineering Bot added this to the 1.133.0 milestone Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants