agentHost: gate flaky Claude file deletion replay - #334691
Draft
roblourens wants to merge 1 commit into
Draft
Conversation
Skip the unstable Windows replay variant while preserving recording and other platform coverage. Include Agent Host logs in failed E2E output so provider process exits expose their diagnostics. (Written by Copilot) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Reviewer readiness is mixed, and the new gate remains missing from the harness reference table.
Review tier: Balanced
Findings: None
What changed in this PR
Gates flaky Claude file-deletion replay on Windows and improves E2E failure diagnostics.
Changes:
- Skips the affected Windows replay while preserving other coverage.
- Prints Agent Host logs after failed tests.
- Documents the known failure and reproduction steps.
| File | Description |
|---|---|
src/vs/platform/agentHost/test/node/e2e/suites/fileOperationsSuite.ts |
Applies the targeted replay gate. |
src/vs/platform/agentHost/test/node/e2e/suites/agentHostE2ESuites.ts |
Clarifies diagnostic logging behavior. |
src/vs/platform/agentHost/test/node/e2e/README.md |
Updates troubleshooting guidance. |
src/vs/platform/agentHost/test/node/e2e/providers/claudeAgentHostE2E.integrationTest.ts |
Enables the Claude-specific gate. |
src/vs/platform/agentHost/test/node/e2e/KNOWN_ISSUES.md |
Documents the Windows failure and reproduction. |
src/vs/platform/agentHost/test/node/e2e/harness/agentHostE2ETestHarness.ts |
Adds gate configuration and Agent Host log output. |
Suppressed comments (1)
src/vs/platform/agentHost/test/node/e2e/harness/agentHostE2ETestHarness.ts:388
- Document this new per-test gate in the README’s “Provider config & per-test gates” table. That table is the E2E harness reference for targeted platform/provider gates, but it currently lists the neighboring Linux shell and Windows subagent gates without this Windows file-delete gate, so maintainers cannot discover its replay-versus-recording behavior from the harness documentation.
/** Provider's file-delete shell turn can terminate its bundled runtime during Windows replay. */
readonly fileDeleteReplayUnstableOnWindows?: boolean;
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
deletes a workspace filedeterministic replay on Windows while retaining recording and all other platform/provider coverageSource of flakiness
The failing run in #334648 reached the recorded delete turn, then the bundled Claude runtime exited with code 1. The replay fixture and portable
nodecommand are deterministic, and adjacent rename and shell-command scenarios completed on the same Windows worker. This identifies the provider subprocess as the source rather than the filesystem assertion, replay proxy, or command portability.The unchanged Windows job rerun subsequently passed, confirming the intermittent nature of the runtime exit.
Validation
npm run transpile-client./scripts/test-integration.sh --run src/vs/platform/agentHost/test/node/e2e/providers/claudeAgentHostE2E.integrationTest.ts --grep "deletes a workspace file"git diff --check(Written by Copilot)