Increase timeouts for the smoke tests#320130
Merged
Merged
Conversation
The Copilot CLI smoke tests were failing because overrideAuthType was not set, causing the CLI SDK to use HMAC auth (the default) against the mock LLM server. The mock server cannot validate HMAC signatures, so /models calls returned 401, model fetching failed, and prompt rendering threw "Unexpected generated prompt structure". Add the missing ['github.copilot.advanced.debug.overrideAuthType', '"token"'] setting to both chatSessions.test.ts and copilotCli.test.ts (matching what agentsWindow.test.ts already does), and remove the it.skip that was added as a temporary workaround. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adjusts smoke-test configuration to make Copilot CLI / Chat Sessions flows more reliable under slow startup conditions, primarily by increasing timeouts and ensuring the mock server can be called without HMAC validation.
Changes:
- Increased smoke test timeouts for Chat Sessions and for command availability in the smoke-test extension host.
- Updated smoke test user settings to force token auth when redirecting Copilot traffic to a mock server.
- Re-enabled previously skipped Copilot CLI-related smoke tests.
Show a summary per file
| File | Description |
|---|---|
| test/smoke/src/areas/chat/copilotCli.test.ts | Adds token-auth override setting and re-enables the Copilot CLI editor smoke test. |
| test/smoke/src/areas/chat/chatSessions.test.ts | Increases per-test timeout, adds token-auth override setting, and re-enables the Copilot CLI session test. |
| test/smoke/extensions/vscode-smoketest-ext-host/extension.js | Doubles the wait-for-command timeout to reduce flakes during command registration. |
Copilot's findings
Comments suppressed due to low confidence (1)
test/smoke/src/areas/chat/chatSessions.test.ts:87
- This change re-enables a previously skipped smoke test. The PR title is about increasing timeouts, but this also changes test coverage/execution in CI (which can affect runtime and stability). Please call this out in the PR title/description, or keep the test skipped if it isn't ready to run by default yet.
it('Test Copilot CLI session', async function () {
- Files reviewed: 2/2 changed files
- Comments generated: 0
Yoyokrazy
approved these changes
Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.