Do not merge) Test cli sanity test fail on 1.0.48#317437
Draft
anthonykim1 wants to merge 18 commits into
Draft
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Copilot extension’s bundled @github/copilot CLI dependency and adds coverage to ensure Copilot CLI chat sessions work both in extension sanity tests and VS Code smoke tests, including CI wiring to provide required environment secrets.
Changes:
- Bump
@github/copilotfrom1.0.39to^1.0.48(plus lockfile updates). - Add a new smoke test that opens a Copilot CLI chat session in an editor and validates it returns a non-empty response.
- Add a Copilot CLI sanity test in the Copilot extension and update CI smoke-test steps to fetch Copilot automation secrets before running Electron smoke tests.
Show a summary per file
| File | Description |
|---|---|
| test/smoke/src/main.ts | Registers the new Copilot CLI smoke-test suite. |
| test/smoke/src/areas/chat/copilotCli.test.ts | Adds an opt-in UI smoke test for Copilot CLI chat in an editor session. |
| test/smoke/extensions/vscode-smoketest-ext-host/package.json | Declares a new smoke-test command for opening a Copilot CLI chat session. |
| test/smoke/extensions/vscode-smoketest-ext-host/extension.js | Implements the smoke-test command to enable needed settings and open the Copilot CLI session editor. |
| test/automation/src/chat.ts | Extends automation helpers to support chat interactions in panel vs editor locations and adds latest-response retrieval. |
| extensions/copilot/src/extension/chatSessions/copilotcli/vscode-node/test/copilotcli.sanity-test.ts | Adds Copilot CLI-specific sanity coverage (workbench command + participant streaming + follow-up). |
| extensions/copilot/package.json | Bumps @github/copilot version and includes minor formatting/encoding changes. |
| extensions/copilot/package-lock.json | Locks @github/copilot and platform packages to 1.0.48. |
| build/azure-pipelines/win32/steps/product-build-win32-test.yml | Fetches Copilot secrets and exports them before running Electron smoke tests on Windows. |
| build/azure-pipelines/linux/steps/product-build-linux-test.yml | Fetches Copilot secrets and exports them before running Electron smoke tests on Linux. |
| build/azure-pipelines/darwin/steps/product-build-darwin-test.yml | Fetches Copilot secrets and exports them before running Electron smoke tests on macOS. |
Copilot's findings
Files not reviewed (1)
- extensions/copilot/package-lock.json: Language not supported
- Files reviewed: 10/11 changed files
- Comments generated: 2
| const extension = vscode.extensions.getExtension('Github.copilot-chat'); | ||
| assert.ok(extension, 'Extension is not available'); | ||
| realContext = await extension.activate(); | ||
| assert.ok(realContext, '`extension.activate()` did not return context`'); |
|
|
||
| function createGitHubSession(token: string, kind: 'permissive' | 'any'): vscode.AuthenticationSession { | ||
| return { | ||
| id: token, |
Contributor
Author
|
~requires-eval-assessment terminalbench2,vscbench claude-opus-4.7,gpt-5 vscode-agent-host vscodeSettings:
chat.agentHost.enabled: true
chat.agentHost.customTerminalTool.enabled: false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.