Skip to content

Add browser tool smoke tests#311123

Merged
kycutler merged 1 commit into
mainfrom
kycutler/browsertoolstests
Apr 18, 2026
Merged

Add browser tool smoke tests#311123
kycutler merged 1 commit into
mainfrom
kycutler/browsertoolstests

Conversation

@kycutler
Copy link
Copy Markdown
Contributor

No description provided.

Co-authored-by: Copilot <copilot@github.com>
Copilot AI review requested due to automatic review settings April 17, 2026 23:57
@kycutler kycutler self-assigned this Apr 17, 2026
@kycutler kycutler enabled auto-merge (squash) April 17, 2026 23:58
@github-actions
Copy link
Copy Markdown
Contributor

Screenshot Changes

Base: 96843429 Current: ea2345fc

Changed (28)

chat/aiCustomizations/aiCustomizationManagementEditor/McpBrowseMode/Light
Before After
before after
agentSessionsViewer/CompletedRead/Dark
Before After
before after
agentSessionsViewer/CompletedRead/Light
Before After
before after
agentSessionsViewer/CompletedUnread/Dark
Before After
before after
agentSessionsViewer/CompletedUnread/Light
Before After
before after
agentSessionsViewer/NeedsInput/Light
Before After
before after
agentSessionsViewer/FailedWithDuration/Dark
Before After
before after
agentSessionsViewer/FailedWithDuration/Light
Before After
before after
agentSessionsViewer/FailedWithoutDuration/Dark
Before After
before after
agentSessionsViewer/FailedWithoutDuration/Light
Before After
before after
agentSessionsViewer/WithDiffChanges/Dark
Before After
before after
agentSessionsViewer/WithDiffChanges/Light
Before After
before after
agentSessionsViewer/WithFileChangesList/Dark
Before After
before after
agentSessionsViewer/WithFileChangesList/Light
Before After
before after
agentSessionsViewer/WithBadge/Dark
Before After
before after
agentSessionsViewer/WithBadge/Light
Before After
before after
agentSessionsViewer/WithMarkdownBadge/Dark
Before After
before after
agentSessionsViewer/WithMarkdownBadge/Light
Before After
before after
agentSessionsViewer/WithDescription/Dark
Before After
before after
agentSessionsViewer/WithDescription/Light
Before After
before after
agentSessionsViewer/WithBadgeAndDiff/Dark
Before After
before after
agentSessionsViewer/WithBadgeAndDiff/Light
Before After
before after
agentSessionsViewer/CloudProvider/Dark
Before After
before after
agentSessionsViewer/CloudProvider/Light
Before After
before after
agentSessionsViewer/BackgroundProvider/Dark
Before After
before after
agentSessionsViewer/BackgroundProvider/Light
Before After
before after
agentSessionsViewer/ClaudeProvider/Dark
Before After
before after
agentSessionsViewer/ClaudeProvider/Light
Before After
before after

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new VS Code API integration test suite to smoke-test the integrated browser “chat tools” (open page, type, click, read, run Playwright), and updates the test workspace HTML to provide simple DOM elements for interaction.

Changes:

  • Add browser.tools.test.ts integration tests that enable browser chat tools + global auto-approve and invoke vscode.lm browser tools.
  • Extend testWorkspace/index.html with an input + button + output area to support tool-driven typing/clicking verification.
Show a summary per file
File Description
extensions/vscode-api-tests/testWorkspace/index.html Adds minimal UI elements and click handler for browser-tool interaction tests.
extensions/vscode-api-tests/src/singlefolder-tests/browser.tools.test.ts New integration suite exercising browser LM tools (open/type/click/read/run Playwright) with auto-approve enabled.

Copilot's findings

Comments suppressed due to low confidence (2)

extensions/vscode-api-tests/src/singlefolder-tests/browser.tools.test.ts:99

  • This test depends on reaching https://google.com/, which can be blocked in CI (no outbound internet), by enterprise policy, or by the agent network filter (when enabled, an empty allowlist blocks all domains). To keep the suite deterministic, prefer a local/file URL (already covered by the next test) or use a stable domain like example.com with explicit allowlisting when network filtering is active (and/or make the assertion tolerant of navigation failures).
	test('Open a page from the web', async function () {
		this.timeout(60000);

		const output = await invokeTool('open_browser_page', { url: 'https://google.com/' });

		assert.match(output, /Page ID:/, `Expected output to contain "Page ID:", got: ${output}`);
	});

extensions/vscode-api-tests/src/singlefolder-tests/browser.tools.test.ts:109

  • fileUrl is built from workspaceFolders[0].uri.fsPath. In remote integration runs, the workspace folder URI is typically remote (e.g. vscode-remote:) and the file path will not exist on the local UI machine where the integrated browser/Playwright runs, causing the browser page open to fail. Consider skipping this test when vscode.env.remoteName is set, or avoid local file paths by using a data URL / about:blank + run_playwright_code to page.setContent(...) to create the test DOM.
		// Build a file:// URL to the test workspace's index.html
		const workspaceFolders = vscode.workspace.workspaceFolders;
		assert.ok(workspaceFolders && workspaceFolders.length > 0, 'Expected a workspace folder');
		const indexHtmlPath = path.join(workspaceFolders[0].uri.fsPath, 'index.html');
		const fileUrl = vscode.Uri.file(indexHtmlPath).toString();

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

@kycutler kycutler merged commit e60d312 into main Apr 18, 2026
30 checks passed
@kycutler kycutler deleted the kycutler/browsertoolstests branch April 18, 2026 00:19
@vs-code-engineering vs-code-engineering Bot added this to the 1.117.0 milestone Apr 18, 2026
@vs-code-engineering vs-code-engineering Bot locked and limited conversation to collaborators Jun 2, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants