feat(testing): support alternative semantic index endpoints#308394
Merged
feat(testing): support alternative semantic index endpoints#308394
Conversation
Updates to support semantic search with blackbird in MSBench. - ScenarioAutomationWorkspaceChunkSearchService: directly calls Blackbird localhost:4443 embeddings endpoint bypassing production strategies - ScenarioAutomationEndpointProviderImpl: add fallback to copilot-base when model family resolution fails in scenario automation - Move IWorkspaceChunkSearchService registration into isScenarioAutomation if/else branch in services.ts - Move IGithubCodeSearchService and IGithubAvailableEmbeddingTypesService registrations to shared (non-branched) section Moved from microsoft/vscode-copilot-chat#3947 Co-authored-by: gryan11 <415784+gryan11@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds scenario-automation-specific semantic search support by introducing a lightweight workspace chunk search service that queries a local Blackbird embeddings endpoint, and by adjusting service registrations and endpoint resolution behavior to better support MSBench runs.
Changes:
- Add
ScenarioAutomationWorkspaceChunkSearchServicethat directly callshttp://localhost:4443/api/embeddings/code/searchfor embeddings-based search. - Add a fallback in
ScenarioAutomationEndpointProviderImpl.getChatEndpointto usecopilot-basewhen resolving a model family fails in scenario automation. - Refactor service registrations so
IWorkspaceChunkSearchServiceis selected per scenario-automation vs production branch, while GitHub code search-related services are registered unconditionally.
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/platform/workspaceChunkSearch/node/scenarioAutomationWorkspaceChunkSearchService.ts | New scenario-automation chunk search implementation that calls a local embeddings endpoint and parses results into FileChunkAndScore. |
| extensions/copilot/src/extension/prompt/vscode-node/scenarioAutomationEndpointProviderImpl.ts | Adds try/catch fallback to copilot-base when model family resolution fails under scenario automation. |
| extensions/copilot/src/extension/extension/vscode-node/services.ts | Moves IWorkspaceChunkSearchService registration into the scenario-automation branch and hoists GitHub code search service registrations to the shared section. |
Copilot's findings
- Files reviewed: 3/3 changed files
- Comments generated: 5
Member
Author
|
@copilot apply changes based on the comments in this thread |
Agent-Logs-Url: https://github.com/microsoft/vscode/sessions/be3a88b1-d355-40f5-94df-a71c1d2429b6 Co-authored-by: rwoll <11915034+rwoll@users.noreply.github.com>
Contributor
Implemented the requested thread feedback in commit 227bd4b: wired cancellation into the fetch call, applied glob include/exclude filtering, switched |
Copilot
AI
changed the title
Add localhost embeddings support for scenario automation
Address review feedback for scenario automation workspace chunk search
Apr 7, 2026
mjbvz
approved these changes
Apr 8, 2026
Contributor
The vscode repo's strict TypeScript config requires override modifiers on methods that override base class members. Add them to all methods in TestFetcherService. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
mjbvz
previously approved these changes
Apr 8, 2026
Fix: add missing override modifiers in test mock class
Add 'override' keyword to members that override base class members: - TestGitService.repositories (overrides IGitService) - TestLogService.trace (overrides ILogService) - TestLogService.error (overrides ILogService) Fixes TS4114 compilation errors in Copilot CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TylerLeonhardt
approved these changes
Apr 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This follow-up tightens the scenario-automation Blackbird path added in this PR by aligning behavior with production expectations from review feedback. It closes gaps around glob filtering, cancellation propagation, and robustness of response handling, and adds focused unit coverage for this integration point.
Service behavior updates
ScenarioAutomationWorkspaceChunkSearchService.searchFileChunksnow appliesWorkspaceChunkSearchOptions.globPatterns(include/exclude) before emitting chunks.IFetcherService.makeAbortController()andsignal, with graceful empty-result returns on abort.{ chunks: [] }) with error logging parity to other failure paths.CancellationTokenimport was converted to type-only in this file.New targeted unit tests
scenarioAutomationWorkspaceChunkSearchService.spec.tsunderplatform/workspaceChunkSearch/test/node/.