Memoize getFirstNonCopilotModel in scenario automation endpoint provider#319638
Merged
Conversation
In scenario-automation runs against a BYOK no-auth user, `ScenarioAutomationEndpointProviderImpl.getChatEndpoint` is invoked many times per agent step (main model + utility + utility-small + tool detection + per-MCP-call). Each call hit `getFirstNonCopilotModel`, which always called `lm.selectChatModels()` (empty selector). The empty selector fans out across every registered language-model vendor and re-resolves each one — sustained ~4 Hz for an entire 10-minute agent turn in BYOK eval runs, ~2.4k empty-selector calls per turn, with 99.99 percent reporting `no changes`. Cache the first non-copilot model for the lifetime of the provider and invalidate on `lm.onDidChangeChatModels`, debounced through a `MicrotaskDelay` `Delayer` so synchronous bursts collapse into a single invalidation. The change listener is installed lazily on first cache use, so production paths that never hit the no-auth branch do not subscribe. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces scenario-automation overhead in the Copilot extension by memoizing the “first non-copilot” language model selection used by the no-auth/BYOK automation path, avoiding repeated lm.selectChatModels() fan-out calls.
Changes:
- Cache the first non-copilot
LanguageModelChatlookup for the lifetime ofScenarioAutomationEndpointProviderImpl. - Invalidate the cache on
lm.onDidChangeChatModels, debounced via a microtaskDelayerto coalesce bursty model-set updates. - Install the change listener lazily on first cache use to avoid unnecessary subscriptions on production paths.
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/extension/prompt/vscode-node/scenarioAutomationEndpointProviderImpl.ts | Adds memoization + debounced invalidation for resolving the first non-copilot chat model in scenario automation runs. |
Copilot's findings
- Files reviewed: 1/1 changed files
- Comments generated: 2
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
- Repair syntax error introduced by autofix (extra closing brace after the IIFE block in _resolveFirstNonCopilotModel that prematurely closed the class). - Add diagnostic logs so an eval bundle at default level shows the BYOK redirect engaging (info on resolve and on cache invalidation) and trace logs identifying which redirect branch ran for each request. - Promote the silent `no models found` throw to error and the capi-proxy family-resolve fallback from trace to warn so both are visible at default log level. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
rzhao271
approved these changes
Jun 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix https://github.com/microsoft/vscode-copilot-evaluation/issues/4075
In scenario-automation runs against a BYOK no-auth user,
ScenarioAutomationEndpointProviderImpl.getChatEndpointis invoked many times per agent step (main model + utility + utility-small + tool detection + per-MCP-call). Each call hitgetFirstNonCopilotModel, which always calledlm.selectChatModels()(empty selector). The empty selector fans out across every registered language-model vendor and re-resolves each one — sustained ~4 Hz for an entire 10-minute agent turn in BYOK eval runs, ~2.4k empty-selector calls per turn, with 99.99 percent reportingno changes.Cache the first non-copilot model for the lifetime of the provider and invalidate on
lm.onDidChangeChatModels, debounced through aMicrotaskDelayDelayerso synchronous bursts collapse into a single invalidation. The change listener is installed lazily on first cache use, so production paths that never hit the no-auth branch do not subscribe.