Skip to content

[Feature] Execution Subagent Custom Model#311602

Merged
vikramnitin9 merged 9 commits intomicrosoft:mainfrom
vikramnitin9:vikram/exec_subagent_custom_model
Apr 23, 2026
Merged

[Feature] Execution Subagent Custom Model#311602
vikramnitin9 merged 9 commits intomicrosoft:mainfrom
vikramnitin9:vikram/exec_subagent_custom_model

Conversation

@vikramnitin9
Copy link
Copy Markdown
Member

Tweaks to enable the use of a custom model with the execution subagent, in preparation for the Insiders flight this week.

  1. Added an option to use a custom copilot proxy endpoint, with an experiment-based flag.
  2. Updated the execution subagent prompt based on our experience training the custom model.
  3. Small gating change in some prompts to fix a bug where the string execution_subagent could potentially appear in the system prompt even when disabled.

Tagging @bhavyaus and @roblourens for review.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables running the Execution Subagent against a custom/agentic-proxy model (via experiment-based config), and updates system prompts to better align with the execution subagent’s tool-calling constraints.

Changes:

  • Added an experiment-based flag to route the execution subagent through the agentic proxy (with a default proxy model).
  • Updated execution subagent prompt instructions for stricter terminal tool usage.
  • Adjusted prompt gating so execution_subagent isn’t mentioned unless the tool is actually enabled.
Show a summary per file
File Description
extensions/copilot/src/platform/endpoint/node/proxyAgenticExecutionEndpoint.ts Adds a proxy-backed endpoint implementation for execution-subagent model routing.
extensions/copilot/src/platform/configuration/common/configurationService.ts Introduces ExecutionSubagentUseAgenticProxy experiment-based config key and clarifies model behavior.
extensions/copilot/src/extension/prompts/node/agent/executionSubagentPrompt.tsx Tightens execution subagent terminal tool rules and provides richer examples.
extensions/copilot/src/extension/prompts/node/agent/defaultAgentInstructions.tsx Prevents referencing the execution subagent tool name when it’s disabled.
extensions/copilot/src/extension/prompts/node/agent/anthropicPrompts.tsx Mirrors the same gating fix for Anthropic prompts and minor import reordering.
extensions/copilot/src/extension/prompt/node/executionSubagentToolCallingLoop.ts Adds agentic-proxy endpoint selection and model fallback logic for the execution subagent.
extensions/copilot/package.nls.json Adds localized description for the new setting and updates model-setting description.
extensions/copilot/package.json Contributes the new github.copilot.chat.executionSubagent.useAgenticProxy setting.

Copilot's findings

  • Files reviewed: 8/8 changed files
  • Comments generated: 2

Comment thread extensions/copilot/src/extension/prompt/node/executionSubagentToolCallingLoop.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 8/8 changed files
  • Comments generated: 3

Comment on lines +86 to +88
const modelName = this._configurationService.getExperimentBasedConfig(ConfigKey.Advanced.ExecutionSubagentModel, this._experimentationService) as ChatEndpointFamily | undefined;
const useAgenticProxy = this._configurationService.getExperimentBasedConfig(ConfigKey.Advanced.ExecutionSubagentUseAgenticProxy, this._experimentationService);

Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ExecutionSubagentModel is read as a plain string setting but is cast to ChatEndpointFamily | undefined. Since ChatEndpointFamily is currently only 'copilot-base' | 'copilot-fast', this cast is unsound and hides invalid values at compile time. Consider treating the config value as string | undefined and explicitly narrowing/validating before calling endpointProvider.getChatEndpoint(...) (e.g., only call it when the value is one of the supported families), while still allowing arbitrary proxy model names when useAgenticProxy is enabled.

Copilot uses AI. Check for mistakes.
Comment thread extensions/copilot/src/platform/endpoint/node/proxyAgenticExecutionEndpoint.ts Outdated
Comment on lines +89 to +93
if (useAgenticProxy) {
// Use agentic proxy with ExecutionSubagentModel or default to DEFAULT_AGENTIC_PROXY_MODEL
const agenticProxyModel = modelName || ExecutionSubagentToolCallingLoop.DEFAULT_AGENTIC_PROXY_MODEL;
return this.instantiationService.createInstance(ProxyAgenticExecutionEndpoint, agenticProxyModel);
}
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When useAgenticProxy is enabled, getEndpoint() creates a new ProxyAgenticExecutionEndpoint instance on every call. This method is invoked from buildPrompt, getAvailableTools, and fetch, so the subagent may construct multiple endpoint instances per invocation/turn. Consider memoizing the created endpoint on the loop instance (or routing through IEndpointProvider caching) to avoid repeated instantiation and potential extra per-endpoint setup work.

See below for a potential fix:

	private endpointPromise: ReturnType<IEndpointProvider['getChatEndpoint']> | undefined;

	/**
	 * Get the endpoint to use for the execution subagent
	 */
	private async getEndpoint() {
		if (!this.endpointPromise) {
			this.endpointPromise = (async () => {
				const modelName = this._configurationService.getExperimentBasedConfig(ConfigKey.Advanced.ExecutionSubagentModel, this._experimentationService) as ChatEndpointFamily | undefined;
				const useAgenticProxy = this._configurationService.getExperimentBasedConfig(ConfigKey.Advanced.ExecutionSubagentUseAgenticProxy, this._experimentationService);

				if (useAgenticProxy) {
					// Use agentic proxy with ExecutionSubagentModel or default to DEFAULT_AGENTIC_PROXY_MODEL
					const agenticProxyModel = modelName || ExecutionSubagentToolCallingLoop.DEFAULT_AGENTIC_PROXY_MODEL;
					return this.instantiationService.createInstance(ProxyAgenticExecutionEndpoint, agenticProxyModel);
				}

				if (modelName) {
					try {
						// Try to get the specified model
						const endpoint = await this.endpointProvider.getChatEndpoint(modelName);
						if (endpoint.supportsToolCalls) {
							return endpoint;
						}

						this._logService.warn(`Configured execution subagent model ${modelName} does not support tool calls, falling back to main agent endpoint`);
						return await this.endpointProvider.getChatEndpoint(this.options.request);
					} catch (error) {
						// Model not available, fallback to main agent endpoint
						this._logService.warn(`Failed to get model ${modelName}, falling back to main agent endpoint: ${error}`);
						return await this.endpointProvider.getChatEndpoint(this.options.request);
					}
				}

				// No model name specified, use main agent endpoint
				return await this.endpointProvider.getChatEndpoint(this.options.request);
			})();
		}

		return await this.endpointPromise;

Copilot uses AI. Check for mistakes.
Comment thread extensions/copilot/src/extension/prompts/node/agent/executionSubagentPrompt.tsx Outdated
Comment thread extensions/copilot/src/extension/prompt/node/executionSubagentToolCallingLoop.ts Outdated
@vikramnitin9 vikramnitin9 requested a review from bhavyaus April 22, 2026 01:54
@bhavyaus bhavyaus force-pushed the vikram/exec_subagent_custom_model branch from 91535d2 to 9a2eff4 Compare April 22, 2026 17:40
}
}


Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra line

bhavyaus
bhavyaus previously approved these changes Apr 22, 2026
roblourens
roblourens previously approved these changes Apr 22, 2026
@vikramnitin9 vikramnitin9 dismissed stale reviews from roblourens and bhavyaus via bb9133f April 22, 2026 20:20
@vikramnitin9 vikramnitin9 requested a review from bhavyaus April 22, 2026 20:34
@vikramnitin9 vikramnitin9 force-pushed the vikram/exec_subagent_custom_model branch from 5903eac to 6f0c78a Compare April 22, 2026 20:38
@vikramnitin9 vikramnitin9 enabled auto-merge (squash) April 22, 2026 20:55
@vikramnitin9 vikramnitin9 merged commit b646274 into microsoft:main Apr 23, 2026
54 of 56 checks passed
@vs-code-engineering vs-code-engineering Bot added this to the 1.118.0 milestone Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants