Enable experiment-driven model selection in smoke-copilot workflow#32255
Merged
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
Add model experiment variant in smoke-copilot workflow
Enable experiment-driven model selection in smoke-copilot workflow
May 15, 2026
Copilot created this pull request from a session on behalf of
pelikhan
May 15, 2026 02:26
View session
pelikhan
approved these changes
May 15, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an experiment dimension to route the smoke Copilot sub-agent model through experiment-selected variants.
Changes:
- Adds
subagent_modelexperiment variants (small,large). - Updates
file-summarizermodel selection to reference the selected experiment value. - Regenerates the compiled workflow lock file with experiment wiring.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/smoke-copilot.md |
Adds the model experiment and wires it into the inline sub-agent definition. |
.github/workflows/smoke-copilot.lock.yml |
Regenerated workflow reflecting the added experiment outputs and prompt environment bindings. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 1
| ## agent: `file-summarizer` | ||
| --- | ||
| model: small | ||
| model: ${{ experiments.subagent_model }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New Feature
What does this feature do?
This PR adds a model A/B experiment to an existing agentic workflow and routes a
modelfield through the selected experiment variant. It validates that workflow model selection can be parameterized byexperiments.*values.Why is this feature needed?
We need explicit coverage for experiment-based model assignment to ensure model configuration is not static and can participate in controlled variant rollouts.
Implementation details
Added model experiment dimension
Introduced a new
experiments.subagent_modelvariant set in.github/workflows/smoke-copilot.md:smalllargeBound model field to experiment output
Updated the
file-summarizersub-agent definition to use the experiment-selected model via expression-based assignment.Regenerated compiled workflow
Recompiled
smoke-copilotso the generated lock file reflects experiment wiring and runtime expression expansion.