Fix migrate-static-to-wrapper skill activation on evals

[Evangelink]

Problem

migrate-static-to-wrapper was failing skill activation on the eval, and its routing overlapped with the sibling skill generate-testability-wrappers.

Changes

1. Isolated activation fix — sharpened the frontmatter description boundary so migration prompts (where the abstraction is already registered in DI) reliably activate migrate-static-to-wrapper instead of generate-testability-wrappers. Isolated activation is now ✅ across all scenarios.
2. Plugin-arm routing partition — eval artifacts showed generate-testability-wrappers out-routing (and even stealing) migration scenarios in plugin mode. Rewrote both descriptions around the real discriminator — does the abstraction already exist / is it registered? (migrate = replace existing call sites; generate = first-time wrapper creation/registration). Plugin-arm activation for the migration scenarios is now materially improved and generate no longer steals them.
3. Ambient-context content gap — added a concrete "Static classes: use ambient context" section (a static settable TimeProvider seam defaulting to TimeProvider.System, overridden in tests) so the genuinely-hard static-class scenario reaches the rubric's expected answer instead of recommending a non-static rewrite (2.0/5 → 2.7/5).
4. Fixture headroom — added a realistic local-vs-UTC pitfall (DateTime.Now alongside DateTime.UtcNow) to the CouponService fixture plus a matching rubric item, so migrations that conflate local and UTC time are penalized.

About the remaining ❌ scenario cells

The evaluation gate passes (evaluation-status: pass). The residual red cells are not activation bugs — analysis of the results.json artifacts shows they are the skill-validator's documented Pattern #8 (baseline already good, no headroom) plus Pattern #3 (high run-to-run variance, CV up to ~480%):

• On the simple migration scenarios the no-skill baseline already scores 5.0/5 (Opus-4.6 does a mechanical migration perfectly unaided), so loading the skill can't raise quality — it only adds token/time/tool overhead, yielding a marginally negative weighted score despite a flawless migration.
• migrate-static-to-wrapper sits ~9,200 chars into the 15,000-char alphabetical skill-menu budget, so it is not truncated; plugin-arm flips are single-run routing variance, which the footnotes themselves suggest addressing with --runs 5.

Per the guide's "improving the skill vs. gaming the eval" guidance, no further description edits are warranted for these saturated/noisy cases.

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Evangelink]

/evaluate

[Copilot]

Pull request overview

Updates skill descriptions in the dotnet-test plugin to reduce routing overlap and ensure the migrate-static-to-wrapper skill activates reliably in eval scenarios where the abstraction (e.g., TimeProvider) is already available via DI.

Changes:

• Expanded migrate-static-to-wrapper “USE FOR” triggers to include “replace call sites / add constructor parameter when already registered in DI” and “scoped files only” phrasing.
• Tightened generate-testability-wrappers “DO NOT USE” guidance to explicitly hand off to migrate-static-to-wrapper when the abstraction already exists or is already registered in DI.

Show a summary per file

File	Description
plugins/dotnet-test/skills/migrate-static-to-wrapper/SKILL.md	Adds more specific activation phrases to better target call-site migration when DI registration already exists and for scoped-file-only migrations.
plugins/dotnet-test/skills/generate-testability-wrappers/SKILL.md	Narrows the “DO NOT USE” guidance to reduce overlap and route existing-abstraction call-site migrations to the correct skill.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 2/2 changed files
• Comments generated: 0

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
generate-testability-wrappers	Generate TimeProvider adoption for DateTime.UtcNow	4.0/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill, bash / ✅ generate-testability-wrappers; tools: skill, glob	🟡 0.23	✅
generate-testability-wrappers	Generate custom Environment wrapper	3.0/5 → 4.0/5 🟢	✅ generate-testability-wrappers; tools: skill, glob	🟡 0.23	✅
generate-testability-wrappers	Recommend System.IO.Abstractions for file system calls	2.3/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill, report_intent, glob, view	🟡 0.23	✅
generate-testability-wrappers	Decline wrapper generation for already-abstracted code	2.7/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill / ℹ️ not activated (expected)	🟡 0.23	✅ [1]
generate-testability-wrappers	Provide ambient context alternative when DI is unavailable	2.3/5 → 3.3/5 🟢	✅ generate-testability-wrappers; tools: report_intent, skill, view, glob, create, edit / ⚠️ NOT ACTIVATED	🟡 0.23	❌ [2]
migrate-static-to-wrapper	Migrate DateTime.UtcNow to TimeProvider in a service class	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill, bash / ⚠️ NOT ACTIVATED	✅ 0.10	❌ [3]
migrate-static-to-wrapper	Migrate only in scoped files, leaving others untouched	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill, bash / ⚠️ NOT ACTIVATED	✅ 0.10	❌ [4]
migrate-static-to-wrapper	Update test doubles when migrating a service to TimeProvider	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill / ⚠️ NOT ACTIVATED	✅ 0.10	❌ [5]
migrate-static-to-wrapper	Handle a genuinely static class that cannot take constructor injection	2.0/5 → 2.0/5	⚠️ NOT ACTIVATED	✅ 0.10	❌ [6]
migrate-static-to-wrapper	Decline migration when wrapper does not exist yet	3.3/5 → 4.3/5 🟢	✅ migrate-static-to-wrapper; tools: skill	✅ 0.10	✅ [7]

[1] ⚠️ High run-to-run variance (CV=56%) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=146%) — consider re-running with --runs 5
[3] (Plugin) Quality unchanged but weighted score is -2.5% due to: tokens (68513 → 104639)
[4] ⚠️ High run-to-run variance (CV=51%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -17.3% due to: judgment, time (33.7s → 62.9s), tokens (83238 → 119044), tool calls (10 → 14)
[5] (Plugin) Quality unchanged but weighted score is -2.2% due to: tokens (98446 → 138415)
[6] (Isolated) Quality unchanged but weighted score is -18.2% due to: judgment, quality, tool calls (3 → 4)
[7] ⚠️ High run-to-run variance (CV=177%) — consider re-running with --runs 5

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

To investigate failures, paste this to your AI coding agent:

For PR 864 in dotnet/skills, download eval artifacts with gh run download 28848798985 --repo dotnet/skills --pattern "skill-validator-results-*" --dir ./eval-results, then fetch https://raw.githubusercontent.com/dotnet/skills/28eb0889efb756b0d2288689e2b9a4462e0ca874/eng/skill-validator/src/docs/InvestigatingResults.md and follow it to analyze the results.json files. Diagnose each failure, suggest fixes to the eval.yaml and skill content, and tell me what to fix first.

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
generate-testability-wrappers	Generate TimeProvider adoption for DateTime.UtcNow	5.0/5 → 5.0/5	✅ generate-testability-wrappers; tools: report_intent, skill, view, edit, bash / ✅ generate-testability-wrappers; tools: skill, report_intent, grep, view, edit	🟡 0.23	❌ [1]
generate-testability-wrappers	Generate custom Environment wrapper	3.0/5 → 4.0/5 🟢	✅ generate-testability-wrappers; tools: skill, glob, bash	🟡 0.23	✅
generate-testability-wrappers	Recommend System.IO.Abstractions for file system calls	2.7/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill, report_intent, glob, view	🟡 0.23	✅ [2]
generate-testability-wrappers	Decline wrapper generation for already-abstracted code	2.3/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill / ℹ️ not activated (expected)	🟡 0.23	✅ [3]
generate-testability-wrappers	Provide ambient context alternative when DI is unavailable	3.3/5 → 4.7/5 🟢	✅ generate-testability-wrappers; tools: report_intent, skill, view, glob, create, edit / ⚠️ NOT ACTIVATED	🟡 0.23	❌
migrate-static-to-wrapper	Migrate DateTime.UtcNow to TimeProvider in a service class	4.7/5 → 5.0/5 🟢	✅ migrate-static-to-wrapper; tools: skill, bash / ⚠️ NOT ACTIVATED	✅ 0.08	❌ [4]
migrate-static-to-wrapper	Migrate only in scoped files, leaving others untouched	5.0/5 → 4.7/5 🔴	✅ migrate-static-to-wrapper; tools: skill / ✅ migrate-static-to-wrapper; tools: skill, bash	✅ 0.08	❌
migrate-static-to-wrapper	Update test doubles when migrating a service to TimeProvider	5.0/5 → 4.0/5 🔴	✅ migrate-static-to-wrapper; tools: glob, skill / ⚠️ NOT ACTIVATED	✅ 0.08	❌ [5]
migrate-static-to-wrapper	Handle a genuinely static class that cannot take constructor injection	2.0/5 → 2.7/5 🟢	✅ migrate-static-to-wrapper; tools: skill, edit, bash, glob / ⚠️ NOT ACTIVATED	✅ 0.08	✅ [6]
migrate-static-to-wrapper	Decline migration when wrapper does not exist yet	3.3/5 → 5.0/5 🟢	✅ migrate-static-to-wrapper; tools: skill, glob	✅ 0.08	✅ [7]

[1] (Plugin) Quality unchanged but weighted score is -10.0% due to: tokens (12829 → 65641), tool calls (0 → 4), time (12.3s → 29.7s)
[2] ⚠️ High run-to-run variance (CV=59%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=74%) — consider re-running with --runs 5
[4] ⚠️ High run-to-run variance (CV=87%) — consider re-running with --runs 5. (Isolated) Quality improved but weighted score is -19.1% due to: judgment, tokens (83126 → 122401), time (35.2s → 63.4s), quality, tool calls (10 → 14)
[5] ⚠️ High run-to-run variance (CV=141%) — consider re-running with --runs 5
[6] ⚠️ High run-to-run variance (CV=4046%) — consider re-running with --runs 5
[7] ⚠️ High run-to-run variance (CV=199%) — consider re-running with --runs 5

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

To investigate failures, paste this to your AI coding agent:

For PR 864 in dotnet/skills, download eval artifacts with gh run download 28850458039 --repo dotnet/skills --pattern "skill-validator-results-*" --dir ./eval-results, then fetch https://raw.githubusercontent.com/dotnet/skills/c328c97acf355367c1655b3cbcb3abdcfa28cf3c/eng/skill-validator/src/docs/InvestigatingResults.md and follow it to analyze the results.json files. Diagnose each failure, suggest fixes to the eval.yaml and skill content, and tell me what to fix first.

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions

[github-actions]

✅ Evaluation passed for c328c97. cc @dotnet/dotnet-testing — please review.

[Evangelink]

/evaluate

[Copilot]

Copilot's findings

• Files reviewed: 4/4 changed files
• Comments generated: 5

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
generate-testability-wrappers	Generate TimeProvider adoption for DateTime.UtcNow	4.3/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: report_intent, skill / ✅ generate-testability-wrappers; tools: skill, report_intent, glob, view, edit, bash	🟡 0.22	✅ [1]
generate-testability-wrappers	Generate custom Environment wrapper	3.0/5 → 4.7/5 🟢	✅ generate-testability-wrappers; tools: skill, glob, bash / ✅ generate-testability-wrappers; tools: skill, bash	🟡 0.22	✅
generate-testability-wrappers	Recommend System.IO.Abstractions for file system calls	2.3/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill, report_intent, glob, view / ✅ generate-testability-wrappers; tools: skill	🟡 0.22	✅
generate-testability-wrappers	Decline wrapper generation for already-abstracted code	2.7/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill / ℹ️ not activated (expected)	🟡 0.22	✅ [2]
generate-testability-wrappers	Provide ambient context alternative when DI is unavailable	2.7/5 → 4.3/5 🟢	✅ generate-testability-wrappers; tools: report_intent, view, skill, glob, create, edit / ⚠️ NOT ACTIVATED	🟡 0.22	❌ [3]
migrate-static-to-wrapper	Migrate DateTime.UtcNow to TimeProvider in a service class	5.0/5 → 4.7/5 🔴	✅ migrate-static-to-wrapper; tools: skill, bash, grep / ✅ migrate-static-to-wrapper; tools: skill	✅ 0.11	❌
migrate-static-to-wrapper	Migrate only in scoped files, leaving others untouched	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill, bash, grep / ✅ migrate-static-to-wrapper; tools: skill, bash	✅ 0.11	❌ [4]
migrate-static-to-wrapper	Update test doubles when migrating a service to TimeProvider	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill / ⚠️ NOT ACTIVATED	✅ 0.11	❌ [5]
migrate-static-to-wrapper	Handle a genuinely static class that cannot take constructor injection	2.3/5 → 2.0/5 🔴	⚠️ NOT ACTIVATED	✅ 0.11	❌
migrate-static-to-wrapper	Decline migration when wrapper does not exist yet	4.0/5 → 5.0/5 🟢	✅ migrate-static-to-wrapper; tools: skill, glob	✅ 0.11	✅

[1] ⚠️ High run-to-run variance (CV=484%) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=71%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=70%) — consider re-running with --runs 5
[4] ⚠️ High run-to-run variance (CV=52%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -26.3% due to: judgment, quality, tokens (79510 → 139412), time (32.0s → 52.9s), tool calls (11 → 15)
[5] ⚠️ High run-to-run variance (CV=69%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -3.7% due to: tokens (98704 → 133048), quality

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

To investigate failures, paste this to your AI coding agent:

For PR 864 in dotnet/skills, download eval artifacts with gh run download 28863198615 --repo dotnet/skills --pattern "skill-validator-results-*" --dir ./eval-results, then fetch https://raw.githubusercontent.com/dotnet/skills/afe30ff2ecc9824f05665958d61b497e23e92949/eng/skill-validator/src/docs/InvestigatingResults.md and follow it to analyze the results.json files. Diagnose each failure, suggest fixes to the eval.yaml and skill content, and tell me what to fix first.

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions

[github-actions]

Skill Coverage Report

	Plugin	Skill	Covered	Coverage
✅	dotnet-test	generate-testability-wrappers	22/22	100%
✅	dotnet-test	migrate-static-to-wrapper	21/21	100%

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
generate-testability-wrappers	Generate TimeProvider adoption for DateTime.UtcNow	4.7/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill, report_intent, view, edit, bash / ✅ generate-testability-wrappers; tools: skill, report_intent, grep, glob, view, edit, bash	🟡 0.26	❌ [1]
generate-testability-wrappers	Generate custom Environment wrapper	3.0/5 → 4.0/5 🟢	✅ generate-testability-wrappers; tools: skill, glob, bash	🟡 0.26	✅
generate-testability-wrappers	Recommend System.IO.Abstractions for file system calls	3.7/5 → 4.7/5 🟢	✅ generate-testability-wrappers; tools: skill, report_intent, glob, view / ✅ generate-testability-wrappers; tools: skill	🟡 0.26	✅
generate-testability-wrappers	Decline wrapper generation for already-abstracted code	2.0/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill / ℹ️ not activated (expected)	🟡 0.26	✅ [2]
generate-testability-wrappers	Provide ambient context alternative when DI is unavailable	2.3/5 → 2.3/5	✅ generate-testability-wrappers; tools: skill, report_intent, view, glob, edit / ⚠️ NOT ACTIVATED	🟡 0.26	❌ [3]
migrate-static-to-wrapper	Migrate DateTime.UtcNow to TimeProvider in a service class	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill, bash, grep / ✅ migrate-static-to-wrapper; tools: skill, grep, bash	✅ 0.07	❌ [4]
migrate-static-to-wrapper	Migrate only in scoped files, leaving others untouched	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill, bash	✅ 0.07	❌ [5]
migrate-static-to-wrapper	Update test doubles when migrating a service to TimeProvider	5.0/5 → 5.0/5	✅ migrate-static-to-wrapper; tools: skill / ✅ migrate-static-to-wrapper; tools: glob, skill	✅ 0.07	❌ [6]
migrate-static-to-wrapper	Handle a genuinely static class that cannot take constructor injection	2.0/5 → 2.0/5	⚠️ NOT ACTIVATED	✅ 0.07	❌ [7]
migrate-static-to-wrapper	Decline migration when wrapper does not exist yet	2.7/5 → 5.0/5 🟢	✅ migrate-static-to-wrapper; tools: skill, glob / ✅ migrate-static-to-wrapper; tools: skill	✅ 0.07	✅ [8]

[1] ⚠️ High run-to-run variance (CV=178%) — consider re-running with --runs 5. (Plugin) Quality improved but weighted score is -5.1% due to: tokens (12801 → 188491), tool calls (0 → 12), time (11.8s → 53.5s)
[2] ⚠️ High run-to-run variance (CV=53%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=71%) — consider re-running with --runs 5
[4] (Isolated) Quality unchanged but weighted score is -22.0% due to: judgment, quality, tokens (83160 → 130570), time (35.4s → 60.9s), tool calls (10 → 17)
[5] ⚠️ High run-to-run variance (CV=80%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -18.9% due to: judgment, tokens (84338 → 136613), time (38.5s → 75.4s), tool calls (11 → 17)
[6] ⚠️ High run-to-run variance (CV=82%) — consider re-running with --runs 5. (Plugin) Quality unchanged but weighted score is -3.4% due to: tokens (103573 → 167610)
[7] (Plugin) Quality unchanged but weighted score is -12.6% due to: quality, tokens (38174 → 64456), time (18.0s → 28.2s), tool calls (3 → 4)
[8] ⚠️ High run-to-run variance (CV=69%) — consider re-running with --runs 5

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

To investigate failures, paste this to your AI coding agent:

For PR 864 in dotnet/skills, download eval artifacts with gh run download 28864713055 --repo dotnet/skills --pattern "skill-validator-results-*" --dir ./eval-results, then fetch https://raw.githubusercontent.com/dotnet/skills/e6e4ee4f3360febabdfa374c16fe4aa6ed17e71b/eng/skill-validator/src/docs/InvestigatingResults.md and follow it to analyze the results.json files. Diagnose each failure, suggest fixes to the eval.yaml and skill content, and tell me what to fix first.

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions