Add vmr-codeflow-status Copilot skill by lewing · Pull Request #124109 · dotnet/runtime

lewing · 2026-02-06T22:15:53Z

Summary

Adds a new Copilot Agent Skill (.github/skills/vmr-codeflow-status/) that automates analysis of VMR codeflow PR health for dotnet repositories. Handles both backflow (VMR → product repos) and forward flow (product repos → VMR).

What it does

PR Analysis Mode:

Parses PR metadata — Extracts VMR commit, subscription ID, build info from the PR body
Validates snapshot — Cross-references PR body commit against branch "Backflow from" commit messages to detect stale metadata
Checks freshness — Compares PR's snapshot against the current source branch HEAD, showing how many commits behind and which repos have updates
Shows pending forward flow — For behind backflow PRs, finds open forward flow PRs that would close part of the freshness gap
Detects staleness & conflicts — Finds Maestro "codeflow cannot continue" warnings and "Conflict detected" messages (with conflicting files and resolve commands)
Analyzes PR commits — Categorizes commits as auto-updates vs manual (which would be lost on close/force-trigger)
Traces fixes (optional) — Checks if a specific fix PR has flowed through VMR into the codeflow PR
Recommends actions — Suggests force trigger, close/reopen, merge as-is, resolve conflicts, or wait, with ready-to-use darc commands

Missing Backflow Mode:

Discovers all branches with backflow subscriptions by examining recently merged PRs
Compares VMR branch HEAD against the last merged backflow commit
Flags branches where the VMR has moved but no new backflow PR exists
Warns when Maestro may be stuck (>6 hours since last merge with VMR ahead)

Example Prompts

"check the codeflow status of [release/11.0.1xx-preview1] Source code updates from dotnet/dotnet sdk#52727"
"is [release/11.0-preview1] Source code updates from dotnet/dotnet #124098 up to date with the VMR?"
"has the runtime revert in [release/11.0-preview1] Revert [browser] Use Runtime=NET for build tasks #123974 reached the codeflow PR [release/11.0.1xx-preview1] Source code updates from dotnet/dotnet sdk#52727?"
"why is this codeflow PR blocked?"
"are there any missing backflow PRs for dotnet/roslyn?"
"report on the codeflow status for dotnet/aspnetcore"
"what forward flow PRs are pending for the preview1 VMR branch?"

Motivation

Investigating stale codeflow PRs is a common and tedious task — checking VMR freshness, finding staleness warnings, tracing whether specific fixes have flowed through. This skill automates the entire workflow and provides actionable darc commands.

Tested against 10+ codeflow PRs across dotnet/sdk, dotnet/runtime, dotnet/roslyn, dotnet/aspnetcore, and dotnet/dotnet on multiple branches (main, preview1, release/10.0.x).

Files

SKILL.md — Skill definition with frontmatter (matching existing skills in this repo)
scripts/Get-CodeflowStatus.ps1 — Main analysis script (~750 lines)
references/vmr-codeflow-reference.md — Reference docs (VMR concepts, darc CLI, API endpoints)

Adds a Copilot Agent Skill that analyzes VMR codeflow PR health: - Parses PR metadata (subscription ID, VMR commit, build info) - Checks VMR freshness (compares PR snapshot vs branch HEAD) - Detects Maestro staleness warnings - Analyzes PR commits (auto-updates vs manual) - Traces specific fixes through VMR pipeline (-TraceFix) - Provides actionable recommendations with darc commands

Copilot

Pull request overview

Adds a new Copilot Agent Skill (vmr-codeflow-status) to automate health analysis of VMR backflow/codeflow PRs by querying GitHub PR metadata, VMR branch freshness, Maestro staleness warnings, and (optionally) tracing whether a specific fix has flowed through VMR.

Changes:

Introduces a PowerShell script to analyze codeflow PR freshness, staleness, commit makeup, and optional fix tracing.
Adds skill documentation (SKILL.md) and a VMR/codeflow reference doc explaining concepts, metadata formats, and darc commands.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.

File	Description
`.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1`	Implements the PR/VMR analysis logic and prints status + recommendations.
`.github/skills/vmr-codeflow-status/references/vmr-codeflow-reference.md`	Provides background/reference material for VMR codeflow concepts and tooling.
`.github/skills/vmr-codeflow-status/SKILL.md`	Defines the skill and documents usage parameters and interpretation.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

.github/skills/vmr-codeflow-status/SKILL.md

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

Bug fixes: - Use full SHAs in compare API calls (Get-ShortSha for display only) - Fix Write-Status 3rd arg bug that would throw on color parse - Remove dead code - Handle behind/diverged status in VMR compare, not just ahead - Remove extra frontmatter fields to match existing skills Simplifications: - Replace 3 separate REST calls with single gh pr view --json - Use raw content header for source-manifest.json (no Base64)

- Use merged_at instead of merged boolean for fix PR detection - URL-encode VMR branch names with / in API paths - Fix Accept header: application/vnd.github.raw (not raw+json) - Fix contradictory recommendation message to use compare status - Initialize behindBy/compareStatus at script scope

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

- Remove unused \ and \ variables - Join gh api / gh pr view output before ConvertFrom-Json for robustness

- Auto-detect flow direction from PR title - Backflow: 'Source code updates from dotnet/dotnet' (VMR -> product repo) - Forward flow: 'Source code updates from dotnet/<repo>' (repo -> VMR) - Adjust freshness comparison to use correct source repo - Use flow-aware labels (VMR Commit vs Source Commit, etc.)

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

…g, fix forward-flow manifest branch

… detection, branch inference

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

…backflow PRs

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

lewing · 2026-02-06T23:29:57Z

❯ what is the current status of codeflow for runtime

● skill(vmr-codeflow-status)

● Let me check the codeflow status for the runtime repository.

● Check runtime codeflow status
  $ & "C:\Users\lewing\.copilot\skills\vmr-codeflow-status\scripts\Get-CodeflowStatus.ps1" -Repository "dotnet/runtime"
  -CheckMissing
  └ 34 lines...

● Now let me get details on the two open backflow PRs.

● Check main branch codeflow PR
  $ & "C:\Users\lewing\.copilot\skills\vmr-codeflow-status\scripts\Get-CodeflowStatus.ps1" -Repository "dotnet/runtime"
  -PRNumber 124095
  └ 51 lines...

● Check preview1 codeflow PR
  $ & "C:\Users\lewing\.copilot\skills\vmr-codeflow-status\scripts\Get-CodeflowStatus.ps1" -Repository "dotnet/runtime"
  -PRNumber 124098
  └ 53 lines...
● Here's the current codeflow status for dotnet/runtime:

  -------------------------------------------------------------------------------------------------------------------

  Open Backflow PRs

  ┌───────────────────────┬─────────┬─────────────────┬───────────────────────┐
  │ Branch                │ PR      │ Status          │ VMR Behind            │
  ├───────────────────────┼─────────┼─────────────────┼───────────────────────┤
  │ main                  │ #124095 │ ⚠️ Needs update │ 33 commits behind VMR │
  ├───────────────────────┼─────────┼─────────────────┼───────────────────────┤
  │ release/11.0-preview1 │ #124098 │ ⚠️ Needs update │ 2 commits behind VMR  │
  └───────────────────────┴─────────┴─────────────────┴───────────────────────┘

  Missing Backflow PR

  ┌──────────────┬────────────────────────┬───────────────────────────────────────────────────────────────────────────
  ─────┐
  │ Branch       │ Issue                  │ Details
       │
  ├──────────────┼────────────────────────┼───────────────────────────────────────────────────────────────────────────
  ─────┤
  │ release/10.0 │ 🚨 Missing backflow PR │ VMR is 33 commits ahead, last PR merged ~22.6 hours ago — Maestro may be
  stuck │
  └──────────────┴────────────────────────┴───────────────────────────────────────────────────────────────────────────
  ─────┘

  Details

   - main (#124095): Created ~6 hours ago, no staleness warnings, no manual commits. Missing updates from 11 repos
  (aspnetcore, nuget, razor, runtime, efcore, roslyn, sdk, etc.). Maestro should auto-update, or you can trigger
  manually: darc trigger-subscriptions --id f7901f87-9f24-40d6-9bc1-564863937237
   - release/11.0-preview1 (#124098): Only 2 commits behind (missing fsharp updates), no staleness. 5 pending forward
  flow PRs into VMR. Should auto-update soon, or trigger: darc trigger-subscriptions --id
  96ab39a6-78f4-4265-9e1a-76875a9d2b5f
   - release/10.0: ⚠️ No open backflow PR exists despite VMR being 33 commits ahead. The last merged PR was ~22.6
  hours ago — this suggests Maestro may be stuck or the subscription needs investigation.

…istently

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

…extraction

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

.github/skills/vmr-codeflow-status/SKILL.md

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

…ensitive commit SHA parsing, align SKILL.md commands

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1

…in CheckMissing

lewing · 2026-02-07T02:36:02Z

@copilot Check the codeflow status of dotnet/sdk#52727 using the vmr-codeflow-status skill in this PR branch.

Copilot · 2026-02-07T02:36:11Z

@lewing I've opened a new pull request, #124122, to work on those changes. Once the pull request is ready, I'll request review from you.

lewing · 2026-02-07T02:38:37Z

@copilot Run the script at .github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1 with -PRNumber 52727 -Repository 'dotnet/sdk' and post the output as a comment on this PR. Do not open a new PR or make any code changes.

Copilot · 2026-02-07T02:38:47Z

@lewing I've opened a new pull request, #124123, to work on those changes. Once the pull request is ready, I'll request review from you.

lewing · 2026-02-07T02:42:23Z

Observation: Coding agent doesn't support diagnostic/reporting skills

We tested invoking this skill via @copilot in PR comments. Both attempts resulted in the coding agent opening new PRs (#124122, #124123 — both closed) rather than running the script and posting the output.

The Agent Skills documentation says:

Agent Skills work with Copilot coding agent, the GitHub Copilot CLI and agent mode in Visual Studio Code Insiders.

And:

When Copilot chooses to use a skill, the SKILL.md file will be injected in the agent's context, giving the agent access to your instructions. It can then follow those instructions, and use any scripts or examples you may have included.

The coding agent likely does load the skill and may even run the script internally, but its only output mechanism is opening a PR with code changes. There's no way for it to "just report back" with diagnostic output in a comment.

This creates a gap for diagnostic/reporting skills (like this codeflow status tool) that are designed to analyze and report rather than modify code. These skills work great in the CLI and VS Code agent mode, but the coding agent can't use them effectively because:

The output is informational, not a code change
The appropriate response is a comment, not a PR
Forcing results into a PR creates noise (unnecessary PRs that need to be closed)

Suggestion: The coding agent could support a "report" or "comment" action mode for skills that are diagnostic in nature, posting the script output as a PR/issue comment instead of always opening a new PR.

@copilot

jeffhandley

Bravo!

lewing · 2026-02-07T08:05:24Z

Follow-up: #124129 promotes eng/Version.Details.xml as the primary VMR snapshot source (instead of commit message parsing). This correctly handles manual backflow operations like darc vmr backflow where the commit message doesn't follow the standard format.

Copilot AI review requested due to automatic review settings February 6, 2026 22:15

github-actions bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Feb 6, 2026

dotnet-policy-service bot assigned lewing Feb 6, 2026

Copilot started reviewing on behalf of lewing February 6, 2026 22:16 View session

lewing marked this pull request as draft February 6, 2026 22:16

Copilot AI reviewed Feb 6, 2026

View reviewed changes

lewing added 2 commits February 6, 2026 16:28

Copilot AI review requested due to automatic review settings February 6, 2026 22:37

Copilot started reviewing on behalf of lewing February 6, 2026 22:38 View session

Copilot AI reviewed Feb 6, 2026

View reviewed changes

Remove dead code and join output before ConvertFrom-Json

fb581d2

- Remove unused \ and \ variables - Join gh api / gh pr view output before ConvertFrom-Json for robustness

Copilot AI review requested due to automatic review settings February 6, 2026 22:47

Copilot started reviewing on behalf of lewing February 6, 2026 22:48 View session

lewing force-pushed the lewing/vmr-codeflow-skill branch from 7f18b9c to 3db9577 Compare February 6, 2026 22:50

Copilot AI reviewed Feb 6, 2026

View reviewed changes

lewing added 2 commits February 6, 2026 16:55

Address review: remove dead code, add gh preflight, truncation warnin…

6a97bd0

…g, fix forward-flow manifest branch

Fix forward-flow TraceFix snapshot ref, stderr capture, maestro login…

376b625

… detection, branch inference

Copilot AI review requested due to automatic review settings February 6, 2026 23:00

Copilot started reviewing on behalf of lewing February 6, 2026 23:01 View session

Copilot AI reviewed Feb 6, 2026

View reviewed changes

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1 Outdated Show resolved Hide resolved

lewing added 2 commits February 6, 2026 17:10

Add try/catch for manifest JSON parsing, use Get-ShortSha consistently

155e8c0

Add -CheckMissing mode to detect expected but missing backflow PRs

1011269

Copilot AI review requested due to automatic review settings February 6, 2026 23:19

Copilot started reviewing on behalf of lewing February 6, 2026 23:20 View session

Show pending forward flow PRs that would close the freshness gap for …

2b2fb20

…backflow PRs

Copilot AI reviewed Feb 6, 2026

View reviewed changes

.github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1 Outdated Show resolved Hide resolved

Handle compare status 'identical', initialize usedBranchSnapshot cons…

79eabf5

…istently

Copilot AI review requested due to automatic review settings February 7, 2026 00:11

Copilot started reviewing on behalf of lewing February 7, 2026 00:12 View session

Copilot AI reviewed Feb 7, 2026

View reviewed changes

lewing added 2 commits February 6, 2026 18:18

Detect manual codeflow-like commits when flow is paused

5e1544c

Fix \r\n in regexes, case-insensitive hex, deduplicate conflict file …

d3ec37f

…extraction

Copilot AI review requested due to automatic review settings February 7, 2026 00:21

Copilot started reviewing on behalf of lewing February 7, 2026 00:22 View session

Copilot AI reviewed Feb 7, 2026

View reviewed changes

Fix UTC time parsing, capture full resolve-conflict command, case-ins…

57559cd

…ensitive commit SHA parsing, align SKILL.md commands

lewing marked this pull request as ready for review February 7, 2026 02:01

Copilot AI review requested due to automatic review settings February 7, 2026 02:01

Copilot started reviewing on behalf of lewing February 7, 2026 02:01 View session

Copilot AI reviewed Feb 7, 2026

View reviewed changes

Fix 2>&1 stderr corruption in gh search, prefix-match SHA comparison …

f00d4b6

…in CheckMissing

Copilot AI mentioned this pull request Feb 7, 2026

[WIP] Add vmr-codeflow-status Copilot skill #124122

Closed

Copilot AI mentioned this pull request Feb 7, 2026

[WIP] Add vmr-codeflow-status Copilot skill #124123

Closed

jeffhandley approved these changes Feb 7, 2026

View reviewed changes

lewing merged commit 2d638dc into main Feb 7, 2026
17 checks passed

lewing deleted the lewing/vmr-codeflow-skill branch February 7, 2026 03:37

lewing added area-skills Agent Skills and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Feb 7, 2026

lewing mentioned this pull request Feb 7, 2026

[release/11.0.1xx-preview1] Source code updates from dotnet/dotnet dotnet/sdk#52727

Open

lewing mentioned this pull request Feb 7, 2026

[skill] Use Version.Details.xml as VMR snapshot fallback for manual backflow #124129

Open

Conversation

lewing commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What it does

Example Prompts

Motivation

Files

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

lewing commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lewing commented Feb 7, 2026

Uh oh!

Copilot AI commented Feb 7, 2026

Uh oh!

lewing commented Feb 6, 2026 •

edited

Loading

lewing commented Feb 6, 2026 •

edited

Loading

lewing commented Feb 7, 2026 •

edited

Loading