-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Add release notes agentic workflow #10334
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
67 commits
Select commit
Hold shift + click to select a range
6aa3045
Add release notes agentic workflow and skill references
richlander 3d196a2
Add release notes system design document
richlander b94d89f
Update release notes skill docs to match actual dotnet-release CLI
richlander c8ba518
Scope to dotnet org repos; skip microsoft/vstest for now
richlander 3439740
Fix schema docs: product field exists, add labels analysis
richlander 9af3d86
Make milestone detection deterministic
richlander 1c255c5
Add head-ref selection logic: check for target tag before using main
richlander 73d556c
Move general docs link to README only; add component-specific links
richlander 83f06d6
Add dotnet_commit field to schema (dotnet-release v1.2.0)
richlander 9e0ed30
Update schema docs for dotnet-release v2.0.0
richlander 1a10de4
Evolve to multi-milestone live system with human interaction
richlander 887240c
Compile agentic workflow; fix toolset names
richlander 4d7add2
Remove stale gist link from changes schema docs
richlander 5105869
Add curated examples reference doc
richlander accd29f
Add JIT/runtime examples to release notes reference
richlander f340d3e
Add ask-for-help guidance: benchmarks, samples, domain context
richlander 68d022c
Split examples by component for token efficiency
richlander d0251eb
Reformat examples: content first, metadata in footer blocks
richlander c23dd4e
Update for dotnet-release v2.1.0; fix example formatting
richlander 9c0d47a
Run every 6 hours instead of daily
richlander ef90d9b
Fix commentary: callback example would benefit from code sample
richlander 5016b21
Replace short ASP.NET example: StatusCodeSelector with code snippet
richlander 19c1636
Add active voice principle to examples guide
richlander 1d92154
Add style note: simplify awkward phrasing in PipeReader example
richlander 05ae36a
Mark PAR example as anti-pattern; add Title/Content/Credit principle
richlander 60b1c8a
Add style note: simplify transition to rule bullets in C# example
richlander 50518fe
Add style note: active voice for MLDsa ease-of-use intro
richlander f1c1a7e
Add principle: link PRs/issues in org/repo #number format
richlander 48940cc
Add principle: avoid jargon and ambiguous words
richlander 63e49d2
Add style note: don't bury the lede with backward-looking passive
richlander 83034f7
Fix markdownlint: setext headings, list numbering, table spacing
richlander 36e528d
Fix broken relative links; exclude lock files from super-linter
richlander 0323096
Fix remaining CI: blank line before list, README link paths
richlander 0991b56
Remove stale monolithic examples.md (replaced by examples/ directory)
richlander 3b44f7e
Add API verification guidance using dotnet-inspect
richlander 6f30d9d
Add nightly SDK download workflow for API verification
richlander 6d81c05
Simplify API verification: query NuGet feed directly, no SDK install
richlander dfe874a
Update api-verification: use dotnet-release build-metadata pipeline
richlander a401333
Editorial rules: prefer short sentences, avoid interrupting clauses
richlander 214ef9a
Editorial rules: three-tier feature ordering
richlander da285a8
Editorial rules: 20/80 rule, two-sentence test, value headlines
richlander 7e200b8
SKILL.md: add examples/ to reference documents
richlander 2053759
Editorial rules: don't overstate maturity in previews
richlander 6eeb131
Editorial rules: filtered features comment block
richlander 3abddf2
Editorial rules: TODO for borderline entries, vet community lists
richlander e4c380d
Format template: add release info and VMR refs to README
richlander 016b30d
Editorial rules: dedup nuance — state changes are new news
richlander 83d3230
Merge branch 'main' into release-notes-agentic-workflow
jeffhandley 7a80b7d
Initialize Agentic Workflows agent
jeffhandley 9d264c1
Set up the select-copilot-pat integration into agentic workflows
jeffhandley bda5c7a
Integrate select-copilot-pat into release-notes workflow
jeffhandley 482744f
Move labeler workflow readme to avoid it being recognized as a workflow
jeffhandley 43e2e1b
Apply review feedback. Set schedule to be daily around 9am PDT.
jeffhandley 7ddb31b
Do not run the release-notes workflow on forks unless manually dispat…
jeffhandley 1fb31f1
Set release-notes workflow timeout to 2 hours
jeffhandley ca7d28a
Merge branch 'main' into release-notes-agentic-workflow
jeffhandley d100e65
Split release notes skill pipeline
richlander a7a8711
Refine release notes scoring guidance
richlander 6c2dd1c
Clarify release-notes editorial guidance
richlander a00e63f
Prefer stronger verbs in release-note headings
richlander 9242e31
Refine release-notes workflow guidance
richlander ecacff1
Remove redundant release-notes docs
richlander f56f9b5
Update .github/workflows/release-notes.md
richlander 0b545af
Update .github/workflows/release-notes.md
richlander 9faac3b
Deduplicate editorial rubric guidance
richlander d9655ed
Clarify preview and GA api-diff labels
richlander 11fcd8f
Rename maintainer release tool to release-notes-gen
richlander File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| { | ||
| "entries": { | ||
| "actions/download-artifact@v8.0.1": { | ||
| "repo": "actions/download-artifact", | ||
| "version": "v8.0.1", | ||
| "sha": "3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c" | ||
| }, | ||
| "actions/github-script@v8": { | ||
| "repo": "actions/github-script", | ||
| "version": "v8", | ||
| "sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd" | ||
| }, | ||
| "actions/setup-dotnet@v5.2.0": { | ||
| "repo": "actions/setup-dotnet", | ||
| "version": "v5.2.0", | ||
| "sha": "c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7" | ||
| }, | ||
| "actions/upload-artifact@v7": { | ||
| "repo": "actions/upload-artifact", | ||
| "version": "v7", | ||
| "sha": "bbbca2ddaa5d8feaa63e36b76fdaad77386f024f" | ||
| }, | ||
| "github/gh-aw-actions/setup@v0.65.1": { | ||
| "repo": "github/gh-aw-actions/setup", | ||
| "version": "v0.65.1", | ||
| "sha": "1831bcd4f397da99da6e6e6b65687557a1a37ac7" | ||
| }, | ||
| "github/gh-aw-actions/setup@v0.65.6": { | ||
| "repo": "github/gh-aw-actions/setup", | ||
| "version": "v0.65.6", | ||
| "sha": "31130b20a8fd3ef263acbe2091267c0aace07e09" | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| --- | ||
| name: editorial-scoring | ||
| description: > | ||
| Apply the shared reader-centric rubric used to rank candidate features for | ||
| release notes, blog posts, and docs. Use this when you need scoring and cut | ||
| guidance independent of any one output format or task. | ||
| --- | ||
|
|
||
| # Editorial Scoring | ||
|
|
||
| Use this skill whenever you need to answer **"How important is this to a reader?"** | ||
|
|
||
| This skill is intentionally **rubric-focused, not task-focused**. It does not generate files on its own. Instead, it provides the shared editorial calibration that other skills should reuse: | ||
|
|
||
| - `generate-features` — assigns the first-pass scores | ||
| - `review-release-notes` — audits and recalibrates those scores | ||
| - `release-notes` — uses the resulting cut to decide what gets written up | ||
| - future blog/docs workflows — can reuse the same rubric with different thresholds | ||
|
|
||
| This is the **canonical home** for the shared scoring scale and 80/20 audience | ||
| filter. Other release-notes docs should point back here instead of restating the | ||
| rubric in full. | ||
|
|
||
| ## Core question | ||
|
|
||
| Score from the perspective of a developer upgrading to the new release — not from the perspective of the engineer who implemented the feature. | ||
|
|
||
| ## Reader-centric scale | ||
|
|
||
| | Score | Reader reaction | What to do with it | | ||
| | ----- | -------------------------------------------------------------------------------- | ---------------------------------------------------------- | | ||
| | `10` | "This is the first feature I'll enable or test." | Lead story | | ||
| | `8+` | "I'm going to use this when I upgrade." | Strong release-note feature | | ||
| | `6+` | "I'm glad I know about this. It will likely come in handy." | Good grouped release-note material | | ||
| | `4+` | "I can see how someone could use this. I'll look up the docs if I ever need it." | Optional mention, grouping candidate, or short paragraph | | ||
| | `2+` | "This one is a mystery to me." | Usually skip unless stronger explanation changes the score | | ||
| | `0` | "This is total gobbledygook — internal .NET engineering work." | Cut it from public-facing content | | ||
|
|
||
| ## 80/20 audience filter | ||
|
|
||
| Default to features that make sense to roughly **80% of the audience**. | ||
|
|
||
| Keep a more specialized feature for the other **20%** only when the remaining 80% can still understand why it matters and react with something like: | ||
|
|
||
| > "Not for me, but I'm glad that's there for the people who need it." | ||
|
|
||
| This is how foundational but initially niche work can still earn a good score. | ||
|
|
||
| ## Strong positive signals | ||
|
|
||
| - New public APIs with obvious user value | ||
| - CLI or workflow improvements developers will notice right away | ||
| - Features that are easy to explain by anchoring them to a familiar tool or workflow | ||
| - Clear performance, reliability, or usability wins | ||
| - Features people have been asking for | ||
| - Changes that require docs, migration notes, or examples to use well | ||
|
|
||
| ## Strong negative signals | ||
|
|
||
| - Test-only work | ||
| - Build or infrastructure churn | ||
| - Refactoring with no user-visible effect | ||
| - VMR sync noise, dependency updates, or repo automation | ||
| - Highly specialized implementation details that read like internal jargon | ||
| - Features that only matter after several stacked conditions are true (for example: uses single-file publish **and** cares deeply about startup **and** is willing to do extra training/tuning) | ||
| - PRs with very thin descriptions where the only concrete signal is an internal runtime/tooling term like cDAC | ||
| - Existing niche surfaces that are barely documented and are not part of the model's normal customer-facing understanding | ||
| - Claimed "new API" features where the actual new public API cannot be identified | ||
| - Bug fixes for features that were never really announced or are still obscure to most readers | ||
| - Old, low-engagement bugs filed internally that sat for a long time without clear external demand | ||
|
|
||
| ## Common scoring mistakes | ||
|
|
||
| - **Technical novelty bias** — "this is clever" is not the same as "users will care" | ||
| - **API inventory mode** — listing everything from an API diff is not release-note curation | ||
| - **Effort bias** — a difficult implementation may still score low if the reader value is small | ||
| - **Insider-language inflation** — terms like JIT, LSRA, cDAC, intrinsics, or pipeline details can sound impressive but still be a `0-4` unless the value is explained clearly | ||
| - **Audience multiplication blindness** — when a feature requires several independent interests or behaviors, multiply those filters together instead of scoring the broadest prerequisite alone | ||
| - **Description vacuum optimism** — if the PR does not explain the user scenario, do not fill in the blanks with a generous story | ||
| - **Documentation blindness** — if an "existing" feature is not findable in Learn docs and does not show up as a known customer scenario, that is a signal to score it around `1`, not to assume hidden importance | ||
| - **Phantom API stories** — if you cannot identify the concrete new public API, do not give API credit based on a vague title alone | ||
| - **Promotion by association** — a fix in a glamorous subsystem is still just a `1` if it only patches an unannounced or niche feature | ||
| - **Demand inversion** — do not mistake an old internal bug with no reactions for evidence that many users need to hear about the fix | ||
| - **Fragmentation bias** — do not dismiss each small related item in isolation when several `2-4` items together form one intelligible reader story, such as a cluster of "Unsafe evolution" changes | ||
|
|
||
| ## Working thresholds | ||
|
|
||
| - **8+** — worthy of its own section | ||
| - **6-7** — usually grouped with similar items, sometimes a short standalone section | ||
| - **0-5** — bug-fix bucket, one-liner, or cut | ||
|
|
||
| Several `3-5` items in the same area can still combine into **one** worthwhile writeup. Keep the individual scores honest, then let the writing stage roll them up into a single themed section. | ||
|
|
||
| ## Breaking changes are separate from score | ||
|
|
||
| Use score for **reader interest/value**, and use `breaking_changes: true` for **reaction/migration significance**. | ||
|
|
||
| That means: | ||
|
|
||
| - a **low-score breaking change** often still deserves a short end-of-notes callout | ||
| - a **high-score breaking change** may deserve both a full feature writeup and a migration note | ||
|
|
||
| Do **not** inflate a narrow breaking change to `7+` just to keep it visible. | ||
|
|
||
| ## Canonical references | ||
|
|
||
| - [`../release-notes/references/feature-scoring.md`](../release-notes/references/feature-scoring.md) — detailed scoring guidance | ||
| - [`../release-notes/references/quality-bar.md`](../release-notes/references/quality-bar.md) — what good release notes look like | ||
| - [`../release-notes/references/examples/README.md`](../release-notes/references/examples/README.md) — editorial examples and why they work |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| --- | ||
| name: generate-changes | ||
| description: > | ||
| Generate `changes.json` for a .NET release milestone by selecting the correct | ||
| VMR base/head refs and running `release-notes-gen generate changes`. Handles | ||
| preview-only multi-branch targeting (`main` vs release branches vs tags) and | ||
| emits the authoritative manifest of what shipped. DO NOT USE FOR: API | ||
| verification/diffs (use api-diff), feature scoring (use generate-features), | ||
| or writing markdown release notes (use release-notes). | ||
| --- | ||
|
|
||
| # Generate `changes.json` | ||
|
|
||
| Produce the authoritative `changes.json` input for a preview, RC, or GA release milestone. | ||
|
|
||
| This is the **VMR-aware data acquisition stage** of the release notes pipeline: | ||
|
|
||
| 1. Determine which milestone(s) are active | ||
| 2. Resolve the correct `--base` and `--head` refs | ||
| 3. Run `release-notes-gen generate changes` | ||
| 4. Write `changes.json` into the correct `release-notes/` folder | ||
|
|
||
| ## When to use | ||
|
|
||
| - A new preview, RC, or GA milestone needs fresh shipped-change data | ||
| - The user wants to know whether **multiple preview milestones** are active at once | ||
| - The release notes branch should be refreshed after the VMR moved forward | ||
| - `changes.json` is missing, stale, or suspected to have the wrong ref selection | ||
|
|
||
| ## Preview-only branch targeting | ||
|
|
||
| This is the subtle part, and it mostly matters for **previews**. | ||
|
|
||
| Multiple preview milestones can be active simultaneously: | ||
|
|
||
| ```text | ||
| Latest shipped in this repo: Preview 3 | ||
| VMR main: Preview 5 | ||
| VMR release branch exists: Preview 4 | ||
|
|
||
| → Generate one `changes.json` for Preview 4 | ||
| → Generate one `changes.json` for Preview 5 | ||
| ``` | ||
|
|
||
| For each target milestone `N`: | ||
|
|
||
| | Milestone state | Base ref | Head ref | | ||
| | --------------------------------------- | ----------- | ------------------ | | ||
| | Tag exists for N | Tag for N-1 | Tag for N | | ||
| | Release branch exists for N, no tag yet | Tag for N-1 | Release branch tip | | ||
| | Only on `main` | Tag for N-1 | `main` | | ||
|
|
||
| **Critical rule:** never use `main` for milestone `N` if `main` has already moved to `N+1`. | ||
|
|
||
| ## Inputs | ||
|
|
||
| The user should provide as much of this as they know: | ||
|
|
||
| - **Target release** — e.g. `.NET 11 Preview 4`, `.NET 10 RC 2`, `.NET 10 GA` | ||
| - **VMR clone path** — defaults to a local clone of `dotnet/dotnet` | ||
| - Optionally, the exact refs if they already know them | ||
|
|
||
| If the user does **not** specify the milestone, infer it from: | ||
|
|
||
| 1. `release-notes/{version}/releases.json` in this repo | ||
| 2. `eng/Versions.props` on `main` in the VMR | ||
| 3. Matching preview tags and release branches in the VMR | ||
|
|
||
| ## Process | ||
|
|
||
| ### 1. Determine the floor from `releases.json` | ||
|
|
||
| Find the latest shipped milestone in this repo. That tells you the lowest in-flight milestone that may need work. | ||
|
|
||
| ### 2. Inspect the VMR | ||
|
|
||
| - Read `eng/Versions.props` on `main` to determine the current prerelease iteration | ||
| - List matching VMR tags for finalized milestones | ||
| - List matching VMR release branches for stabilizing milestones | ||
|
|
||
| Use the [VMR structure reference](../release-notes/references/vmr-structure.md) for naming conventions and branch patterns. | ||
|
|
||
| ### 3. Resolve `--base` and `--head` | ||
|
|
||
| For each active milestone: | ||
|
|
||
| - `--base` is the previous shipped milestone tag | ||
| - `--head` is the milestone tag, release branch tip, or `main`, depending on what exists | ||
|
|
||
| ### 4. Generate the file | ||
|
|
||
| ```bash | ||
| release-notes-gen generate changes <vmr-clone-path> \ | ||
| --base <previous-release-tag> \ | ||
| --head <current-release-ref> \ | ||
| --version "<release-version>" \ | ||
| --date "<yyyy-mm-dd>" \ | ||
| --labels \ | ||
| --output release-notes/<major.minor>/<milestone-path>/changes.json | ||
| ``` | ||
|
|
||
| Examples: | ||
|
|
||
| ```bash | ||
| # Preview milestone | ||
| release-notes-gen generate changes ~/git/dotnet \ | ||
| --base v11.0.0-preview.3.26210.100 \ | ||
| --head origin/release/11.0.1xx-preview4 \ | ||
| --version "11.0.0-preview.4" \ | ||
| --labels \ | ||
| --output release-notes/11.0/preview/preview4/changes.json | ||
|
|
||
| # GA/patch milestone | ||
| release-notes-gen generate changes ~/git/dotnet \ | ||
| --base v10.0.7 \ | ||
| --head v10.0.8 \ | ||
| --version "10.0.8" \ | ||
| --output release-notes/10.0/10.0.8/changes.json | ||
| ``` | ||
|
|
||
| ## Output contract | ||
|
|
||
| The output file must follow the shared schema documented in [changes-schema.md](../release-notes/references/changes-schema.md): | ||
|
|
||
| - top-level `release_version`, `release_date`, `changes`, `commits` | ||
| - stable `id` values in `repo@shortcommit` format | ||
| - same authoritative source of truth used by later skills | ||
|
|
||
| Once `changes.json` exists, the next step is usually `generate-features`. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious why you're removing this sentence. I had added it after the agent kept tripping up on how to consume the output from the tool, and this made it more reliable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That seems like overfit. Did you find that the agent needs that information?