Skip to content

Dreaming: up to 3 atomic PRs + per-PR core-team reviewer requests#14373

Merged
jankratochvilcz merged 3 commits into
mainfrom
dreaming-reviewers-and-multi-pr
Jul 15, 2026
Merged

Dreaming: up to 3 atomic PRs + per-PR core-team reviewer requests#14373
jankratochvilcz merged 3 commits into
mainfrom
dreaming-reviewers-and-multi-pr

Conversation

@jankratochvilcz

Copy link
Copy Markdown
Contributor

What

Two enhancements to the Dreaming learning-atoms workflow (.github/workflows/dreaming.agent.md), requested to make its output more actionable:

  1. Up to 3 atomic PRs per run. create-pull-request.max raised 1 → 3. Distinct, unrelated recurring patterns now each land as their own atomic, independently-reviewable PR (one theme per PR, each on its own git branch). A quiet week or a single-theme week still produces just one PR — the workflow is told not to split for the sake of it.

  2. Per-PR reviewer requests, scoped to the core MSBuild team, max 2. For each PR, the agent identifies the people who were actually involved in the discussions behind that pattern, keeps only core-team members, caps at 2, and requests their review.

Prompt-body changes support this: Step 1 now records each participant's authorAssociation, Step 2 keeps the per-theme participant list, Step 4 branches per pattern for atomicity, and a new Step 5b does reviewer selection.

How "core MSBuild team" is determined

The agent classifies a participant as core team only when their GitHub authorAssociation on the PR is OWNER or MEMBER (org members) — COLLABORATOR/CONTRIBUTOR/NONE are dropped, along with bots and the PR authors. When no individual qualifies, it falls back to the @dotnet/kitten CODEOWNERS team (the MSBuild owners team). This keeps the "only core team" guarantee without hardcoding a username list that would churn.

Platform constraint (why reviewers are recorded in the PR body)

I dug through gh-aw v0.82.9's safe-outputs internals to find the right mechanism, and there is a real limitation worth calling out:

  • create_pull_request has no per-item reviewers field — the agent can't attach reviewers to an individual PR it emits.
  • create-pull-request.reviewers / team-reviewers config does issue a real requestReviewers call, but it is static and uniform — the same reviewers on every PR in the run — so it can't be the specific per-discussion people.
  • The add-reviewer safe output resolves its target PR only from a literal integer pull_request_number or a pull_request-event context. Dreaming runs on schedule/workflow_dispatch and creates its PRs in the same run, so add-reviewer has no PR number to target and would error every time. (I verified this in pr_helpers.cjs getPullRequestNumber and confirmed add_reviewer is never deferred for temp-ID resolution.)

So a formal, dynamic, per-PR, ≤2-specific-people review request isn't achievable through gh-aw safe-outputs today. The faithful, reliable path — which is dynamic per pattern and restricted to the right people — is to record the request in each PR body as a dedicated ## Reviewers (core MSBuild team) section with real @-mentions. This notifies the chosen reviewers and lets a maintainer assign them with one click. If we later want formal auto-assignment, the supported route is a small companion workflow triggered on the Dreaming PR's pull_request opened event (which does have PR context) — happy to add that as a follow-up if desired.

Guardrail preserved

The existing "no contributor call-outs / no individual names" rule still governs the learning-atom content (the edited instruction files). Naming reviewers in the PR body is explicitly carved out as review metadata, not atom content.

Scope

Only the Dreaming workflow's .md + regenerated .lock.yml. Lock diff is limited to max 1→3, the removal of the (non-functional) add-reviewer tooling, and the embedded prompt text. Compiles clean (0 errors, 0 warnings). A human still approves and merges every Dreaming PR.

Enhances the Dreaming learning-atoms workflow:
- Raise create-pull-request max 1 -> 3 so distinct, unrelated recurring
  patterns each land as their own atomic, independently-reviewable PR
  (one theme per PR, each on its own branch).
- Per PR, request review from the <=2 core MSBuild team members who were
  actually involved in the discussions behind that pattern. Core team is
  identified via authorAssociation == OWNER/MEMBER; capped at 2; falls
  back to the @dotnet/kitten CODEOWNERS team when no individual qualifies.

gh-aw safe-outputs has no per-PR reviewer field and its add-reviewer output
cannot target PRs created in the same schedule/dispatch run, so the reviewer
request is recorded in each PR body (dynamic per pattern) rather than via a
static, uniform reviewers config. Recompiled the lock.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b63f3d4-d050-4905-9e00-bee816df8a94
Copilot AI review requested due to automatic review settings July 15, 2026 13:35
@jankratochvilcz
jankratochvilcz requested a review from a team as a code owner July 15, 2026 13:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Dreaming agentic workflow to produce more actionable learning-atom updates by splitting distinct recurring themes into separate PRs and capturing per-theme core-team reviewer intent in PR metadata.

Changes:

  • Increase create-pull-request.max from 1 to 3 so the workflow can open up to three atomic PRs per run (one per distinct recurring pattern).
  • Extend the prompt to preserve per-theme participant metadata (login + authorAssociation) and add a reviewer-selection step for core-team participants (max 2, with @dotnet/kitten fallback) recorded in the PR body.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
.github/workflows/dreaming.agent.md Raises PR limit to 3 and updates the Dreaming prompt to branch per pattern and record per-theme core-team reviewer mentions in the PR body.
.github/workflows/dreaming.agent.lock.yml Regenerates the compiled workflow to reflect the updated prompt text and create_pull_request(max:3) safe-output configuration.

Comment thread .github/workflows/dreaming.agent.md Outdated
Comment thread .github/workflows/dreaming.agent.md Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

24-Dimension Review Summary

Dimensions 2-3, 5-9, 11-17, 19, 21, 23-24 (ChangeWave, binary log, performance, allocations, API surface, evaluation model, concurrency, etc.) — LGTM. Not applicable to this workflow-only PR.

Dimension 1 (Backwards Compatibility) — LGTM. No existing callers depend on the single-PR behavior; the workflow is internal automation.

Dimension 4 (Test Coverage) — LGTM. Workflow agent prompts have no unit tests; the lock file recompilation serves as validation.

Dimension 10 (Design) — The multi-PR design is sound and well-motivated. Splitting unrelated patterns into atomic PRs improves reviewability. The reviewer-request feature is a nice touch for routing reviews to knowledgeable people.

Dimension 18 (Documentation) — Generally clear and self-consistent. One clarification needed on rule 8 (see inline comment about same-theme vs same-file).

Dimension 20 (Scope & PR Discipline) — LGTM. Well-scoped: two related features (multi-PR + reviewer requests) that naturally go together.

Dimension 22 (Correctness) — Two minor edge cases flagged inline:

  1. Branch creation should ensure main is fresh (fetch before checkout).
  2. @-mentions in PR body are notifications, not formal review requests — the instructions should set expectations accordingly.

Overall this is a well-structured improvement. The inline comments are suggestions/questions, not blockers.

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • awmgmcpg
  • patchdiff.githubusercontent.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"
    - "patchdiff.githubusercontent.com"

See Network Configuration for more information.

Generated by Expert Code Review (on open) for #14373 · 117.4 AIC · ⌖ 5.21 AIC · ⊞ 4.8K

Comment thread .github/workflows/dreaming.agent.md Outdated
Comment thread .github/workflows/dreaming.agent.md Outdated
Comment thread .github/workflows/dreaming.agent.md Outdated
@jankratochvilcz
jankratochvilcz requested a deployment to copilot-pat-pool July 15, 2026 13:45 — with GitHub Actions Abandoned
- Clarify @-mentions are notifications for manual assignment, not formal
  GitHub review requests (description, Step 5b, reminders).
- Step 5 body checklist now explicitly requires the ## Reviewers section.
- Step 4 branches from a freshly fetched origin/main (fetch + checkout -B)
  so it is robust to workspace reuse/drift.
- Relax rule 8: a PR may span two files when it is genuinely one theme
  (e.g. SKILL.md + AGENTS.md); split only unrelated patterns.
- Confirm @dotnet/kitten (slug 'kitten') is the correct CODEOWNERS team.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b63f3d4-d050-4905-9e00-bee816df8a94
Comment thread .github/workflows/dreaming.agent.md Outdated
…rAssociation

Per review feedback: authorAssociation == MEMBER matches any dotnet-org
member (hundreds of people), far too broad for 'core MSBuild team'. Switch
the reviewer filter to authoritative membership in the @dotnet/kitten team
(the MSBuild CODEOWNERS team, ~8 people): fetch the roster via
`gh api orgs/dotnet/teams/kitten/members` and keep only participants in it.
Falls back to the @dotnet/kitten handle if the token can't read team
membership. Step 1 now just collects participant logins.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b63f3d4-d050-4905-9e00-bee816df8a94
@jankratochvilcz
jankratochvilcz merged commit 93022af into main Jul 15, 2026
14 checks passed
@jankratochvilcz
jankratochvilcz deleted the dreaming-reviewers-and-multi-pr branch July 15, 2026 16:32
ViktorHofer pushed a commit that referenced this pull request Jul 16, 2026
…itten member) (#14383)

## Context: validating the reviewers + multi-PR change

After #14373 merged, I triggered a live Dreaming run
([29432875878](https://github.com/dotnet/msbuild/actions/runs/29432875878))
to validate the new behavior. Results:

- ✅ **Multi-PR worked** — the run opened **two atomic PRs**, each
touching a single file with a distinct theme (#14381 logging
instructions, #14382 expert-reviewer agent). Exactly the intended
one-pattern-per-PR shape.
- ✅ **Reviewer section rendered** on both, with the softened "please
assign" wording.
- ❌ **One reviewer was wrong.** #14382 listed `@baronfel`, who is
**not** on the `@dotnet/kitten` roster (`gh api
orgs/dotnet/teams/kitten/memberships/baronfel` → 404) **and** was not in
that PR's cited discussion (only `@jankratochvilcz` was). The agent
invented a plausible-sounding name instead of strictly filtering.
(#14381's `@rainersigwald` was correct — active kitten member and the
actual reviewer.)

## Fix

Tighten Step 5b so a reviewer qualifies **only if the login is in
BOTH**:
1. the fetched `orgs/dotnet/teams/kitten/members` output, **and**
2. that pattern's actual discussion-participant list.

Plus explicit guardrails:
- **Never** add a name from memory / general knowledge of "who works on
MSBuild" / because it seems plausible.
- A **pre-emit self-check**: before emitting each PR, verify every
`@`-mention on the reviewers line is in both lists; drop any that fail.
- When in doubt, prefer the `@dotnet/kitten` fallback over guessing an
individual.

## Scope

Only the Dreaming workflow prompt + recompiled lock (1-line lock change:
the embedded prompt hash/text). Compiles clean (0/0). This directly
closes the gap the validation run exposed.

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jankratochvilcz <2202487+jankratochvilcz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants