Skip to content

Per-user reviewer attempts + @-mention fallback + widen toolhive docs paths#776

Merged
rdimitrov merged 2 commits intomainfrom
reviewer-mentions-and-toolhive-docspaths
Apr 22, 2026
Merged

Per-user reviewer attempts + @-mention fallback + widen toolhive docs paths#776
rdimitrov merged 2 commits intomainfrom
reviewer-mentions-and-toolhive-docspaths

Conversation

@rdimitrov
Copy link
Copy Markdown
Member

@rdimitrov rdimitrov commented Apr 22, 2026

Three improvements stacked in one PR, from feedback on PR #759.

1. Stop pre-filtering reviewers by repos/X/collaborators/Y — try to assign them directly

What happened on PR #759: the reviewer-extract step found rdimitrov as the only eligible reviewer. Four other non-bot release contributors (ChrisJBurns, jhrozek, reyortiz3, tgrunnagle) were silently dropped. A local check with a PAT that has read:org confirms all five ARE collaborators on docs-website via the `stackers` team (push permission):

```
ChrisJBurns collaborator (maintain, via stackers team)
jhrozek collaborator (admin, via stackers team)
reyortiz3 collaborator (maintain, via stackers team)
tgrunnagle collaborator (maintain, via stackers team)
rdimitrov collaborator (admin, via stackers team)
```

So the pre-filter was the problem — not team membership. Suspected cause: `GITHUB_TOKEN` in Actions treats the `repos/X/collaborators/Y` endpoint differently for team-based access than a PAT with `read:org`. Haven't pinned the exact rule.

Fix: replace the upfront filter with per-user `gh pr edit --add-reviewer` attempts. Each attempt is independent, so a single 422 doesn't kill the batch. Routes GitHub's authoritative answer ("can this person be a reviewer right now?") directly into ASSIGN vs. MENTION, bypassing our fallible pre-filter.

2. @-mention the rest in the PR body

For contributors GitHub rejects (or any past the review-fatigue cap of 5), render as an @-mention list in a new Release contributors section of the PR body. They still get notified.

3. Expand `toolhive` docs_paths hints

The `toolhive` upstream is a monorepo: CLI, Kubernetes operator, vMCP gateway, plus cross-cutting features that land in `concepts/`, `integrations/`, `tutorials/`, and hand-written reference pages. Previous hints listed only the three `guides-*` folders. In practice the skill's Phase 3 impact map already expands beyond hints, so this is a focus improvement — accurate hints narrow Phase 2's source reading.

Other projects' hints are already scoped correctly. `toolhive-cloud-ui` keeps `docs_paths: []` — no associated docs in this repo yet.

Why this works across scenarios

  • If our filter was right → per-user attempts succeed for the same people, same result.
  • If our filter was wrong (like on Update stacklok/toolhive to v0.23.1 #759) → per-user attempts still succeed for the real collaborators, and rejected ones get @-mentioned rather than silently vanishing.
  • If GitHub's behavior changes in the future → we adapt automatically, no code change.

Future

Detecting Stacklok employees directly (e.g. via the `stackers` team) would still require a PAT with `read:org` stored as a repo secret. Not wired in here — the per-user approach solves the immediate problem without needing extra secrets.

…paths

Two independent but related improvements to the upstream-release-docs
pipeline:

1. Reviewer routing: @-mention contributors we can't auto-assign

   The existing reviewer-extract step batched all non-bot release
   commit authors and filtered to docs-website collaborators. On
   PR #759 that filter silently dropped four of five candidates
   (`reyortiz3`, `ChrisJBurns`, `jhroz`, `tgrunnag` are all upstream
   toolhive contributors but not collaborators on docs-website).
   The filter is mandatory because `gh pr edit --add-reviewer`
   returns 422 for any non-collaborator in the list, dropping the
   valid reviewers alongside the invalid ones.

   Now split the candidates:
     - ASSIGN_LIST (collaborators): batch-assigned as reviewers
       via `--add-reviewer`. Unchanged behavior.
     - MENTION_LIST (everyone else): @-mentioned in a new
       "Release contributors" section of the PR body so they see
       the PR documenting their work even though we can't request
       their review via the API.

   Detecting Stacklok employees directly (e.g. via the `stackers`
   team) requires a PAT with read:org scope -- `GITHUB_TOKEN`
   doesn't have it, and public-org-membership checks only see
   members with public membership. Deferred; noted in a comment.

2. Expand toolhive docs_paths hints

   The `toolhive` upstream is a monorepo shipping the CLI, the
   Kubernetes operator, and the vMCP gateway, plus cross-cutting
   features that land in concepts/, integrations/, tutorials/,
   and hand-written reference pages. The previous hints listed
   only the three guides-* folders. In practice the skill's
   Phase 3 impact map expanded beyond the hints (PR #759 touched
   concepts/, integrations/, tutorials/), so this is a focus
   improvement rather than a bug fix -- more accurate hints let
   Phase 2's source reading home in on the right areas without
   re-scanning unrelated docs.

   Other projects' hints are already accurate for their scope.
   toolhive-cloud-ui intentionally keeps docs_paths: [] -- no
   associated docs in this repo yet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 22, 2026 07:28
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 22, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs-website Ready Ready Preview, Comment Apr 22, 2026 7:38am

Request Review

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the upstream-release-docs GitHub Actions workflow and upstream project hints to better notify relevant upstream contributors and to improve Toolhive documentation targeting during automated release-doc generation.

Changes:

  • Split release compare authors into (1) collaborators to auto-assign as reviewers and (2) non-collaborators to @-mention in the PR body.
  • Expand toolhive docs_paths hints to include additional docs sections and a key hand-written reference page.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.github/workflows/upstream-release-docs.yml Adds contributor @-mentioning in the PR body while keeping reviewer auto-assignment restricted to collaborators to avoid gh pr edit 422 failures.
.github/upstream-projects.yaml Broadens Toolhive docs_paths hints to better reflect monorepo doc touchpoints (concepts/integrations/tutorials + authz policy reference).

Replaces the upfront `gh api repos/X/collaborators/Y` filter with
per-user `gh pr edit --add-reviewer` attempts. Routes GitHub's own
rejection (422) into the mention list instead of relying on our
filter to pre-compute the split.

Why: on PR #759 the collaborator check returned 404 for Stacklok
employees who ARE collaborators via the `stackers` team (push
permission on docs-website) -- `ChrisJBurns`, `jhrozek`, `reyortiz3`,
`tgrunnagle`. Only `rdimitrov` passed, so four reviewers silently
vanished. A local check with a PAT that has read:org confirms all
five are collaborators; the discrepancy appears to be GITHUB_TOKEN
treating team-based access differently at the collaborator endpoint,
though we haven't pinned the exact rule.

The authoritative answer is "will GitHub accept this person as a
reviewer right now" -- asking that question directly (via the
add-reviewer API) avoids the filter being wrong. Each attempt is
independent, so one rejection doesn't kill the batch.

The separate "Add reviewers" step is now redundant -- assignments
happen inline during candidate iteration. Dropped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rdimitrov rdimitrov changed the title @-mention non-collaborator contributors + widen toolhive docs paths Per-user reviewer attempts + @-mention fallback + widen toolhive docs paths Apr 22, 2026
@rdimitrov rdimitrov enabled auto-merge (squash) April 22, 2026 07:50
@rdimitrov rdimitrov merged commit 47b67d5 into main Apr 22, 2026
3 checks passed
@rdimitrov rdimitrov deleted the reviewer-mentions-and-toolhive-docspaths branch April 22, 2026 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants