Summary
The current GitHub surface protocol matrix can report pass for some surfaces without proving that the handling GitHub App actually authored the result.
Problem
The protocol currently defines pass as command acceptance plus run-start evidence. That is weaker than the product requirement that GitHub replies be authored by the handling GitHub App.
Examples:
pr-review-mention currently passes after finding a bot comment containing the token on the PR conversation thread, but the result does not assert performed_via_github_app, exact expected app slug, or exact expected bot author.
discussion-mention can pass via synthetic discussion_comment webhook fallback and sourceKey persistence evidence only, which proves routing/run creation but not GitHub App-authored reply behavior on the discussion surface.
Why this matters
A protocol run can look green even if the wrong actor authored the response or no real GitHub App-authored response occurred on that surface.
Acceptance criteria
- For issue/PR/review/discussion surfaces,
pass should mean GitHub App-authored proof exists for that surface or the documented response surface.
- Protocol checks should validate, where applicable:
- expected bot author login
- expected
performed_via_github_app.slug
- expected response surface
- Synthetic fallback should not report
pass as if it proved real GitHub App authorship on discussions.
- It should be reported as
blocked or a weaker non-proof status unless a live app-authored discussion reply is observed.
- Docs should stop describing protocol
pass as sufficient evidence when it only proves routing/run creation.
Summary
The current GitHub surface protocol matrix can report
passfor some surfaces without proving that the handling GitHub App actually authored the result.Problem
The protocol currently defines
passas command acceptance plus run-start evidence. That is weaker than the product requirement that GitHub replies be authored by the handling GitHub App.Examples:
pr-review-mentioncurrently passes after finding a bot comment containing the token on the PR conversation thread, but the result does not assertperformed_via_github_app, exact expected app slug, or exact expected bot author.discussion-mentioncan pass via syntheticdiscussion_commentwebhook fallback andsourceKeypersistence evidence only, which proves routing/run creation but not GitHub App-authored reply behavior on the discussion surface.Why this matters
A protocol run can look green even if the wrong actor authored the response or no real GitHub App-authored response occurred on that surface.
Acceptance criteria
passshould mean GitHub App-authored proof exists for that surface or the documented response surface.performed_via_github_app.slugpassas if it proved real GitHub App authorship on discussions.blockedor a weaker non-proof status unless a live app-authored discussion reply is observed.passas sufficient evidence when it only proves routing/run creation.