Skip to content

Strengthen GitHub surface protocol so pass requires GitHub App authorship proof #10

@OpenCodeEngineer

Description

@OpenCodeEngineer

Summary

The current GitHub surface protocol matrix can report pass for some surfaces without proving that the handling GitHub App actually authored the result.

Problem

The protocol currently defines pass as command acceptance plus run-start evidence. That is weaker than the product requirement that GitHub replies be authored by the handling GitHub App.

Examples:

  • pr-review-mention currently passes after finding a bot comment containing the token on the PR conversation thread, but the result does not assert performed_via_github_app, exact expected app slug, or exact expected bot author.
  • discussion-mention can pass via synthetic discussion_comment webhook fallback and sourceKey persistence evidence only, which proves routing/run creation but not GitHub App-authored reply behavior on the discussion surface.

Why this matters

A protocol run can look green even if the wrong actor authored the response or no real GitHub App-authored response occurred on that surface.

Acceptance criteria

  • For issue/PR/review/discussion surfaces, pass should mean GitHub App-authored proof exists for that surface or the documented response surface.
  • Protocol checks should validate, where applicable:
    • expected bot author login
    • expected performed_via_github_app.slug
    • expected response surface
  • Synthetic fallback should not report pass as if it proved real GitHub App authorship on discussions.
    • It should be reported as blocked or a weaker non-proof status unless a live app-authored discussion reply is observed.
  • Docs should stop describing protocol pass as sufficient evidence when it only proves routing/run creation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions