Skip to content

[Bug]: PR type label deterministically downgraded on merge — broken GitHub timeline closure check (5th recurrence) #5385

Description

@JSONbored

Summary

The per-PR TYPE label decision (gittensor:bug / gittensor:feature / gittensor:priority) is being deterministically mislabeled on merge again — this is the fifth distinct root cause in this same area in under a week (#3903, #4528, #4816, #4975/#4980, #4986/#4987 were the prior four). Unlike all four of those, which were intermittent races, this one fails on 100% of qualifying merges: any PR whose body closes a linked issue (the single most common case — every "Closes #N" PR) gets its correctly-propagated label stripped back to gittensor:bug within seconds of merging.

Confirmed live right now: #5348, #5359, #5360, #5363, #5371 all merged in the last hour and all currently carry the wrong label. #5359 is the worst case — it lost gittensor:priority (the reward multiplier) entirely, down to bare gittensor:bug.

Area

GitHub App (webhook processing / type-label decision / linked-issue-label propagation)

History

Root cause

fetchLinkedIssueClosedByPullRequest (src/github/backfill.ts) and its helper timelineEventClosesIssueFromPullRequest verify a closed linked issue was closed by this PR by reading GET /issues/{n}/timeline and checking, on events where event === "closed":

event.event === "closed" && event.source?.issue?.number === prNumber && event.source.issue.pull_request !== undefined

GitHub's REST Timeline API never populates a source field on a closed eventsource.issue only ever appears on cross-referenced events, a structurally different event type. Confirmed empirically against three live, real closed events (issues #5195, #5160, #4780, all closed via their linked PR's merge):

{"event":"closed","commit_id":null,"commit_url":null,"actor":{"login":"gittensory-orb[bot]"},"state_reason":null}

No source key at all. So timelineEventClosesIssueFromPullRequest always evaluates false, fetchLinkedIssueClosedByPullRequest always returns "not_closed_by_pull_request" — even when the PR genuinely, unambiguously closed the issue — closedByThisPr is always false, and isLinkedIssueTrustworthy therefore always rejects the single most common propagation case in the entire system. The decision falls through to the title heuristic and overwrites the correct label.

The correct signal does exist, just not via this endpoint/shape. GraphQL's Issue.timelineItems(itemTypes:[CLOSED_EVENT]) { closer } correctly and directly identifies the closing PR:

{ repository(owner:"JSONbored", name:"gittensory") {
    issue(number:5195) { timelineItems(last:5, itemTypes:[CLOSED_EVENT]) {
      nodes { ... on ClosedEvent { closer { __typename ... on PullRequest { number } } } } } } } }
# => {"closer":{"__typename":"PullRequest","number":5360}}   -- correct, matches PR #5360

Test-suite gap that let this ship green: the regression tests added in the same PR (test/unit/queue-5.test.ts, test/unit/linked-issue-label-propagation-fetch.test.ts) stub the /timeline endpoint with a hand-invented response shape ([{"event":"closed","source":{"issue":{"number": prNumber, "pull_request": {}}}}]) that matches the code's assumption, not GitHub's real API — so CI was green while production is now deterministically broken for the common case.

Evidence (production audit_events, github_app.type_label_decision, since the 14:53 UTC deploy)

PR Linked issue (label) Closer confirmed via GraphQL Current (wrong) label Correct label
#5348 #5158 (gittensor:feature) gittensor:bug gittensor:feature
#5359 #4780 (gittensor:priority) PR #5359 gittensor:bug lost gittensor:priority entirely
#5360 #5195 (gittensor:feature) PR #5360 gittensor:bug gittensor:feature
#5363 #5160 (gittensor:feature) PR #5363 gittensor:bug gittensor:feature
#5371 #5161 (gittensor:feature) gittensor:bug gittensor:feature

Two more (#5357, #5364) independently landed on the right label via the title heuristic by coincidence — the propagation mechanism failed there too, just invisibly.

Ruled out / also audited while investigating

  • Confirmed exactly one code path writes gittensor:bug/feature/priority anywhere in src/ (processors.ts, inside maybePublishPrPublicSurface) — no rogue AI/content-based classifier or second writer exists.
  • The periodic sweep (reReviewStoredPullRequest) shares the same code path and is not a separate contributor; it never touches merged/closed PRs at all.
  • Two separate, lower-severity design gaps surfaced during this audit, tracked here for visibility (see linked PR for whether they're addressed together or split out):
    1. resolvePrTypeLabel's bug-vs-feature exclusive tie-break is .gittensory.yml mapping array order (bug listed before feature ⇒ bug wins when both match), not label value — backwards from the stated priority > feature > bug scoring-weight precedence in the bug/feature case.
    2. maybeReReviewOnLinkedIssueChange (wakes a PR when its linked issue gets relabeled) only checks isConvergenceRepoAllowed, unlike the sweep which also falls back to hasInstallation && isAgentConfigured(settings.autonomy) — a structural asymmetry that could leave a PR's label stale indefinitely for a repo outside the GITTENSORY_REVIEW_REPOS allowlist.
    3. (Confirmed intentional, not a bug, no action needed) gittensor:priority is deliberately additive — a PR can carry both gittensor:feature and gittensor:priority simultaneously, with scoring resolving to the higher multiplier. Verified this matches intended behavior; no change needed.

Expected behavior

A merged PR whose body closes a linked issue always inherits that issue's label(s), regardless of the specific webhook/timing that triggers the recompute, verified through a mechanism that actually reflects GitHub's real API shape.

Actual behavior

Every PR that merges and auto-closes its linked issue via a closing keyword gets its label recomputed via the title heuristic instead, because the closure-verification check can never succeed against GitHub's real timeline event shape.

Requirements

  1. Replace fetchLinkedIssueClosedByPullRequest's REST-timeline heuristic with the GraphQL ClosedEvent.closer field (or an equivalent check validated against a real, captured API response, not an assumed shape).
  2. Any new test fixtures must be built from an actual gh api response (or GraphQL response) captured against a real closed issue, not hand-invented.
  3. Fix the bug-vs-feature tie-break to respect label value (feature outranks bug) rather than config array position.
  4. Align maybeReReviewOnLinkedIssueChange's gating with the sweep's autonomy fallback.
  5. Regression tests for all of the above.
  6. Backfill every PR mislabeled by this specific regression since the 14:53 UTC deploy (and any residual pre-deploy stragglers from the prior, still-not-fully-closed race).

Deliverables

  • Code fix in src/github/backfill.ts / src/review/linked-issue-label-propagation-fetch.ts (+ mirrored in packages/gittensory-engine).
  • Tie-break precedence fix in src/settings/pr-type-label.ts.
  • Autonomy-fallback fix in src/queue/processors.ts (maybeReReviewOnLinkedIssueChange).
  • Regression tests for all three, built from real captured API shapes.
  • Deploy to production, verify.
  • Backfill affected PRs, verify programmatically post-write.

Out of scope

  • Making gittensor:priority exclusive instead of additive — confirmed intentional, no change.
  • General queue-latency reduction — unrelated to this failure mode (this one is deterministic, not load-dependent).

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

Status
In Progress

Relationships

None yet

Development

No branches or pull requests

Issue actions