Skip to content

fix(selfhost): source an open PR's Grafana verdict from review_audit, not dead advisories#3535

Merged
JSONbored merged 1 commit into
mainfrom
fix/grafana-review-audit-verdict
Jul 5, 2026
Merged

fix(selfhost): source an open PR's Grafana verdict from review_audit, not dead advisories#3535
JSONbored merged 1 commit into
mainfrom
fix/grafana-review-audit-verdict

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Closes #3511

Summary

  • Follow-up to fix(selfhost): give verdict the same state/merged_at precedence status has #3512. That PR fixed the verdict shown for merged/closed PRs (the vast majority). This closes the remaining gap it flagged: a still-open PR's verdict.

  • Root cause (corrected after deeper investigation): the exporter joined a still-open PR's verdict against advisories.conclusion — but that column only ever holds a PRE-gate rules-severity summary (buildPullRequestAdvisory), which reads neutral/action_required for essentially every PR by construction. It was never the gate's own final merge/close/hold decision, so a still-open PR's verdict was an eternal manual placeholder regardless of what the gate actually decided.

    (Initial investigation assumed persistAdvisory was simply uncalled dead code — a grep without -a silently missed every match in src/queue/processors.ts due to a NUL byte earlier in that file. It has 3 live callers; they just persist the wrong kind of conclusion for this purpose. No app code needed to change once this was understood.)

  • The real signal already exists and is live: review_audit's gittensory-native gate_decision rows, written unconditionally on every self-host instance by recordNativeGateDecision (src/review/parity-wire.ts) on every finalized gate evaluation. Verified directly against production Postgres: 1854/1854 recent rows have a head_sha, the newest is seconds old, and it already includes live hold decisions for currently-open PRs.

  • Fix: point the exporter's still-open-PR verdict join at review_audit (source='gittensory-native', latest row per PR by target_id) instead of advisories. mergemerge, closeclose, holdmanual. The terminal-state precedence from fix(selfhost): give verdict the same state/merged_at precedence status has #3512 (state/merged_at wins first) is unchanged.

  • Verified against live production Postgres: still-open PRs now show real, varied verdicts (12 manual from live holds, 3 commented/merge from live merge decisions, 5 with no decision yet) instead of 100% manual; the merged/closed majority (2794 rows) is unaffected.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

  • git diff --check
  • npm run typecheck (clean — no TypeScript files changed)
  • shellcheck scripts/export-grafana-reporting-db.sh (clean; caught and fixed a real bug during review — a backtick-quoted term in a new SQL comment would have triggered shell command substitution inside the double-quoted query string)
  • npx vitest run test/unit/selfhost-grafana-reporting.test.ts — 13/13 passing (2 existing tests rewritten around review_audit fixtures instead of advisories, 2 new tests added: still-open PR with a live merge/close decision, and a reviewbot-source row correctly ignored so a shadow-parity comparison row never masquerades as the live verdict)
  • Re-verified the exact fixed SQL directly against live production Postgres both before and after the change.
  • npm run test:workers / npm run build:mcp / npm run test:mcp-pack / npm run ui:openapi:check / npm run ui:build — not run individually; no worker/MCP/OpenAPI/UI surface touched.
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries — see above; also covers the "no gate_decision row at all" fail-safe branch (status='manual', verdict=NULL, never fabricated).

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. — N/A; no auth/session/CORS surface touched.
  • API/OpenAPI/MCP behavior is updated and tested where needed. — N/A, no request/response contract change.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. — N/A, no UI change (internal maintainer-only Grafana dashboard).
  • Visible UI changes include a UI Evidence section below. — N/A, not an apps/gittensory-ui change.
  • Public docs/changelogs are updated where needed. — N/A, internal reliability fix, no documented/user-facing behavior change.

… not dead advisories

The exporter's still-open-PR verdict was joined against advisories,
which only ever holds a PRE-gate rules-severity summary (always
neutral/action_required in practice) -- never the gate's own actual
merge/close/hold decision. That decision already has a live, correct,
continuously-populated home: review_audit's gittensory-native
gate_decision rows (written unconditionally on self-host by
recordNativeGateDecision). Point the exporter there instead so a
still-open PR's verdict reflects the gate's real, current state
instead of an eternal placeholder.

Verified against live production Postgres: a still-open PR now shows
its actual hold/merge/close decision instead of always 'manual', while
merged/closed PRs (the majority, from PR #3512) are unaffected.

Closes #3511
@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.09%. Comparing base (2c547b4) to head (f024597).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3535   +/-   ##
=======================================
  Coverage   93.09%   93.09%           
=======================================
  Files         301      301           
  Lines       31450    31450           
  Branches    11483    11483           
=======================================
  Hits        29279    29279           
  Misses       1517     1517           
  Partials      654      654           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 5, 2026
@loopover-orb

loopover-orb Bot commented Jul 5, 2026

Copy link
Copy Markdown

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-05 16:19:36 UTC

2 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
Correctly replaces the exporter's join source from `advisories.conclusion` (a pre-gate rules-severity summary that was permanently `neutral`/`action_required` for essentially every PR by construction) to `review_audit` filtered on `event_type='gate_decision'` and `source='gittensory-native'`, which is the sole authoritative writer (`recordNativeGateDecision`). The `target_id` join key (`repo_full_name || '#' || p.number`) matches the format confirmed in both the test fixtures and the production Postgres row dump cited in the PR description. Four test cases cover the critical branches — `hold`, no-decision null-safe fallback (`verdict IS NULL`, `status='manual'`), `merge`/`close` direct verdict pass-through, and cross-source filtering — all exercising real data shapes rather than fabricated states.

Nits — 6 non-blocking
  • scripts/export-grafana-reporting-db.sh (~line 155): The Postgres 'no source tables' early-exit guard still lists `advisories` but not `review_audit` in its check set. The guard is conservative (any listed table present → proceed, so no false exits), but it is stale and will confuse an operator who reads it to understand what tables the exporter depends on.
  • scripts/export-grafana-reporting-db.sh Postgres CTE: `ORDER BY created_at DESC, id DESC` uses `id` (TEXT, likely UUID) as the tiebreaker. Lexicographic DESC ordering on UUIDs is non-deterministic when two `gate_decision` rows share the same `created_at` timestamp. The SQLite path correctly uses `rowid DESC` (monotonic integer); consider a sequence column or `ctid` as the Postgres tiebreaker for parity.
  • scripts/export-grafana-reporting-db.sh: `g.decision = 'close'` maps to `status='manual'`, identical to `'hold'`, so a dashboard reader cannot distinguish 'gate said close, GitHub not yet synced' from 'genuinely held for review'. This mirrors the pre-existing `failure→manual` convention and is not a regression, but a brief inline comment noting the intentional conflation would head off future confusion.
  • Replace `advisories` with `review_audit` in the Postgres early-exit guard to keep it honest about the actual table dependency introduced by this PR.
  • If `review_audit` exposes a monotonic integer column (auto-increment PK or a sequence), use it instead of `id DESC` (TEXT) as the Postgres CTE tiebreaker to guarantee deterministic latest-row selection under timestamp ties.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3511
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 56 registered-repo PR(s), 46 merged, 521 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 56 PR(s), 521 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 56 PR(s), 521 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • No action.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 5, 2026
@JSONbored
JSONbored merged commit 1fbe48a into main Jul 5, 2026
11 checks passed
@JSONbored
JSONbored deleted the fix/grafana-review-audit-verdict branch July 5, 2026 16:24
JSONbored added a commit that referenced this pull request Jul 5, 2026
…les guard (#3546)

The "does this database have any reporting signal at all" fail-safe
guard still listed only pull_requests/advisories/review_targets/
ai_usage_events -- stale since #3535 made review_audit a genuine
dependency of the verdict computation. In practice pull_requests
already dominates the check on any real deployment, so this was
cosmetic, not a live bug, but a database with only review_audit
present would have been misclassified as empty. Add it to both the
Postgres and SQLite variants of the guard.

Flagged as a non-blocking nit on #3535's own review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

Grafana maintainer dashboard shows verdict=manual for every PR (export CASE ignores state/merged_at)

1 participant