Skip to content

fix(review): explain validation and guardrail holds#3304

Merged
gittensory-orb[bot] merged 4 commits into
mainfrom
codex/fix-review-hold-reasons
Jul 5, 2026
Merged

fix(review): explain validation and guardrail holds#3304
gittensory-orb[bot] merged 4 commits into
mainfrom
codex/fix-review-hold-reasons

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Clarifies configured validation-evidence failures and guardrail manual-hold reasons in ORB review output.

What changed

  • Treat PR-body validation notes as satisfying configured manifest test expectations.
  • Rename/redact the public missing-validation finding so it says changed tests or passing validation evidence are missing without leaking private config.
  • Include matched guardrail paths in manual-review action reasons when available.
  • Add regression coverage for the live webhook gate path plus helper/unit invariants.

Why

The previous review text could say “Maintainer test expectations unmet” even when the PR body included validation evidence, and guardrail hold reasons did not identify the touched path.

Validation

  • npx vitest run test/unit/test-evidence.test.ts test/unit/focus-manifest.test.ts test/unit/public-safe-manifest-finding.test.ts test/unit/agent-actions.test.ts test/unit/gate-check-policy.test.ts test/unit/queue.test.ts
  • npm run typecheck
  • git diff --check

@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@gittensory-orb gittensory-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.5x multiplier. label Jul 5, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 5, 2026

Copy link
Copy Markdown

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-05 05:21:20 UTC

13 files · 1 AI reviewer · no blockers · readiness 93/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This change correctly moves PR-body validation evidence into the manifest-policy gate and keeps predictor/live-gate parity, while also redacting the public missing-validation copy and improving guardrail hold reasons. The live webhook tests exercise the real processor path rather than fabricating only helper inputs, and the public-safe override now covers title/detail/action consistently. I do not see a reachable correctness blocker in the provided diff.

Nits — 5 non-blocking
  • nit: src/signals/test-evidence.ts:54 treats any affirmative-looking clause as validation evidence even if a later adjacent clause reports failure, e.g. `Ran npm run test:ci, failed on shard 2`; consider adding a regression case if that wording is common enough to matter.
  • nit: src/settings/agent-actions.ts:362 hard-codes the visible guardrail path limit as `3`; naming it would make the truncation policy clearer and keep the slice/count logic tied together.
  • nit: test/unit/queue.test.ts:6266 adds three large webhook fixtures with mostly duplicated setup; consider extracting the manifest-gate repository setup and fetch stub once this grows again.
  • src/signals/test-evidence.ts:54: add a targeted test for punctuation-separated failure wording around validation commands, then either document the intentional heuristic or expand the negative scan to cover the neighboring clause.
  • src/settings/agent-actions.ts:362: introduce `const MAX_GUARDRAIL_REASON_PATHS = 3;` and use it for both `slice` and the remaining-count calculation.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 56 registered-repo PR(s), 46 merged, 416 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 56 PR(s), 416 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 56 PR(s), 416 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.90%. Comparing base (1afb2ed) to head (abed057).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3304      +/-   ##
==========================================
+ Coverage   93.88%   93.90%   +0.01%     
==========================================
  Files         282      282              
  Lines       30570    30588      +18     
  Branches    11136    11143       +7     
==========================================
+ Hits        28702    28724      +22     
+ Misses       1211     1208       -3     
+ Partials      657      656       -1     
Files with missing lines Coverage Δ
src/queue/processors.ts 93.02% <100.00%> (+0.08%) ⬆️
src/rules/predicted-gate.ts 100.00% <100.00%> (+3.12%) ⬆️
src/settings/agent-actions.ts 96.04% <100.00%> (+0.11%) ⬆️
src/signals/engine.ts 97.35% <ø> (-0.01%) ⬇️
src/signals/focus-manifest.ts 98.56% <100.00%> (+<0.01%) ⬆️
src/signals/test-evidence.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 5, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui 3be6bfa Commit Preview URL

Branch Preview URL
Jul 05 2026, 02:36 AM

@JSONbored JSONbored force-pushed the codex/fix-review-hold-reasons branch 2 times, most recently from d8bf71d to fa3efae Compare July 5, 2026 04:22
@JSONbored JSONbored force-pushed the codex/fix-review-hold-reasons branch from fa3efae to 0ebfd63 Compare July 5, 2026 04:43
JSONbored added 4 commits July 4, 2026 22:06
… gaps

hasValidationNote matched any mention of testing regardless of polarity, so
a PR body like "No tests run" or "Tests not run" satisfied a configured
manifest test expectation and let manifest_missing_tests disappear on a
reachable live gate path. Add a negation guard covering both word orders
before falling through to the existing affirmative match.

Thread the same helper into the pre-submission predictor so it stops
predicting stricter than the live gate on this exact finding, and close the
codecov/patch gap: cover the truncated guardrail-path list, the all-unsafe
test-expectations case, and a null PR body through the live webhook path.
Simplify the redundant guardrailReason fallback in agent-actions.ts to a
single non-nullable default instead of three unreachable `?? fallback` arms.
…hrases

The first fix enumerated literal negation phrases keyed on the noun forms
"test"/"tests", which missed "Not tested locally." -- the verb form
"tested" only matched the affirmative branch, so a body stating tests were
NOT run could still satisfy a configured manifest test expectation.

Redesign hasValidationNote around a shared test/validation stem definition
and a word-proximity negation check (a negation word within a bounded
window of the stem, in either order) instead of enumerating more literal
templates, so the same class of miss cannot recur for a different
tense/form. The window is clause-bounded (stops at ,.!?;) so an unrelated
negation earlier in the body cannot suppress a later, real validation note.
… body

hasValidationNote ran its negation checks against the entire PR body and
returned false on the first match, so a genuine negated clause ("No tests
run locally.") vetoed a separate, later clause with real affirmative
evidence ("Validated with npm run test:ci."). Split on clause-boundary
punctuation and require only one clause to be an affirmative, non-negated
mention.

@gittensory-orb gittensory-orb Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.5x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant