Skip to content

feat(review): tag AI review findings with a category taxonomy#3634

Merged
loopover-orb[bot] merged 1 commit into
mainfrom
feat/inline-finding-category-taxonomy
Jul 5, 2026
Merged

feat(review): tag AI review findings with a category taxonomy#3634
loopover-orb[bot] merged 1 commit into
mainfrom
feat/inline-finding-category-taxonomy

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Adds a category field to inline AI review findings — one of security, correctness, performance, maintainability, tests, style. The model self-categorizes when asked; a new deterministic path/keyword classifier (classifyFindingCategory in src/review/finding-category-classify.ts) supplies a safe default for whatever it omits or mis-emits, so a caller with the feature on always has a category to render — never a sometimes-present field.
  • Surfaced in two places: the inline PR-review comment label (**Nit (security):** ...) and a new "Finding categories" collapsible in the unified comment (a count per category).
  • Gated by a new review.finding_categories manifest toggle (default off), layered on top of review.inline_comments being on already (mirrors review.suggestions exactly — a category has nothing to categorize without an inline finding) — no new global env var, no DB/OpenAPI surface.

Closes #1958.

Example inline comment label: **Nit (security):** This query is vulnerable to SQL injection.

Example unified-comment collapsible:

| Category | Findings |
| --- | --- |
| Security | 1 |
| Correctness | 2 |

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue (Finding category taxonomy on AI review findings #1958).

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally — 100% line + branch coverage on every changed line, verified via a targeted v8 coverage pass across every touched file (processors.ts's finding-categories gating ternary, ai-review.ts's category parsing + system-prompt suffix, inline-comments.ts's category-tag rendering, unified-comment-bridge.ts's new collapsible) in addition to the full unsharded run.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries: test/unit/finding-category-classify.test.ts (the new pure classifier), test/unit/finding-category-collapsible.test.ts (the new bridge collapsible + wiring, mirroring visual-collapsible.test.ts), test/unit/ai-review.test.ts (category parsing, system-prompt gating, composeInlineFindings carry-through), test/unit/inline-comments.test.ts (label rendering, model-provided vs. fallback category, end-to-end), test/unit/focus-manifest.test.ts (parse/serialize/round-trip/resolve for the new manifest field), and a new test/unit/queue.test.ts end-to-end webhook test with a mocked AI response asserting both surfaces render the category.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — no auth/session/CORS surface touched.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — no API/OpenAPI/MCP schema change; this is a manifest-only review.* field like review.suggestions/review.changed_files_summary.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (N/A — no apps/gittensory-ui changes.)
  • Visible UI changes include a UI Evidence section below with screenshots. (N/A — the visible output is a category tag inside a bot-posted GitHub PR comment, not an apps/gittensory-ui page; see the example labels above instead.)
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. (README.md's "Inline review comments" bullet extended to mention the new toggle.)

Notes

  • category is parsed but never defaulted inside parseModelReview/composeInlineFindings — the deterministic fallback is applied at render time (formatInlineBody, buildFindingCategoryCollapsible), mirroring how review.suggestions already separates parsing (always capture what the model gives) from rendering (gate on the manifest toggle).

Adds a category field (security/correctness/performance/
maintainability/tests/style) to each inline finding: the model
self-categorizes when asked, and a deterministic path/keyword
classifier (classifyFindingCategory) supplies a safe default for
whatever it omits, so a caller with the feature on never sees a
partial tag.

Surfaced in the inline-comment label and a new "Finding categories"
collapsible in the unified comment. Gated by review.finding_categories
in .gittensory.yml (default off, layered on review.inline_comments
like review.suggestions) -- no new global env var, no DB/OpenAPI
surface.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 5, 2026
@loopover-orb

loopover-orb Bot commented Jul 5, 2026

Copy link
Copy Markdown

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-05 20:39:35 UTC

14 files · 1 AI reviewer · no blockers · readiness 82/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR adds a `category` taxonomy field to AI-review inline findings, gated by a new `review.finding_categories` manifest toggle layered on top of `review.inline_comments`/`review.suggestions`, mirroring the existing #1956 suggestions pattern almost line-for-line (parse-with-enum-guard, deterministic fallback classifier, dual-surface rendering in inline comments and the unified comment's new collapsible). The wiring is complete across schema/defaults/parsing/serialization in focus-manifest.ts, the AND-gating with inlineFindings is applied consistently at both the prompt-construction site (ai-review.ts) and the render-gating site (processors.ts), and tests cover the enum guard, fallback classification, rendering toggles, and end-to-end processor flow. No correctness defects found; this is a narrow, well-scoped, byte-identical-when-off feature addition.

Nits — 6 non-blocking
  • src/review/inline-comments.ts: `shouldRenderFindingCategories` is byte-identical to `shouldRenderSuggestions` (same signature, same body) — consider a single shared `andGate(base, toggle)` helper to avoid drifting duplicate logic.
  • The flagged 'generic_secret_assignment' at test/unit/queue.test.ts around the GITHUB_APP_PRIVATE_KEY line looks like a false positive — it's `await generatePrivateKeyPem()`, a freshly generated test key, not a hardcoded credential, but worth a quick confirm.
  • src/queue/processors.ts, src/services/ai-review.ts, src/signals/focus-manifest.ts, and src/review/unified-comment-bridge.ts are already large files and this PR adds more surface to each rather than extracting the finding-category logic further — no action needed now but worth watching as these keep growing.
  • The `FINDING_CATEGORY_SUFFIX` prompt-instruction string in ai-review.ts duplicates the enum list already exported as `FINDING_CATEGORIES` — consider deriving the suffix text from that array so the two can't drift.
  • Consider extracting the `base && toggle === true` gating pattern (now duplicated across `shouldRenderSuggestions`/`shouldRenderFindingCategories`) into one shared predicate in inline-comments.ts.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #1958
Related work ⚠️ 3 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 56 registered-repo PR(s), 46 merged, 493 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 56 PR(s), 493 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 56 PR(s), 493 issue(s).
  • Related work: Titles/paths share 6 meaningful terms. (issue #2150, issue #2159)
  • Related work: Titles/paths share 6 meaningful terms. (issue #2150, issue #2185)
  • Related work: Titles/paths share 6 meaningful terms. (issue #2149, issue #2159)
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Review top overlaps.
  • Add a concise scope and risk note.
  • No action.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.23%. Comparing base (ad581ac) to head (2fa0aa0).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3634      +/-   ##
==========================================
+ Coverage   93.20%   93.23%   +0.02%     
==========================================
  Files         313      314       +1     
  Lines       31910    31952      +42     
  Branches    11677    11698      +21     
==========================================
+ Hits        29741    29789      +48     
  Misses       1517     1517              
+ Partials      652      646       -6     
Files with missing lines Coverage Δ
src/queue/processors.ts 93.75% <100.00%> (+0.15%) ⬆️
src/review/finding-category-classify.ts 100.00% <100.00%> (ø)
src/review/inline-comments.ts 100.00% <100.00%> (ø)
src/review/unified-comment-bridge.ts 98.00% <100.00%> (+0.20%) ⬆️
src/services/ai-review.ts 96.97% <100.00%> (+0.02%) ⬆️
src/signals/focus-manifest.ts 99.00% <100.00%> (+<0.01%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 87043af into main Jul 5, 2026
10 checks passed
@loopover-orb
loopover-orb Bot deleted the feat/inline-finding-category-taxonomy branch July 5, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Finding category taxonomy on AI review findings

1 participant