Skip to content

feat(scoring): add noise-threshold to cap per-rule findings#23

Merged
hummbl-dev merged 1 commit into
mainfrom
feat/claude/noise-threshold
Apr 18, 2026
Merged

feat(scoring): add noise-threshold to cap per-rule findings#23
hummbl-dev merged 1 commit into
mainfrom
feat/claude/noise-threshold

Conversation

@hummbl-dev
Copy link
Copy Markdown
Owner

Summary

  • Adds filter_noise() to scoring.py — caps findings per rule_id at a configurable threshold
  • Adds --noise-threshold flag to all 6 scoring subcommands (analyze, score, score-url, diff, audit-fleet, github-top)
  • Reports suppressed findings count when active
  • Addresses 10 of 20 open gaps from the github-top audit run (false_positive_candidate class)

Usage

arbiter score /path/to/repo --noise-threshold 50    # cap each rule at 50 findings
arbiter github-top --noise-threshold 30             # cleaner fleet scoring

Test plan

  • No filtering when disabled (None)
  • Threshold zero disables
  • Caps at threshold per rule
  • Multiple rules independently capped
  • Returns suppressed counts dict
  • Rules below threshold unaffected
  • Preserves finding order
  • Full suite: 257 passed

🤖 Generated with Claude Code

When a single rule fires excessively (e.g. 100+ times in a large
repo), it inflates penalties and overwhelms output. --noise-threshold
caps findings per rule_id, reporting what was suppressed.

Wired into all 6 scoring subcommands: analyze, score, score-url,
diff, audit-fleet, and github-top.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@hummbl-dev hummbl-dev merged commit 57deb43 into main Apr 18, 2026
3 checks passed
@hummbl-dev hummbl-dev deleted the feat/claude/noise-threshold branch April 18, 2026 15:04
@hummbl-dev hummbl-dev restored the feat/claude/noise-threshold branch April 18, 2026 15:04
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac5d91f8ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/arbiter/scoring.py
Comment on lines +135 to +136
if not threshold:
return findings, {}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject negative noise thresholds

filter_noise treats any non-zero integer as enabled, so a negative --noise-threshold is accepted and causes every finding to be dropped (counts[...] <= threshold is never true). This produces artificially perfect scores and an incorrect suppressed count (total - threshold exceeds actual findings), which can silently corrupt CI gating if a typo or bad config passes -1; the new CLI flags do not validate this input before calling this function.

Useful? React with 👍 / 👎.

@hummbl-dev hummbl-dev deleted the feat/claude/noise-threshold branch April 19, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant