Summary
Every finding carries confidence and costImpactUsd. Severity is computed from those signals, not hardcoded per detector.
Why
Today, a high severity finding on a free endpoint and a high severity finding on a gpt-4o polling loop look identical. Users can't filter by confidence on local-rule findings (only AI ones). Severity can't be calibrated against benchmark data.
Current state
- AI findings carry confidence (filtered by
eco.aiReview.minConfidence)
- Local-rule findings have no confidence
- Severity hardcoded per detector in
src/scanner/local-waste-detector.ts
What to do
- Add to every detector emission:
interface FindingSignals {
confidence: number; // 0..1
costImpactUsd: number | null;
frequencyClass: FrequencyClass;
}
- Local detectors start with fixed confidence per type, refined by C1 calibration.
- Derive severity at one place:
function deriveSeverity(s: FindingSignals): "high" | "medium" | "low" {
const score = s.confidence * (s.costImpactUsd ?? 0);
if (score >= 100) return "high";
if (score >= 10) return "medium";
return "low";
}
- Tune thresholds against the benchmark.
- Add UI: filter by confidence, sort by cost impact.
Acceptance criteria
Depends on
- C1 (per-detector confidence values)
- D1 (threshold tuning)
Reference
Full design: https://github.com/recost-dev/extension/blob/main/docs/accuracy/findings.md#c3-confidence-as-a-first-class-field-severity-derived-from-signals
Summary
Every finding carries
confidenceandcostImpactUsd. Severity is computed from those signals, not hardcoded per detector.Why
Today, a
highseverity finding on a free endpoint and ahighseverity finding on a gpt-4o polling loop look identical. Users can't filter by confidence on local-rule findings (only AI ones). Severity can't be calibrated against benchmark data.Current state
eco.aiReview.minConfidence)src/scanner/local-waste-detector.tsWhat to do
Acceptance criteria
FindingNodehasconfidence+costImpactUsdDepends on
Reference
Full design: https://github.com/recost-dev/extension/blob/main/docs/accuracy/findings.md#c3-confidence-as-a-first-class-field-severity-derived-from-signals