hedged_claim reports the number of hedge patterns, not the number of hedged claims

[royalpinto007]

Desired outcome

The hedged_claim finding reports a number that matches what it says it is counting.

Why it matters

agentrace/checks.py, check_unverified_claim:

hits = []
for p in hedges:
    for m in re.finditer(p, run.result, re.I):
        hits.append(_context(run.result, m.start()))
        break
...
f"{len(hits)} hedged claim(s). ..."

The break leaves the inner loop after the first match, so hits gains at most one entry per pattern. len(hits) is therefore the number of distinct hedge phrasings found, never the number of hedged claims. A result that says "probably" fifteen times reports "1 hedged claim(s)". A result that says "appears to be" once and "unverified" once reports "2".

That is the wrong signal for the check's own stated purpose. The docstring is about a hedge being flattened into a fact downstream, and the reader is being asked to judge how much hedging there is. The number they are shown is capped at 6, the length of the hedges list, no matter how hedged the output is. Given the module docstring's line that "a checker that cries wolf gets ignored", a checker that under-counts by an order of magnitude is the same problem from the other direction.

Using re.finditer and then immediately breaking also makes the intent ambiguous to a reader: it looks like a counting loop that someone meant to let run.

Steps

1. Decide what the number means, then make the code say it. Either count every occurrence across all patterns (drop the break, keep only the first one or two contexts as evidence), or keep one hit per pattern and reword the message to "N kind(s) of hedging language".
2. Counting occurrences is the more useful of the two, since it distinguishes one hedge from fifteen.
3. Add a case in tests/test_agentrace.py with a result containing the same hedge three times and assert the reported count.

Small, one function.

Claiming this

Comment below to claim it. A reply usually comes within a day.