cost scorer passes with a perfect score when the provider reports no cost

[royalpinto007]

Desired outcome

The cost scorer does not award a perfect score when the provider reported no cost at all.

Why it matters

src/scorers/cost.ts:

const actual = ctx.response.costUsd ?? 0;
const passed = actual <= budgetUsd;

costUsd is optional on ProviderResponse (src/types.ts line 42) and is genuinely absent in normal use. Both real providers only compute it when a price is configured:

const costUsd =
  usage && this.config.costPer1kTokens
    ? (usage.totalTokens / 1000) * this.config.costPer1kTokens
    : undefined;

(src/providers/anthropic.ts around line 80, and the same in src/providers/openai-compatible.ts.) So anyone who adds a cost scorer without also setting costPer1kTokens on their provider gets actual = 0, passed = true, score = 1, and a reason line reading $0.000000 vs budget $.... The budget gate is on, green, and measuring nothing.

That is the same failure shape as the open issue about a zero-case run reporting passed: a gate that silently disables itself is worse than no gate, because the user believes they are protected. It also inflates the aggregate score for every case, since a scorer returning 1 for free pulls the weighted average up.

The $0.000000 in the reason is not a strong enough signal, because for a truly free provider (the mock) zero is also the correct answer.

Steps

1. Distinguish absent from zero: when ctx.response.costUsd === undefined, do not pass. Return passed: false with a reason like "provider reported no cost (set costPer1kTokens on the provider)", or introduce an explicit skipped-style outcome if the type system allows one.
2. Whatever is chosen, make sure it does not quietly count as a full-credit score in aggregateScore (src/runner.ts line 69).
3. Add a test in tests/scorers.test.ts with a response whose costUsd is undefined, asserting the scorer does not report a passing perfect score.
4. Mention the costPer1kTokens requirement in the cost scorer docs in README.md.

Claiming this

Comment below to claim it. A reply usually comes within a day.