shipit-watcher 1.4.0

Per-agent datasets

wt.agent_dataset_name(agent) → myapp-eval-inbox-manager. One dataset per agent, for the same reason prompts are namespaced per agent: a failing case for one says nothing about another, and a shared dataset makes every run an average of unrelated cases.

Cost is no longer silently zero

Streaming calls through a gateway routinely arrive with every cost field at zero — the response is reassembled from chunks and the proxy's figure never makes it back. Cost is now computed from tokens instead, with two guards:

• The routing prefix is stripped. Pricing knows gemini-2.5-flash; openai/gemini-2.5-flash is a routing instruction that matches no pricing entry.
• Unpriced models report nothing, not a guess. litellm's helpers price models they do not recognise at a generic rate; putting that number on a private deployment is worse than reporting none, because an invented figure looks authoritative.

Going to production

A new README section covering the five things to set up in order — environment, configure-once-at-startup, the ledger migration, seeding the prompt registry without overwriting edits made in Langfuse, and letting datasets fill themselves — plus a copy-pasteable verification block.

290 tests.