Skip to content

Releases

TrueCost 0.1.0 and the results ledger

Choose a tag to compare

@saisandeshk saisandeshk released this 27 Oct 08:29

The published results ledger behind every number in results/: 309 runs on NVIDIA Brev H100s, 13,940 graded tasks and 416,119 metered model calls (tokens, list price, GPU time and measured energy per call), including the invoice-fraud demonstration trials. It also holds Claude Code and Codex on Aider Polyglot exercises (read from the agents' own logs, with their plan-usage readings) and the hosted-API quickstart on NVIDIA's API catalog: 333 runs, 14,545 tasks and 418,812 calls in all.

  • truecost demo downloads it once (to ~/.truecost/results.db) and opens the console on it.
  • python evaluation/scripts/figures.py ~/.truecost/results.db redraws the figures and tables from the raw call records.

SQLite, xz-compressed (about 130 MB uncompressed). It holds call metadata and graded outcomes, not conversation text.

rater-retail.jsonl.xz: the independent safety rater's verdicts on 9,238 retail transcripts (every database write, judged against the written store policy, next to the rule checker's verdict). python evaluation/scripts/rater_summary.py rater-retail.jsonl regenerates the rater numbers in results/.