Skip to content

v1.108.312 — A count with no time basis is read as per-request

Latest

Choose a tag to compare

@jgravelle jgravelle released this 31 Aug 01:00

Every published schema-token figure now carries its basis.

schema_tokens_avoided shipped as a bare count — in get_session_stats's tool_surface block and in jcodemunch-mcp surface. A count with no time basis gets one supplied by the reader, and the one a reader supplies is per request.

That is the exact framing benchmarks/codex_surface/ rules out in our own words, having measured 86% of baseline input cached (1,938,176 of 2,247,575). The tool-schema block is stable, so it is paid at full rate roughly once per cache lifetime and at cache-read rates (~0.1x) thereafter. That artifact also records that this repository used the "N tokens in every request" framing before measuring. The artifact knew and the shipped field did not — the same gap v1.108.311 closed one field over.

Schema tokens avoided: 617 (estimator: bytes/4)
  basis: one_time_at_full_rate_then_cache_read
  The tool-schema block is stable across requests, so it is paid at full rate
  approximately once per cache lifetime and at cache-read rates (~0.1x)
  thereafter. This count is payload size, NOT a per-request saving...

The basis travels on both surfaces. A person reading surface is exactly who supplies the wrong basis, so a machine-readable field the CLI did not print would have left the human surface carrying the defect.

The fix is a label, never a scaled number. A count quietly multiplied by the cache-read rate answers neither the payload question nor the cost question, and nothing on the wire would show it had happened — the same rule as analyze_perf's raw hit_rate, kept beside hit_rate_basis rather than replaced. A test asserts the counts still equal the summed weights.

⚠ Third instance of one shape, after hit_rate_basis and basis: excess_calls: a figure whose period or denominator is unstated gets a wrong one supplied for free.

Three bindings in tests/test_tier_switch_cost.py (20 total), each red against its own reintroduction: basis fields removed from the payload, count silently discounted, CLI stops printing it.

Tests: 9025 passed, 13 skipped, 0 failed; 3.13 CI reproduce 9019 passed, 19 skipped; ruff clean; all 8 matrix jobs plus lint green on c9fda11 before anything shipped.