Skip to content

v1.5.0 — your threshold, not somebody else's

Choose a tag to compare

@Priyanshu-byte-coder Priyanshu-byte-coder released this 10 Sep 19:34
· 13 commits to main since this release

A degradation threshold measured on one agent was being shown on another.

If your OpenCode sessions degraded at 60% context fill, Claude Code's statusline told you knee ~60% — even when Claude Code's own curve had no knee at all. Calibration stored a single curve blended across every agent, model and project, and every live surface read that one number.

Model mixing was worse: a 200k-window model and a 1M-window model have different curves and different denominators, so averaging them described neither. The per-scope curves were already being computed for the report, then thrown away before reaching the surface where they matter.

Fixed

Curves are scoped. The snapshot (schema 2) now stores a curve per agent, per model family and per agent+model pair alongside the global one. The statusline, the warning hook and contextrot status each resolve the scope matching the session in front of them:

agent+model  ->  model  ->  agent  ->  global

Every rung must clear the trust floor on its own, so a thin scope falls through to a broader one instead of quoting a threshold built from noise.

A curve that blends over the other axis — one model across several agents, or one agent across several models — still gets quoted, because it fixes at least one variable. But it renders as knee ~60% (all agents) rather than being passed off as yours. With a single-agent, single-model history nothing is labelled: there the global curve simply is your curve.

The old schema is treated as absent rather than upgraded. This is a cache, and the next contextrot run rewrites it.

Whole model families were grouped as "unknown". Found while verifying the above on real data: model_family() recognised only Opus, Sonnet and Haiku. So claude-fable-5 — a first-class Anthropic model already priced with a 1M window — landed in an "unknown" bucket together with every GPT, Qwen and Nemotron step, and a single curve was fitted across all of them. That is the same contamination, one layer down.

Fable and Mythos now group as Anthropic families, other vendors get their own family key derived from the model id (gpt-5.6-terra and gpt-5.6-sol share gpt-5.6), and "unknown" is reserved for things that genuinely are not a model id, such as Claude Code's <synthetic> placeholder.

Added

  • contextrot doctor tables every measured curve — scope, steps, threshold, and how deep the data actually reaches — so it is visible at a glance which slice the statusline is quoting, and which scopes blend:
  Measured curve           Steps  Threshold  Measured to
  Claude Code + Opus 5    21,367  none                 —
  Claude Code + Opus 4.8   3,619  none          60% full
  Claude Code + Fable 5    2,484  none          40% full
  Opus 5                  21,378  none                 —  blends all agents
  Claude Code             28,269  none                 —  blends all models
  all agents and models   28,448  none                 —  blends all agents
  • contextrot status --format json gained scope, scope_label and scope_is_fallback.

Changed

  • load_calibration() returns a CalibrationSet; callers pick a curve with .resolve(agent=..., model=...). Live surfaces still take a single Calibration, so they cannot render a number from the wrong scope by accident.
  • model_family() / model_label() moved to a dependency-free contextrot.modelkey, still importable from analysis.by_model. The live surfaces need them on every render, and reaching them through the analysis package pulled in the adapters and the whole statistics layer — about 16 ms of import time on a path that runs every few seconds.

Upgrade: pip install --upgrade contextrot, then run a plain contextrot once to rewrite the calibration with scoped curves.

Full changelog: https://github.com/Priyanshu-byte-coder/contextrot/blob/main/CHANGELOG.md