Skip to content

v0.2.79

Choose a tag to compare

@github-actions github-actions released this 09 Oct 15:23
· 49 commits to main since this release

[0.2.79] — 2026-10-09

Changed

  • Every AA figure names the effort it was measured at.
    • /models shows the headline AA Index with its effort ("57.6 Max"), in the
      cell, the hover card and the screen-reader label. The headline is each
      model's top measured effort.
    • The /recommend matrix rates each pick at the effort it runs.
      update/derive_ratings.py --write now also writes
      docs/effort-ratings.json: the letters a model's run at each of its other
      measured efforts earns. That run takes its headline row's place among the
      live models, and derive() is unchanged. Both crons commit the file, and
      --check fails when it is stale.
    • For example, Sonnet 5.5 at Medium reads A in coding, planning, agentic and
      long-context and B in knowledge, where its headline (Max) reads S, S, S,
      A and A. A category AA did not measure at that effort keeps its headline
      letter.
    • Display only: the scorer already reads evidence at the pick's effort.

Fixed

  • A model's headline benchmark row is its top measured effort. That was
    true of 55 of 56 models; AA's default row for Grok 4.6 is High although it
    measured XHigh. update/fetch_aa_benchmarks.py now re-bases any such model
    on its top effort, and the default row becomes one of its levels. Grok 4.6
    reads 44.2 at XHigh (was 44.3 at High). By rank, Opus 5 moves up to an S
    in coding, and the generated prose follows. No ladder row changes.
  • Muse Spark 1.3 is token-measured (#1028): 2.449 at XHigh and at Max,
    replacing the uniform 1.6 and 2.5. No ladder row changes.