You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Every AA figure names the effort it was measured at.
/models shows the headline AA Index with its effort ("57.6 Max"), in the
cell, the hover card and the screen-reader label. The headline is each
model's top measured effort.
The /recommend matrix rates each pick at the effort it runs. update/derive_ratings.py --write now also writes docs/effort-ratings.json: the letters a model's run at each of its other
measured efforts earns. That run takes its headline row's place among the
live models, and derive() is unchanged. Both crons commit the file, and --check fails when it is stale.
For example, Sonnet 5.5 at Medium reads A in coding, planning, agentic and
long-context and B in knowledge, where its headline (Max) reads S, S, S,
A and A. A category AA did not measure at that effort keeps its headline
letter.
Display only: the scorer already reads evidence at the pick's effort.
Fixed
A model's headline benchmark row is its top measured effort. That was
true of 55 of 56 models; AA's default row for Grok 4.6 is High although it
measured XHigh. update/fetch_aa_benchmarks.py now re-bases any such model
on its top effort, and the default row becomes one of its levels. Grok 4.6
reads 44.2 at XHigh (was 44.3 at High). By rank, Opus 5 moves up to an S
in coding, and the generated prose follows. No ladder row changes.
Muse Spark 1.3 is token-measured (#1028): 2.449 at XHigh and at Max,
replacing the uniform 1.6 and 2.5. No ladder row changes.