Skip to content

Bounty Operator 0.9.4

Choose a tag to compare

@ichikuruma22-star ichikuruma22-star released this 09 Oct 17:13
· 35 commits to main since this release
Immutable release. Only release title and notes can be modified.

Site

  • /benchmark has a new table, "Right, wrong and not finished". It splits
    every ranked model's inputs into those it answered right, those it
    answered wrong and those it did not finish, with the reason. An unfinished
    input scores as wrong, so a low rank can mean either, and the table tells
    them apart: of 792 inputs, 646 were answered right, 85 wrong and 61 were
    not finished.
  • The same section says why two models rank low without one wrong answer.
    Claude Sonnet 5.5 lost 13 inputs, 11 cut off at the output limit and 2
    timed out. That is the agent run at the highest effort; a review in the
    workbench is one request, and Sonnet 5.5 is its default model on
    OpenRouter.