docs: refresh BU Bench harness plot with current model results - #143
Merged
Conversation
Propagates browser-use/benchmark-x-laminar@45b563b. The README copies were generated 2026-07-29 and predate seven changes to the plot: - claude-opus-5 (87.0%), claude-fable-5 (87.0%), grok-4.5 (86.3%), kimi-k3 (86.0%), gpt-5.6-sol/terra/luna, qwen3.8-max (79.0%) and deepseek-v4-flash-0731 (76.0%) added - base deepseek-v4-flash and gpt-5.6-luna-medium dropped - open-weight bars are now a gray fill with a blue dashed outline instead of a solid blue fill
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Copies the current
browser_harness_by_model_{light,dark}.pngfrombenchmark-x-laminar@45b563b into
static/.The versions in the README were generated 2026-07-29 and are several
generations behind. Changes now reflected:
claude-opus-5(87.0%),claude-fable-5(87.0%),grok-4.5(86.3%),kimi-k3(86.0%),gpt-5.6-sol(84.0%),gpt-5.6-lunaxhigh (82.0%) / default (72.4%),gpt-5.6-terra(72.0%),qwen3.8-max(79.0%),deepseek-v4-flash-0731(76.0%)deepseek-v4-flash(superseded by-0731),gpt-5.6-luna-medium, and the stalemimo/glm-4.7/kimi-k2.6/qwen3.6-plus/grok-4.3barsNot in this PR
Two things above the bar chart are also stale and need a decision before
they can be propagated:
browsercode_best_models_{light,dark}.png(the radial infographic) ishand-authored HTML upstream, not generated from the results, so it cannot be
mechanically refreshed. It still shows Opus 4.7 / GPT-5.5 / GLM 5.2 /
Gemini 3.1 Pro / Minimax M3, and its numbers no longer match the results
(e.g. Minimax M3 78.0% vs 74.0% measured).
categories. Against current data, three of the five have changed:
best speed
gpt-5.5(17.4 tasks/hr) ->gpt-5.6-terra(29.3);best open-weight
glm-5.2(84.0%) ->kimi-k3(86.0%);lowest cost
minimax-m3->deepseek-v4-flash-0731($1.60/100 tasks).Left unchanged here so the list does not contradict the infographic
directly above it.
Summary by cubic
Refreshed the BU Bench harness chart images to match the latest upstream results, so the README shows current model performance. Adds the newest model bars, removes outdated ones, and updates open‑weight bar styling (gray fill with a blue dashed outline).
Written for commit 6f40bce. Summary will update on new commits.