What's the canonical pinchbench right now

[chrispatil]

I love pinchbench.com but it's showing not updated since 4/13, and the "newer"-sounding versions have far fewer tests. Is there a plan to update and make v2 the official?

[ScuttleBot]

👋 Hi @chrispatil! I'm @olearycrew's OpenClaw bot, helping triage this repo.

The canonical PinchBench is the main branch of the pinchbench/skills repo, which currently tracks toward v2.0.0. The leaderboard at pinchbench.com shows results from the last official runs (April 13, 2026) using benchmark version 1.2.1.

You're right that the newer versions have fewer official runs — that's because v2.0.0-rc introduces significantly more tasks (55+, up from ~23), and re-running all models against the expanded suite takes time and compute. There's an open request to refresh the official runs (#344).

TL;DR:

• For running benchmarks locally: Use the latest main branch (v2.0.0-rc2 infrastructure)
• For comparing against leaderboard results: The public leaderboard currently shows v1.2.1 results
• Official refresh: Pending — tracked in Could the Official Runs Be Updated? #344

Feel free to ignore if this isn't helpful!

[olearycrew]

We are actively working on making 2.0.0 official once we have a set of official runs and repeatable results. So there isn't a specific date in mind, but we are working on it.

I had hoped for this week, but we are a bit behind given some issues that have come up.