Skip to content

Web Fetch benchmark 2.0.1

Latest

Choose a tag to compare

@lanyue-llk lanyue-llk released this 23 Oct 09:41

The catalog contains 550 tasks across five suites, with frozen task/reference hashes and a bound scoring source.

Benchmark tags use bench-vMAJOR.MINOR.PATCH. Browser Eval software releases use vMAJOR.MINOR.PATCH; reports record both identities. This naming change leaves task inputs, acceptance rules and the scorer unchanged.

Edition 2.0.1 includes complete compatibility fingerprints for ordered-tree references, thresholds and list identity/name/scope/price rules. Changes to those rules require a minor edition and rescoring saved outputs under the same edition.

Validation: 2,579 tests and 321 subtests passed.

Release identity and comparison

This is the expanded 550-entry catalog: 150 Web Fetch, 100 popular-site, 100 dynamic-content, 100 access-restriction and 100 article tasks. Source commit d0e78e71cf5145217cb1f2b71420a7c4194d80dd.

The 2.x series separates this population from the historical 430-task report. Patch 2.0.1 completes compatibility fingerprints without changing the measured task contract. Later scoring revisions are not included in this tag.

Select the immutable benchmark tag or full commit to reproduce their exact code. This single benchmark tag is the release entry point.