Repository navigation
Why this release
0.3.0 adds the merged Web Search/SERP workflow to Browser Eval and includes the shared Web Fetch capture and source-validation fixes. It is a minor software release because it adds a track and compatible capabilities to the 0.x interface.
Changes since 0.2.0
- Add reusable Search/SERP collection, scoring and reports (#58).
- Keep targeted evidence refreshes within their declared scope (#57).
- Share source validation and bounded browser capture (#61).
- Align package, lock and runtime versions; verify release-tag identity and keep project citations version-neutral (#68).
Included benchmark versions
| Component | Version | Pinned source |
|---|---|---|
| Browser Eval | 0.3.0 | f5c56767c54b65f86fc2397c4be7deb91e1705d0 |
| LexBench Web Fetch | benchmark 2.0.1 | d0e78e71cf5145217cb1f2b71420a7c4194d80dd |
| LexBench Web Search | benchmark 1.0.0 | d312653cbdc0396ffefa49ced37f82f7ca764da2 |
Fetch's catalog contains 550 entries. Search selects scenarios-v2 (149 cases) and factual-v1 (831 cases); the earlier scenarios-v1 snapshot remains available. A report's own manifest selects its actual population. WebMainBench, WCXB, Browser Use and Trajectory Replay retain their separate dataset and environment contracts.
The pending Fetch rule revision (#12 in lexbench-webfetch) and Search edition 1.1.0 are not included. Installing this software does not change historical report scores or adopt newer upstream benchmarks.
Validation and installation
The release commit has the same source tree as the tested PR: make ci passed 7,028 tests and 2,972 subtests, fixture workflows and four isolated installation configurations. Optional external-input/environment tests account for 129 skips. Release metadata and matching-tag checks pass on the merged commit. The attached wheel and source archive were built from that commit.
uv pip install 'git+https://github.com/lexmount/browser-eval.git@v0.3.0'Python 3.11 or later is required. Additional tracks retain their documented optional dependencies. Existing report and benchmark bindings remain unchanged. Compare software changes.
Search catalog binding
The pinned Search 1.0.0 repository revision contains scorer changes beyond the scoring source recorded in its frozen catalog. Its strict catalog verification fails, even though the installed source remains identified by the immutable dependency pin. For the original measurement contract, combine the 1.0.0 catalog with its recorded scorer commit, as documented in that benchmark release.
Browser Eval PR #69 adopts the verified Search 1.1.0 dependency for subsequent work. Dependency synchronization does not itself schedule another software release. Compare Search scores only after rescoring both cohorts under the same benchmark edition. This 0.3.0 tag and its original dependencies remain unchanged.