Browser Eval 0.3.0

Why this release

0.3.0 adds the merged Web Search/SERP workflow to Browser Eval and includes the shared Web Fetch capture and source-validation fixes. It is a minor software release because it adds a track and compatible capabilities to the 0.x interface.

Changes since 0.2.0

• Add reusable Search/SERP collection, scoring and reports (#58).
• Keep targeted evidence refreshes within their declared scope (#57).
• Share source validation and bounded browser capture (#61).
• Align package, lock and runtime versions; verify release-tag identity and keep project citations version-neutral (#68).

Included benchmark versions

Component	Version	Pinned source
Browser Eval	0.3.0	f5c56767c54b65f86fc2397c4be7deb91e1705d0
LexBench Web Fetch	benchmark 2.0.1	d0e78e71cf5145217cb1f2b71420a7c4194d80dd
LexBench Web Search	benchmark 1.0.0	d312653cbdc0396ffefa49ced37f82f7ca764da2

Fetch's catalog contains 550 entries. Search selects scenarios-v2 (149 cases) and factual-v1 (831 cases); the earlier scenarios-v1 snapshot remains available. A report's own manifest selects its actual population. WebMainBench, WCXB, Browser Use and Trajectory Replay retain their separate dataset and environment contracts.

The pending Fetch rule revision (#12 in lexbench-webfetch) and Search edition 1.1.0 are not included. Installing this software does not change historical report scores or adopt newer upstream benchmarks.

Validation and installation

The release commit has the same source tree as the tested PR: make ci passed 7,028 tests and 2,972 subtests, fixture workflows and four isolated installation configurations. Optional external-input/environment tests account for 129 skips. Release metadata and matching-tag checks pass on the merged commit. The attached wheel and source archive were built from that commit.

uv pip install 'git+https://github.com/lexmount/browser-eval.git@v0.3.0'

Python 3.11 or later is required. Additional tracks retain their documented optional dependencies. Existing report and benchmark bindings remain unchanged. Compare software changes.

Search catalog binding

The pinned Search 1.0.0 repository revision contains scorer changes beyond the scoring source recorded in its frozen catalog. Its strict catalog verification fails, even though the installed source remains identified by the immutable dependency pin. For the original measurement contract, combine the 1.0.0 catalog with its recorded scorer commit, as documented in that benchmark release.

Browser Eval PR #69 adopts the verified Search 1.1.0 dependency for subsequent work. Dependency synchronization does not itself schedule another software release. Compare Search scores only after rescoring both cohorts under the same benchmark edition. This 0.3.0 tag and its original dependencies remain unchanged.

Browser Eval 0.2.0

Why this release

0.2.0 gives report builds a shared software identity and provides reusable Web Fetch reporting with an independently pinned LexBench evaluator.

Included state

• Software: Browser Eval 0.2.0, commit 30645af1ba2520b4d089cbef32979e088cffe277.
• Web Fetch dependency: benchmark 2.0.1, commit d0e78e71cf5145217cb1f2b71420a7c4194d80dd.
• Web Fetch, Browser Trajectory Replay and Browser Use have their own runners and report contracts. This release predates the merged Web Search integration.
• The 550-entry Fetch catalog does not determine a report's selected population; each report retains its own inputs and scorer bindings.

Compared with 0.1.0, this release includes the extracted benchmark dependency, shared report-build provenance and Apache-2.0 project licensing. Compare source changes.

The original annotated tag has been restored to its original commit. No code or historical measurements were replaced.

Browser Eval 0.1.0

Why this release

0.1.0 preserves the original installable Browser Eval software checkpoint for reproducing earlier work.

Included state

• Software commit: 03c9a5e94619000480c92b8e517c16165dc2a7f4.
• Workflows: Web Fetch, fixed Browser Trajectory Replay and Agent Browser Use, plus an offline success/failure example.
• Web Fetch scoring implementations live inside this Browser Eval revision; no separate LexBench Fetch or Search package is declared.
• The default report profile in this revision selects a 200-task WebMainBench sample. Other report populations are defined by their own profiles; the software version does not select one universal benchmark.

This is a historical 0.x interface, before the extracted benchmark dependency and later Search integration. Use the tag's own installation instructions and dependency lock. The original annotated tag has been restored without changing its source.