Benchmarks

Benchmarks

Every public claim is measured; this page carries the numbers and the method. Sources: BENCHMARKS.md, bench/, ONEPAGER.md.

navbench — the headline table (2026-10-01)

Apple M1, 8 GB RAM, macOS 26.7.1. Corpus: 100 local pages (50 static / 50 JS-built) on loopback HTTP — no real site crawled. Medians; RAM = peak of the whole process tree. navette read converts to markdown (more work than Playwright's innerText probe); both Playwright numbers include its full client stack.

Metric	navette (system WebKit)	Playwright + Chromium	Lightpanda (CDP)
Install size (what lands on disk)	0.6 MB	215.8 MB	96.1 MB
Cold start: launch → first page read (ms, median of 3)	669 ms	934 ms	33 ms
Navigate → readable content (ms, median)	11 ms	12 ms	12 ms
Act: type + click round-trip (ms, median)	2 ms	33 ms	18 ms
Peak RAM, whole process tree (MB)	79 MB	599 MB	41 MB
Crawl: 100 pages navigate+read (s)	1.5 s	1.8 s	0.9 s

Honest reading of the two rows Lightpanda wins — fresh-process boot and peak RAM: that's the price of rendering. Lightpanda parses a partial DOM and cannot screenshot (3.3–3.4 ms reads through raw CDP are the fastest page-reading number in the zero-bias probe). navette is the fastest full-rendering reader (17.9–19.4 ms vs Chromium's 28–34 ms through the identical client). If your agent only reads static pages, use fetch + readability; if it needs JS, sessions, actions and vision, that price is the product.

Cross-engine medians (CI runners, v1.6.0)

30-page local corpus, warm loop, bench/bench.py via the bench workflow (run of 2026-10-07). Shared CI machines — treat as relative, not absolute:

Engine	navigate+read	/read	/screenshot
macOS arm64 (WKWebView)	median 15.0 ms · p95 26.4 ms	median 0.8 ms	median 22.8 ms
Linux x64 (WebKitGTK)	median 5.7 ms · p95 6.0 ms	median 0.9 ms	median 2018 ms
Windows x64 (WebView2)	median 23.4 ms · p95 30.7 ms	median 1.7 ms	median 45.2 ms

Honest reading: warm navigate+read is single-digit-to-low-tens of milliseconds everywhere. The Linux screenshot path (on-screen move + GTK pump + X11 capture + restore) costs ~2 s — the price of capturing an engine wry doesn't let us snapshot directly; Windows (PrintWindow) and macOS (takeSnapshot) stay in the tens of milliseconds. Re-run with gh workflow run bench.

Daemon warmth (resident mode)

Path	Latency
Cold start via resident daemon (KeepAlive)	24–37 ms
First page on a warm daemon	24 ms
Re-warm after --idle-release drops a session	~100 ms (pre-warm path; the daemon never exits)

Corpus: recipe vs cold exploration (2026-10-13)

See Site Recipes for the full table — the short version: same task, same logged-in session, same human pacing → −98.8% bytes (109,296 → 1,326), ~27,000 → ~330 tokens, 23 s → 5 s, and stable dedup IDs from the first call. The replay script is bench/recipe/bench-notifications.sh.

Tollbooth — the walled web (v1.8.0+)

bench/tollbooth.py measures how hostile the modern web is to agents: 200 URLs × 5 categories, signals for robots.txt AI clauses, llms.txt, and x402/paywall signals (round 2). Results: bench/tollbooth-results.{csv,json}, summary in bench/tollbooth-summary.md. The dataset is CC BY 4.0 — take it, reuse it.

Also in the same lane: demo/signed-webhook — sealed counterparty events, fraud analysis as a diff over an HMAC-chained mock (closes #5).

Real-web success rate

Across the demo passes (Demos): navette completes 95–100% of real-web journeys vs 85% for the Playwright baseline on the same task set — the system WebView carries the full fingerprint and storage stack of a real browser.

Reproducing

gh workflow run bench          # cross-engine numbers on CI runners
python3 bench/bench.py         # local: same-machine, medians over repeats
python3 bench/cdp_raw.py       # the zero-bias raw-CDP probe (Lightpanda vs Chromium vs navette)
python3 bench/tollbooth.py     # the walled-web dataset

Protocol rules: same machine, same page set (static, JS-heavy, SPA), medians over repeated runs; the claims live and die by this — corrections go into the tables, not the footnotes.