read_page mode:'content' — rendered pages through the extraction pipeline

[jonybur]

Follow-on to #187, aimed at the OM2W failure taxonomy (budget exhaustion on tab-driven tasks — the dominant fixable bucket at the 26.7% baseline).

• read_page gains mode:'content': grabs the rendered DOM's outerHTML (post-JS — pages fetch_url can't see) and routes it through the same offscreen Readability+Turndown extractor and the same spill-and-page machinery fetch_url content pipeline: clean-markdown extraction + spill-and-page #187 shipped. One new tiny injected grabber; no new vendoring, no page-world library injection, same untrusted fence. Fail-open to the default snapshot everywhere (non-readerable / Firefox / extraction error / grab failure).
• Default snapshot mode untouched (content mode returns no interactables; the tool description teaches READ vs OPERATE).
• Paging-loop counter-pressure: the fetch_url content pipeline: clean-markdown extraction + spill-and-page #187 measurement caught one task walking a whole spilled document (465k actor tokens); read_web_cache now teaches deliberate paging.

Zero added authority: same gated dispatch, same fence, the markdown transform runs on bytes the actor could already read.

Tests: content-mode suite (extraction, spill+footer-outside-fence, all fail-open paths, default-mode untouched). Full battery green (2747 bun tests, strict typecheck, eslint, tscheck floor, no gen drift).

Measurement: the fetch suite covers the fetch path; a rendered-page before/after needs a tab-heavy task set — proposed as the same before/after protocol on the web-actor suite once #119 (which carries it) merges, then an OM2W-30 re-run to see if the baseline moves.

🤖 Generated with Claude Code

[jonybur]

Measured: Online-Mind2Web re-run on this branch — 46.7% (14/30), up from the 26.7% baseline

Same 30 tasks, same model (Opus 4.8), same judge (WebJudge/o4-mini), fresh export on this branch (= main incl. #119+#187, plus this PR's content mode).

	July 7 baseline	this branch
Pass rate	26.7% (8/30)	46.7% (14/30)
Median steps/task	11	8
25-step-cap deaths	6	4

Per-task: 6 both-pass, 8 gained, 2 lost (ordinary n=1 live variance — both are "didn't state the final answer" at similar step counts), 14 both-fail. The gains are the mechanism working, most visibly AeroAPI: 26 steps (capped, answer cut off) → 4 steps, PASS — the actor read the pricing page instead of thrashing against it. Amazon add-to-cart completed at 14→8 steps; the editor's-choice hunt came in under the cap at 26→19.

Attribution, honestly: the baseline predates the July merges, so this delta measures the whole content pipeline (#187 extraction + this PR's read_page content mode + paging) together with main's July fixes (#183, #184). Isolating this PR alone would cost another run; the composed number is the one that matters for the roadmap.

Context: this puts peerd above browser-use's independently verified 40% (full-300, Princeton HAL) on this 30-task subset — with the usual caveats (subset, ±8pp sampling noise, n=1 per task). The remaining failure mass is the incomplete-final-action bucket (14 both-fail), which is the next lever.

Scorecards: out/6aa56e07-pipeline{,_result} vs out/6aa56e07{,_result} under scripts/cdp/om2w/.

[NotASithLord]

Adversarial review pass on the diff, plus local verification (checked out the branch, ran the new test file and bun install).

Implementation: sound. mode:'content' is a faithful, minimal-risk extension of the #187 pipeline:

• Reuses the exact same webOffscreenClient.extractMarkdown + windowText/pagingFooter spill machinery fetch_url already ships, rather than reimplementing it — verified by diffing the two call sites (fetch-url.js:129-165 vs the new block in read-page.js). Same fencing (wrapUntrusted, footer outside the fence), same fail-open shape.
• Fail-open is real and multi-layered: no webOffscreenClient, readerable:false, a thrown extraction error, and a failed executeScript grab (grabbed?.html falsy) all fall through to the unchanged default snapshot path below. Ran the test suite locally (bun test tests/peerd-runtime/tools/read-page-content.test.ts) — 4/4 pass, and they do cover all three fail-open branches plus the untouched-default-mode case.
• readOuterHtmlInjected's classic-function-with-'use strict' shape matches the file's existing readPageInjected convention (both are chrome.scripting.executeScript-injected bodies, consistent with CLAUDE.md's injected-script exemption).
• The CAPABILITY_CONSUMERS.webCache update (adding 'read_page') is correct and necessary — content mode now reads ctx.webCache on overflow, so a narrowed/spawned actor granted read_page but not fetch_url/read_web_cache would otherwise silently lose the closure. Confirmed against spawn.js's strip logic — fails open to a direct slice, not a crash, same as fetch_url's own !webCache branch.
• 2MB injected-grab cap is a sane belt-and-suspenders bound ahead of the offscreen extractor's own cap.

CI is fully green (12/12 checks). No other correctness, security, or test-coverage issues found.

Blocking: missing DCO sign-off. Neither commit on this branch (21d3847e, the merge commit 15eaeebe) carries a Signed-off-by trailer. CONTRIBUTING.md/the PR template require git commit -s for every PR, and this exact gap blocked #170 last review pass. Please amend/rebase with -s before merge.

Once sign-off is added, this looks mergeable as-is.

---

Generated by Claude Code

[jonybur]

Signed — the branch is now a single DCO-signed commit (70df9cd); the unsigned merge commit is gone (history flattened onto main, tree verified byte-identical to the reviewed tip, so the 46.7% OM2W run above measured exactly this code).

[NotASithLord]

Dropping the DCO block — the sign-off checkbox is being removed from the PR template (history was 78% unsigned; the convention was never enforced). Prior review found the implementation sound and CI is 12/12 green, so approving on the merits.

---

Generated by Claude Code