browser-bakeoff

Run the same authenticated task — log into a target billing portal (configured via env) and fetch the bill (download the most recent statement PDF) — against three AI-browser providers, side-by-side, and compare DX / speed / reliability / cost. Success = a real PDF lands in results/.

• Browser Use — an AI agent that drives a browser. You hand it a prompt; it does the task.
• Browserbase — browser infrastructure (cloud Chromium) + the Stagehand SDK on top.
• Anchor — infra + agent + the best persistent-login story (OmniConnect).

All three are configured here for US residential proxy + stealth + captcha solving + recording/live-view, so it's an apples-to-apples comparison.

What each API does (cheat sheet)

	Browser Use	Browserbase	Anchor
Base URL	api.browser-use.com/api/v2	api.browserbase.com/v1	api.anchorbrowser.io/v1
Auth header	X-Browser-Use-API-Key	X-BB-API-Key	anchor-api-key
NL "do this task"	POST /tasks	Stagehand act/agent	POST /tools/perform-web-task or session agent/run
Self-drive (CDP)	POST /browsers → cdpUrl	POST /sessions → connectUrl	POST /sessions → cdp_url
Structured output	structuredOutput JSON Schema	Stagehand extract + Zod	output_schema JSON Schema
Secrets channel	secrets	Stagehand variables	secret_values
Persistent auth	Profiles	Contexts	Profiles + OmniConnect (MFA/OTP)
Built-in agent LLM	yes (bu-3) or BYO	no — bring your own	yes (pick engine) or BYO
Free tier	3 concurrent, no CC	1 hr / 3 concurrent / 15-min	$5 credits, 5 concurrent

Cost drivers to watch: proxy bandwidth (BU $5/GB · BB $10–12/GB · Anchor $8/GB) and agent steps/tokens. Browser time is cheap (~$0.02–0.05/hr).

Fetching the downloaded file (the actual deliverable)

The interesting difference for "fetch a bill" is how each captures a browser download:

	Browser Use	Browserbase	Anchor
Enable downloads	automatic	CDP Browser.setDownloadBehavior, path = "downloads"	session pdf_viewer.active:false
List files	task outputFiles[]	GET /v1/downloads?sessionId=	GET /v1/sessions/{id}/downloads
Get bytes	per-file presigned URL (120s TTL)	GET /v1/downloads/{id} + Accept: application/octet-stream	…/downloads/{id}/fetch → presigned SignedUrl
Effort	lowest (auto-capture)	highest (must wire CDP + Playwright download event)	medium

All three are implemented in src/adapters/*. Files are saved as results/<provider>-<filename>.

Setup

cd browser-bakeoff
npm install
cp .env.example .env     # then fill in keys, TARGET_URL/TARGET_DOMAIN, + site creds (PowerShell: copy .env.example .env)

Keys: Browser Use → cloud.browser-use.com · Browserbase → dashboard (API key + project id) · Anchor → app.anchorbrowser.io/api-key · plus an OPENAI_API_KEY for Stagehand.

Run

npm run bake                          # all three, sequentially
npm run bake -- --only=anchor         # just one
npm run bake -- --only=browser-use,anchor

Each run prints a comparison table and a live-view/replay link per provider, and saves the full output (including raw agent responses) to results/bakeoff-<timestamp>.json.

How it's wired

• src/schema.ts — the one shared task: a Zod schema (→ JSON Schema for BU/Anchor) + the NL prompt + start URL. Edit the prompt/schema here to retarget.
• src/adapters/*.ts — one per provider, all returning the same RunResult.
  • browserUse.ts / anchor.ts use documented REST (fetch) for reliability.
  • browserbase.ts uses the Stagehand SDK because that's the surface worth feeling.
• src/runner.ts — CLI: select providers, run, tabulate, save artifacts.

Gotchas baked in as comments

• Browser Use structuredOutput: passed as an object; if the API rejects it, switch to JSON.stringify(...) (flagged in browserUse.ts).
• Anchor agent result lands at data.result on newer versions, data on older — handled.
• Browserbase/Stagehand needs an LLM key and pins STAGEHAND_MODEL (default openai/gpt-4o); it has no built-in agent model. Login is driven explicitly with variables to keep creds out of the prompt.
• Login selectors aren't hardcoded (the agents resolve them), but a multi-step, bot-protected login may need prompt / proxy iteration if it blocks.

Safety

Credentials live only in .env (gitignored) and flow through each provider's sensitive-data channel. Use a limited/test account. This drives your own account portal — read-only extraction.