agent-browser

A purpose-built browser for AI agents. Today: a working Chrome extension that captures browser sessions in the agent-trace schema. Tomorrow: a forked Chromium that emits semantic events from the engine itself — see docs/RFC.md.

Status: v0.1. Extension is real and works. The forked-browser plan is in docs/RFC.md.

Quickstart (extension)

1. Open chrome://extensions in Chrome or Edge.
2. Toggle Developer mode on.
3. Click Load unpacked, point at the extension/ directory.
4. Pin the extension icon to the toolbar.
5. Click the icon, enter a task description (e.g., "log in to billing portal"), click Start recording.
6. Browse. The extension captures clicks, form submits, text typing (passwords auto-redacted), navigations, and the most common antibot challenges (Turnstile, reCAPTCHA, hCaptcha, DataDome, PerimeterX).
7. Click Stop & download trace. JSONL file appears in ~/Downloads/agent-browser/<session_id>.jsonl.
8. View it in agent-viewer, parse it with agent-trace, or step through it with agent-debugger.

What the extension captures

Event	Source
session_start / session_end	popup start/stop
nav	chrome.webNavigation.onCommitted (browser-level, not racy)
click	content script, on button / a / [role=button] / input[type=submit|button]
submit	content script, on form
type	content script with 400ms debounce; type=password and autocomplete=*-password redact automatically
antibot_detected	content script DOM heuristics for Cloudflare Turnstile, reCAPTCHA, hCaptcha, DataDome, PerimeterX

What it doesn't capture (yet)

Per the RFC, these need engine-level instrumentation that's out of reach for a Manifest V3 extension:

• Network requests + responses (visible via DevTools / CDP, not via extension API in a complete way)
• Sub-frame navigations on cross-origin iframes
• LLM calls (these come from your agent harness, not the browser — wire agent-trace's Tracer.llm_call() directly)
• Deterministic replay (requires patching the renderer's clock + PRNG)
• Antibot vendors beyond the five heuristics shipped

Why this matters

Agents that run browsers in production fail in a dozen distinct ways. Without structured events at the browser level, every team's failure-classification system is brittle DOM regex. This extension proves the schema works in a real browser; the RFC describes the path from "extension that catches 80% of signals" to "forked Chromium that catches them all."

If you find a failure mode the schema can't represent, that's the highest-priority feedback — open an issue against agent-trace.

The full stack

agent-browser  ◄── captures events from a real browser
      │
      ▼  (writes JSONL trace)
agent-trace    ◄── schema + Python library + CLI
      │
      ├──►  agent-viewer    (browser UI for one trace)
      └──►  agent-debugger  (pdb breakpoints on semantic events)

All four repos are MIT-licensed.

Architecture

extension/
  manifest.json   — MV3 manifest, requires storage/downloads/scripting/tabs
  background.js   — service worker; holds session buffer in chrome.storage.session
                    routes start/stop/event messages; emits nav events from
                    chrome.webNavigation
  content.js      — runs at document_start; captures clicks/submits/typing,
                    auto-redacts password fields, scans for antibot DOM signals
  popup.html/.js  — start/stop UI, live event counter
docs/
  RFC.md          — long-term vision (forked Chromium)

Roadmap (extension)

• Icons. Currently shipping without — Chrome falls back to a default. Quick add.
• Real-time POST to localhost. Optionally stream events to http://localhost:9999/event so a long-running agent harness can ingest them live, not just on stop. Falls back to download if no server.
• CDP companion. A Python lib that attaches via chrome.debugger to capture network events the extension API misses. Same JSONL format.
• CRX distribution. Sign + publish to the Chrome Web Store. Lower priority than features.

For the forked-browser roadmap, see docs/RFC.md.

License

MIT.